2024-09-21T01:26:03.379 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T01:26:03.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:26:03.499 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913386' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIhRXEOThFPD4u+COUG0OvE5haxZk14gROz3DIzIhtKWKlHV3HndXDxidJu/DGyX5Gmp4nEpNNteWkP1NEnwK7I= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqtOzS38pqndK6jx0AwA1m2YAjuPJSL0v370dJUptk+6F/IuM0mEYPCYASOYY37U/oEGfWQB9e+hz54G6NdBn0= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQBuYvOYJiNeoxSVm2AcZ1EwghfzH9soLc9BwHKzIaINrGfTk28e40zzeDGcO1/JXUNS1s3aOrw77zYA72MzWQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T01:26:03.500 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T01:26:03.500 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T01:26:03.500 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T01:26:03.502 INFO:teuthology.task.internal:Checking packages... 2024-09-21T01:26:03.521 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T01:26:03.521 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T01:26:03.522 INFO:teuthology.packaging:ref: None 2024-09-21T01:26:03.522 INFO:teuthology.packaging:tag: None 2024-09-21T01:26:03.522 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:26:03.522 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:26:03.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:26:03.664 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T01:26:03.665 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T01:26:03.676 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T01:26:03.677 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T01:26:03.690 INFO:teuthology.task.internal:Saving configuration 2024-09-21T01:26:03.710 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T01:26:03.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T01:26:03.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:20:22.288087', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIhRXEOThFPD4u+COUG0OvE5haxZk14gROz3DIzIhtKWKlHV3HndXDxidJu/DGyX5Gmp4nEpNNteWkP1NEnwK7I='} 2024-09-21T01:26:03.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:20:22.286313', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqtOzS38pqndK6jx0AwA1m2YAjuPJSL0v370dJUptk+6F/IuM0mEYPCYASOYY37U/oEGfWQB9e+hz54G6NdBn0='} 2024-09-21T01:26:03.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:20:22.284455', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQBuYvOYJiNeoxSVm2AcZ1EwghfzH9soLc9BwHKzIaINrGfTk28e40zzeDGcO1/JXUNS1s3aOrw77zYA72MzWQ='} 2024-09-21T01:26:03.791 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T01:26:03.799 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T01:26:03.799 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T01:26:03.799 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T01:26:03.799 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T01:26:03.890 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd64fcfb5b0>, signals=[15]) 2024-09-21T01:26:03.891 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T01:26:03.902 INFO:teuthology.task.internal:Opening connections... 2024-09-21T01:26:03.902 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:26:03.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:26:03.979 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-09-21T01:26:03.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:26:04.063 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-09-21T01:26:04.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:26:04.144 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T01:26:04.156 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-09-21T01:26:04.175 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-09-21T01:26:04.175 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-09-21T01:26:04.232 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:26:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:26:04.234 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:26:04.234 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-09-21T01:26:04.259 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-09-21T01:26:04.279 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-09-21T01:26:04.279 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:26:04.338 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:26:04.339 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:26:04.339 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:26:04.339 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-09-21T01:26:04.359 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-09-21T01:26:04.377 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-09-21T01:26:04.377 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-09-21T01:26:04.435 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:26:04.436 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:26:04.436 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:26:04.436 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:26:04.436 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:26:04.436 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:26:04.436 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-09-21T01:26:04.459 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T01:26:04.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T01:26:04.486 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T01:26:04.486 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:26:04.488 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:26:04.491 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:26:04.507 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T01:26:04.517 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T01:26:04.517 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:26:04.546 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:26:04.549 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:26:04.649 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T01:26:04.728 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T01:26:04.742 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T01:26:04.742 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T01:26:04.742 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T01:26:04.743 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:26:04.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:26:04.760 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-21T01:26:04.818 INFO:teuthology.orchestra.run.smithi029.stdout:5.14.0-503.el9.x86_64 2024-09-21T01:26:04.818 INFO:teuthology.task.kernel:Running kernel on smithi029: 5.14.0-503.el9.x86_64 2024-09-21T01:26:04.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum install -y kernel 2024-09-21T01:26:08.356 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T01:26:10.508 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-21T01:26:15.902 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-21T01:26:17.397 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-21T01:26:18.369 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-21T01:26:23.943 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T01:26:25.943 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T01:26:25.944 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T01:26:26.051 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-21T01:26:26.067 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-21T01:26:26.067 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-09-21T01:26:26.067 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout:Install 4 Packages 2024-09-21T01:26:26.068 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:26:26.069 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 79 M 2024-09-21T01:26:26.069 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 126 M 2024-09-21T01:26:26.069 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-21T01:26:26.381 INFO:teuthology.orchestra.run.smithi029.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 124 kB/s | 36 kB 00:00 2024-09-21T01:26:40.430 INFO:teuthology.orchestra.run.smithi029.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.1 MB/s | 16 MB 00:14 2024-09-21T01:27:00.988 INFO:teuthology.orchestra.run.smithi029.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 846 kB/s | 29 MB 00:34 2024-09-21T01:27:07.130 INFO:teuthology.orchestra.run.smithi029.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 862 kB/s | 35 MB 00:41 2024-09-21T01:27:07.130 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:27:07.130 INFO:teuthology.orchestra.run.smithi029.stdout:Total 1.9 MB/s | 79 MB 00:41 2024-09-21T01:27:07.839 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-21T01:27:08.242 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-21T01:27:08.242 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-21T01:27:09.210 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-21T01:27:09.210 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-21T01:27:11.755 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-21T01:27:12.487 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:27:12.635 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:27:15.385 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:27:15.579 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:27:20.780 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:27:20.835 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:27:25.964 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:28:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:28:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:28:07.542 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:28:07.542 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:28:07.542 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:28:07.542 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T01:28:08.584 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T01:28:08.585 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T01:28:08.585 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:28:08.585 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-21T01:28:09.033 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:28:09.134 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:28:09.135 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:28:09.135 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T01:28:09.135 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T01:28:09.135 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T01:28:09.135 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T01:28:09.135 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi029.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T01:28:09.135 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:28:09.195 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:28:09.196 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:28:09.196 DEBUG:teuthology.orchestra.run.smithi029:> sudo rpm -qi grub2-tools 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Name : grub2-tools 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Epoch : 1 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Version : 2.06 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Release : 82.el9 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: x86_64 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Group : Unspecified 2024-09-21T01:28:09.227 INFO:teuthology.orchestra.run.smithi029.stdout:Size : 8274670 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:License : GPLv3+ 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Packager : builder@centos.org 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Vendor : CentOS 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Summary : Support tools for GRUB. 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:Description : 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T01:28:09.228 INFO:teuthology.orchestra.run.smithi029.stdout:hardware devices. 2024-09-21T01:28:09.229 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:28:09.229 INFO:teuthology.orchestra.run.smithi029.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T01:28:09.229 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T01:28:09.229 INFO:teuthology.task.kernel:Updating grub on smithi029 to boot 5.14.0-508.el9.x86_64 2024-09-21T01:28:09.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T01:28:10.021 INFO:teuthology.orchestra.run.smithi029.stderr:Generating grub configuration file ... 2024-09-21T01:28:11.555 INFO:teuthology.orchestra.run.smithi029.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T01:28:11.573 INFO:teuthology.orchestra.run.smithi029.stderr:done 2024-09-21T01:28:11.575 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-21T01:28:11.591 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.CcOBgpzHOk 2024-09-21T01:28:11.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CcOBgpzHOk 2024-09-21T01:28:11.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.CcOBgpzHOk 2024-09-21T01:28:11.821 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.CcOBgpzHOk is 6KB 2024-09-21T01:28:11.841 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.CcOBgpzHOk 2024-09-21T01:28:11.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T01:28:11.928 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T01:28:11.929 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T01:28:11.929 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T01:28:11.929 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T01:28:11.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T01:28:12.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo shutdown -r now 2024-09-21T01:28:42.087 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T01:28:42.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-21T01:28:42.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:29:42.090 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T01:29:51.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-21T01:29:51.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:29:58.857 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-21T01:29:59.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-21T01:29:59.289 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T01:29:59.289 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-21T01:29:59.304 INFO:teuthology.orchestra.run.smithi029.stdout:5.14.0-508.el9.x86_64 2024-09-21T01:29:59.304 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T01:29:59.305 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:29:59.305 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T01:30:00.305 DEBUG:teuthology.parallel:result is None 2024-09-21T01:30:00.305 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T01:30:00.314 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T01:30:00.314 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:30:00.317 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:30:00.319 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:30:00.338 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T01:30:00.344 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T01:30:00.351 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T01:30:00.351 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:30:00.371 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:30:00.378 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:30:00.506 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T01:30:00.513 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T01:30:00.513 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:30:00.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:30:00.531 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:30:00.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:30:00.549 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:30:00.551 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:30:00.575 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:30:00.637 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.645 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.655 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.665 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.676 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.743 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:30:00.744 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T01:30:00.757 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T01:30:00.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:30:00.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:30:00.762 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:30:00.791 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T01:30:00.800 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T01:30:00.800 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:30:00.825 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:30:00.834 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:30:00.851 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:00.913 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:00.963 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.080 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:30:01.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:30:01.142 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:01.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:01.241 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.323 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.372 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:30:01.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:30:01.443 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:01.484 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:30:01.549 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.588 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:30:01.635 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:30:01.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:30:01.702 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-09-21T01:30:01.704 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-09-21T01:30:01.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-09-21T01:30:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:30:01.778 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:30:01.780 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:30:02.095 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T01:30:02.102 INFO:teuthology.task.internal:Starting timer... 2024-09-21T01:30:02.103 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T01:30:02.112 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T01:30:02.123 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T01:30:02.123 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2024-09-21T01:30:02.226 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2024-09-21T01:30:02.228 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-09-21T01:30:02.285 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-09-21T01:30:02.287 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-09-21T01:30:02.332 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-09-21T01:30:02.334 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T01:30:02.334 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2024-09-21T01:30:02.354 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2024-09-21T01:30:02.354 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-09-21T01:30:02.380 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-09-21T01:30:02.380 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-09-21T01:30:02.407 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-09-21T01:30:02.407 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi029.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-09-21T01:30:02.408 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:30:02.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:30:02.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:30:02.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:30:02.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:30:02.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:30:02.493 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T01:30:02.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:30:02.527 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:30:02.634 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:30:02.665 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T01:30:02.675 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T01:30:02.677 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T01:30:02.778 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T01:30:02.788 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T01:30:02.789 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi029.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T01:34:12.850 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-09-21T01:34:12.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-21T01:34:12.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:34:12.929 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-21T01:34:13.011 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-21T01:34:13.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-21T01:34:13.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:34:13.093 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-21T01:34:13.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-21T01:34:13.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-21T01:34:13.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:34:13.263 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-21T01:34:13.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-21T01:34:13.356 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T01:34:13.368 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T01:34:13.368 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:34:13.368 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:34:13.371 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:34:13.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:34:13.374 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:34:13.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:34:13.402 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:34:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:34:13.418 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:34:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:34:13.427 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:34:13.443 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:34:13.445 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2024-09-21T01:34:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-09-21T01:34:13.457 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2024-09-21T01:34:13.458 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-09-21T01:34:13.468 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-09-21T01:34:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:34:13.472 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:34:13.479 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-09-21T01:34:13.486 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:34:13.487 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:34:13.494 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:34:13.508 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:34:13.543 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:34:13.546 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:34:13.546 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-21T01:34:13.546 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.546 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.546 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.547 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.547 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:34:13.550 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:34:13.550 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-09-21T01:34:13.550 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.550 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.550 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.551 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.575 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:34:13.578 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:34:13.579 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-21T01:34:13.579 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.579 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.579 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.579 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:34:13.580 INFO:teuthology.run_tasks:Running task install... 2024-09-21T01:34:13.588 DEBUG:teuthology.task.install:project ceph 2024-09-21T01:34:13.588 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T01:34:13.588 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T01:34:13.588 INFO:teuthology.task.install:Using flavor: default 2024-09-21T01:34:13.594 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T01:34:13.594 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T01:34:13.594 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:34:13.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:34:13.597 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:34:13.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:34:13.598 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:34:13.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:34:13.748 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:34:13.748 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:34:13.756 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:34:13.756 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:34:13.759 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:34:13.759 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:34:13.857 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:34:13.857 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:34:13.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:34:13.870 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:34:13.870 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:34:13.871 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:34:13.875 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:34:13.875 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:34:13.875 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:34:13.884 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:34:13.884 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:34:13.898 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:34:13.898 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:34:13.901 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:34:13.901 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:34:13.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:34:13.964 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:34:13.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:34:14.028 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:34:14.041 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:34:14.042 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:34:14.093 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2024-09-21T01:34:14.095 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2024-09-21T01:34:14.106 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-09-21T01:34:14.107 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-09-21T01:34:14.109 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-09-21T01:34:14.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-09-21T01:34:14.331 INFO:teuthology.orchestra.run.smithi190.stdout:45 files removed 2024-09-21T01:34:14.331 INFO:teuthology.orchestra.run.smithi029.stdout:45 files removed 2024-09-21T01:34:14.345 INFO:teuthology.orchestra.run.smithi066.stdout:45 files removed 2024-09-21T01:34:14.359 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:34:14.361 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:34:14.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:34:14.928 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-21T01:34:14.938 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 280 kB/s | 84 kB 00:00 2024-09-21T01:34:14.942 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-21T01:34:15.220 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T01:34:15.231 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T01:34:15.234 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T01:34:15.498 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T01:34:15.507 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T01:34:15.510 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T01:34:16.109 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-21T01:34:16.916 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.3 MB 00:01 2024-09-21T01:34:18.010 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-21T01:34:19.920 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-21T01:34:21.933 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-09-21T01:34:25.577 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-21T01:34:26.533 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.5 MB 00:00 2024-09-21T01:34:27.345 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.5 MB 00:03 2024-09-21T01:34:28.209 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-21T01:34:28.905 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-21T01:34:29.088 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T01:34:29.783 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T01:34:30.022 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T01:34:31.832 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-21T01:34:32.896 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T01:34:34.469 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T01:34:35.432 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T01:34:36.597 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:36.597 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:36.609 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:34:36.610 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:34:36.611 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:34:36.612 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:34:36.613 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:34:36.616 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:34:36.616 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:34:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-21T01:34:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-21T01:34:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-09-21T01:34:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:34:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:34:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:34:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:34:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:34:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:34:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:34:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:34:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:34:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:34:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-21T01:34:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:Install 149 Packages 2024-09-21T01:34:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 8 Packages 2024-09-21T01:34:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:34:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 228 M 2024-09-21T01:34:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-21T01:34:37.455 INFO:teuthology.orchestra.run.smithi190.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-21T01:34:37.638 INFO:teuthology.orchestra.run.smithi190.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.7 MB/s | 873 kB 00:00 2024-09-21T01:34:37.643 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:37.644 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:37.658 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:34:37.659 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:34:37.660 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:34:37.661 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:34:37.663 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:34:37.666 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:34:37.667 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:34:37.730 INFO:teuthology.orchestra.run.smithi190.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T01:34:37.734 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:34:37.749 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:34:37.750 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:34:37.751 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:34:37.752 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:34:37.753 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:34:37.754 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:34:37.755 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:34:37.756 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:34:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:34:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:34:37.759 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout:Install 149 Packages 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 8 Packages 2024-09-21T01:34:37.760 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:34:37.761 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 228 M 2024-09-21T01:34:37.761 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-09-21T01:34:37.814 INFO:teuthology.orchestra.run.smithi190.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T01:34:37.914 INFO:teuthology.orchestra.run.smithi190.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-21T01:34:37.955 INFO:teuthology.orchestra.run.smithi190.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 10 MB/s | 2.3 MB 00:00 2024-09-21T01:34:38.198 INFO:teuthology.orchestra.run.smithi190.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.0 MB 00:00 2024-09-21T01:34:38.465 INFO:teuthology.orchestra.run.smithi190.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T01:34:38.498 INFO:teuthology.orchestra.run.smithi190.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 764 kB/s | 25 kB 00:00 2024-09-21T01:34:38.503 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T01:34:38.918 INFO:teuthology.orchestra.run.smithi066.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-09-21T01:34:38.932 INFO:teuthology.orchestra.run.smithi190.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 14 MB/s | 10 MB 00:00 2024-09-21T01:34:39.090 INFO:teuthology.orchestra.run.smithi190.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T01:34:39.116 INFO:teuthology.orchestra.run.smithi190.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.2 MB/s | 226 kB 00:00 2024-09-21T01:34:39.141 INFO:teuthology.orchestra.run.smithi190.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 542 kB/s | 27 kB 00:00 2024-09-21T01:34:39.166 INFO:teuthology.orchestra.run.smithi190.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 633 kB/s | 32 kB 00:00 2024-09-21T01:34:39.208 INFO:teuthology.orchestra.run.smithi190.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T01:34:39.233 INFO:teuthology.orchestra.run.smithi190.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T01:34:39.258 INFO:teuthology.orchestra.run.smithi190.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T01:34:39.286 INFO:teuthology.orchestra.run.smithi190.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 9.9 MB/s | 507 kB 00:00 2024-09-21T01:34:39.342 INFO:teuthology.orchestra.run.smithi190.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 773 kB/s | 45 kB 00:00 2024-09-21T01:34:39.443 INFO:teuthology.orchestra.run.smithi190.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.4 MB/s | 148 kB 00:00 2024-09-21T01:34:39.526 INFO:teuthology.orchestra.run.smithi190.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T01:34:39.551 INFO:teuthology.orchestra.run.smithi190.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.5 MB/s | 161 kB 00:00 2024-09-21T01:34:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T01:34:39.602 INFO:teuthology.orchestra.run.smithi190.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T01:34:39.627 INFO:teuthology.orchestra.run.smithi190.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T01:34:39.653 INFO:teuthology.orchestra.run.smithi190.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T01:34:39.708 INFO:teuthology.orchestra.run.smithi066.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.1 MB/s | 873 kB 00:00 2024-09-21T01:34:39.711 INFO:teuthology.orchestra.run.smithi190.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T01:34:39.778 INFO:teuthology.orchestra.run.smithi190.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T01:34:39.804 INFO:teuthology.orchestra.run.smithi190.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 372 kB/s | 34 kB 00:00 2024-09-21T01:34:39.817 INFO:teuthology.orchestra.run.smithi066.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.3 MB/s | 148 kB 00:00 2024-09-21T01:34:39.829 INFO:teuthology.orchestra.run.smithi190.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T01:34:40.238 INFO:teuthology.orchestra.run.smithi190.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-21T01:34:40.296 INFO:teuthology.orchestra.run.smithi190.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T01:34:40.372 INFO:teuthology.orchestra.run.smithi190.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 658 kB/s | 49 kB 00:00 2024-09-21T01:34:40.747 INFO:teuthology.orchestra.run.smithi190.stdout:(34/157): ceph-test-19.3.0-5120.g18432218.el9.x 18 MB/s | 40 MB 00:02 2024-09-21T01:34:40.772 INFO:teuthology.orchestra.run.smithi190.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 41 kB/s | 16 kB 00:00 2024-09-21T01:34:40.800 INFO:teuthology.orchestra.run.smithi190.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.3 MB/s | 285 kB 00:00 2024-09-21T01:34:40.850 INFO:teuthology.orchestra.run.smithi190.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.9 MB/s | 784 kB 00:00 2024-09-21T01:34:40.861 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:40.862 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:34:40.876 INFO:teuthology.orchestra.run.smithi029.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:34:40.877 INFO:teuthology.orchestra.run.smithi029.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:34:40.878 INFO:teuthology.orchestra.run.smithi029.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:34:40.879 INFO:teuthology.orchestra.run.smithi029.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:34:40.881 INFO:teuthology.orchestra.run.smithi029.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:34:40.884 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:34:40.885 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:34:40.892 INFO:teuthology.orchestra.run.smithi190.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 200 kB/s | 18 kB 00:00 2024-09-21T01:34:40.951 INFO:teuthology.orchestra.run.smithi066.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-09-21T01:34:40.952 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-21T01:34:40.978 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-21T01:34:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-21T01:34:40.978 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-21T01:34:40.978 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:34:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:34:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:34:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:34:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:34:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:34:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:34:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:34:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:34:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:34:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:34:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout:Installing weak dependencies: 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout:Install 149 Packages 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 8 Packages 2024-09-21T01:34:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:34:40.991 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 228 M 2024-09-21T01:34:40.991 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-21T01:34:41.043 INFO:teuthology.orchestra.run.smithi190.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 11 MB/s | 14 MB 00:01 2024-09-21T01:34:41.109 INFO:teuthology.orchestra.run.smithi190.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 308 kB/s | 80 kB 00:00 2024-09-21T01:34:41.168 INFO:teuthology.orchestra.run.smithi190.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T01:34:41.202 INFO:teuthology.orchestra.run.smithi190.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-21T01:34:41.227 INFO:teuthology.orchestra.run.smithi190.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-21T01:34:41.252 INFO:teuthology.orchestra.run.smithi190.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T01:34:41.286 INFO:teuthology.orchestra.run.smithi190.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T01:34:41.311 INFO:teuthology.orchestra.run.smithi190.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T01:34:41.336 INFO:teuthology.orchestra.run.smithi190.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T01:34:41.386 INFO:teuthology.orchestra.run.smithi190.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-21T01:34:41.409 INFO:teuthology.orchestra.run.smithi066.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 1.9 MB/s | 5.5 MB 00:02 2024-09-21T01:34:41.467 INFO:teuthology.orchestra.run.smithi066.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-09-21T01:34:41.487 INFO:teuthology.orchestra.run.smithi190.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-21T01:34:41.529 INFO:teuthology.orchestra.run.smithi190.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-21T01:34:41.550 INFO:teuthology.orchestra.run.smithi029.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-21T01:34:41.554 INFO:teuthology.orchestra.run.smithi190.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-21T01:34:41.579 INFO:teuthology.orchestra.run.smithi190.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 107 kB 00:00 2024-09-21T01:34:41.605 INFO:teuthology.orchestra.run.smithi190.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-21T01:34:41.630 INFO:teuthology.orchestra.run.smithi190.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-21T01:34:41.672 INFO:teuthology.orchestra.run.smithi190.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 271 kB/s | 18 kB 00:00 2024-09-21T01:34:41.706 INFO:teuthology.orchestra.run.smithi190.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 147 kB/s | 11 kB 00:00 2024-09-21T01:34:41.748 INFO:teuthology.orchestra.run.smithi190.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T01:34:41.773 INFO:teuthology.orchestra.run.smithi190.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-21T01:34:41.848 INFO:teuthology.orchestra.run.smithi190.stdout:(59/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-21T01:34:41.873 INFO:teuthology.orchestra.run.smithi190.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-21T01:34:41.899 INFO:teuthology.orchestra.run.smithi190.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-21T01:34:41.941 INFO:teuthology.orchestra.run.smithi190.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-21T01:34:41.966 INFO:teuthology.orchestra.run.smithi190.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-09-21T01:34:42.000 INFO:teuthology.orchestra.run.smithi190.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-21T01:34:42.025 INFO:teuthology.orchestra.run.smithi190.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T01:34:42.050 INFO:teuthology.orchestra.run.smithi190.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T01:34:42.075 INFO:teuthology.orchestra.run.smithi190.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-21T01:34:42.167 INFO:teuthology.orchestra.run.smithi190.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T01:34:42.209 INFO:teuthology.orchestra.run.smithi190.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T01:34:42.225 INFO:teuthology.orchestra.run.smithi029.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 873 kB 00:00 2024-09-21T01:34:42.276 INFO:teuthology.orchestra.run.smithi190.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-21T01:34:42.301 INFO:teuthology.orchestra.run.smithi190.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 6.6 MB/s | 910 kB 00:00 2024-09-21T01:34:42.317 INFO:teuthology.orchestra.run.smithi029.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T01:34:42.327 INFO:teuthology.orchestra.run.smithi190.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T01:34:42.377 INFO:teuthology.orchestra.run.smithi190.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T01:34:42.410 INFO:teuthology.orchestra.run.smithi190.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-21T01:34:42.451 INFO:teuthology.orchestra.run.smithi066.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 4.8 MB/s | 5.0 MB 00:01 2024-09-21T01:34:42.478 INFO:teuthology.orchestra.run.smithi190.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-09-21T01:34:42.511 INFO:teuthology.orchestra.run.smithi190.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-21T01:34:42.536 INFO:teuthology.orchestra.run.smithi190.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-21T01:34:42.587 INFO:teuthology.orchestra.run.smithi190.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-21T01:34:42.620 INFO:teuthology.orchestra.run.smithi190.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 306 kB/s | 26 kB 00:00 2024-09-21T01:34:42.712 INFO:teuthology.orchestra.run.smithi190.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-21T01:34:42.738 INFO:teuthology.orchestra.run.smithi190.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 127 kB/s | 19 kB 00:00 2024-09-21T01:34:42.763 INFO:teuthology.orchestra.run.smithi190.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-21T01:34:42.788 INFO:teuthology.orchestra.run.smithi190.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-21T01:34:42.826 INFO:teuthology.orchestra.run.smithi029.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 4.5 MB/s | 2.3 MB 00:00 2024-09-21T01:34:42.846 INFO:teuthology.orchestra.run.smithi190.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-21T01:34:42.872 INFO:teuthology.orchestra.run.smithi190.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T01:34:42.947 INFO:teuthology.orchestra.run.smithi190.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-21T01:34:42.973 INFO:teuthology.orchestra.run.smithi190.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 614 kB/s | 77 kB 00:00 2024-09-21T01:34:42.998 INFO:teuthology.orchestra.run.smithi190.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-21T01:34:43.002 INFO:teuthology.orchestra.run.smithi029.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 5.5 MB 00:01 2024-09-21T01:34:43.031 INFO:teuthology.orchestra.run.smithi190.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-21T01:34:43.057 INFO:teuthology.orchestra.run.smithi190.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-21T01:34:43.090 INFO:teuthology.orchestra.run.smithi190.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-21T01:34:43.116 INFO:teuthology.orchestra.run.smithi190.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-21T01:34:43.119 INFO:teuthology.orchestra.run.smithi029.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 5.2 MB/s | 1.5 MB 00:00 2024-09-21T01:34:43.141 INFO:teuthology.orchestra.run.smithi190.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-09-21T01:34:43.208 INFO:teuthology.orchestra.run.smithi190.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-09-21T01:34:43.233 INFO:teuthology.orchestra.run.smithi190.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-09-21T01:34:43.292 INFO:teuthology.orchestra.run.smithi190.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-21T01:34:43.317 INFO:teuthology.orchestra.run.smithi190.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 429 kB/s | 36 kB 00:00 2024-09-21T01:34:43.467 INFO:teuthology.orchestra.run.smithi190.stdout:(98/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-21T01:34:43.486 INFO:teuthology.orchestra.run.smithi029.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-21T01:34:43.493 INFO:teuthology.orchestra.run.smithi190.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 236 kB/s | 47 kB 00:00 2024-09-21T01:34:43.518 INFO:teuthology.orchestra.run.smithi190.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-21T01:34:43.543 INFO:teuthology.orchestra.run.smithi190.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-09-21T01:34:43.569 INFO:teuthology.orchestra.run.smithi190.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T01:34:43.594 INFO:teuthology.orchestra.run.smithi190.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 844 kB/s | 64 kB 00:00 2024-09-21T01:34:43.619 INFO:teuthology.orchestra.run.smithi190.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-09-21T01:34:43.669 INFO:teuthology.orchestra.run.smithi190.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T01:34:43.703 INFO:teuthology.orchestra.run.smithi190.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T01:34:43.728 INFO:teuthology.orchestra.run.smithi190.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-21T01:34:43.762 INFO:teuthology.orchestra.run.smithi190.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-21T01:34:43.787 INFO:teuthology.orchestra.run.smithi190.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-21T01:34:43.812 INFO:teuthology.orchestra.run.smithi190.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T01:34:43.837 INFO:teuthology.orchestra.run.smithi190.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T01:34:43.854 INFO:teuthology.orchestra.run.smithi190.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 8.9 MB/s | 151 kB 00:00 2024-09-21T01:34:43.905 INFO:teuthology.orchestra.run.smithi190.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-21T01:34:43.947 INFO:teuthology.orchestra.run.smithi190.stdout:(114/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-21T01:34:43.989 INFO:teuthology.orchestra.run.smithi190.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-21T01:34:44.010 INFO:teuthology.orchestra.run.smithi066.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 3.7 MB/s | 20 MB 00:05 2024-09-21T01:34:44.031 INFO:teuthology.orchestra.run.smithi190.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 811 kB/s | 32 kB 00:00 2024-09-21T01:34:44.056 INFO:teuthology.orchestra.run.smithi190.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-09-21T01:34:44.060 INFO:teuthology.orchestra.run.smithi066.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 509 kB/s | 25 kB 00:00 2024-09-21T01:34:44.081 INFO:teuthology.orchestra.run.smithi190.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T01:34:44.115 INFO:teuthology.orchestra.run.smithi190.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T01:34:44.128 INFO:teuthology.orchestra.run.smithi029.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 7.4 MB/s | 20 MB 00:02 2024-09-21T01:34:44.140 INFO:teuthology.orchestra.run.smithi190.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T01:34:44.166 INFO:teuthology.orchestra.run.smithi190.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T01:34:44.179 INFO:teuthology.orchestra.run.smithi029.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 503 kB/s | 25 kB 00:00 2024-09-21T01:34:44.199 INFO:teuthology.orchestra.run.smithi190.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-21T01:34:44.258 INFO:teuthology.orchestra.run.smithi190.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-09-21T01:34:44.283 INFO:teuthology.orchestra.run.smithi190.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-21T01:34:44.308 INFO:teuthology.orchestra.run.smithi190.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-21T01:34:44.325 INFO:teuthology.orchestra.run.smithi190.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-09-21T01:34:44.342 INFO:teuthology.orchestra.run.smithi190.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-09-21T01:34:44.359 INFO:teuthology.orchestra.run.smithi190.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 577 kB/s | 19 kB 00:00 2024-09-21T01:34:44.376 INFO:teuthology.orchestra.run.smithi190.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 585 kB/s | 20 kB 00:00 2024-09-21T01:34:44.401 INFO:teuthology.orchestra.run.smithi190.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:34:44.435 INFO:teuthology.orchestra.run.smithi190.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T01:34:44.436 INFO:teuthology.orchestra.run.smithi066.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 5.3 MB/s | 10 MB 00:01 2024-09-21T01:34:44.454 INFO:teuthology.orchestra.run.smithi029.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 11 MB/s | 10 MB 00:00 2024-09-21T01:34:44.460 INFO:teuthology.orchestra.run.smithi190.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T01:34:44.494 INFO:teuthology.orchestra.run.smithi190.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-21T01:34:44.515 INFO:teuthology.orchestra.run.smithi029.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.7 MB/s | 226 kB 00:00 2024-09-21T01:34:44.518 INFO:teuthology.orchestra.run.smithi066.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.7 MB/s | 226 kB 00:00 2024-09-21T01:34:44.528 INFO:teuthology.orchestra.run.smithi190.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T01:34:44.553 INFO:teuthology.orchestra.run.smithi190.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-21T01:34:44.569 INFO:teuthology.orchestra.run.smithi066.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 542 kB/s | 27 kB 00:00 2024-09-21T01:34:44.578 INFO:teuthology.orchestra.run.smithi190.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T01:34:44.603 INFO:teuthology.orchestra.run.smithi190.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T01:34:44.611 INFO:teuthology.orchestra.run.smithi066.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 762 kB/s | 32 kB 00:00 2024-09-21T01:34:44.629 INFO:teuthology.orchestra.run.smithi190.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T01:34:44.679 INFO:teuthology.orchestra.run.smithi190.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-21T01:34:44.705 INFO:teuthology.orchestra.run.smithi190.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-09-21T01:34:44.730 INFO:teuthology.orchestra.run.smithi190.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T01:34:44.753 INFO:teuthology.orchestra.run.smithi066.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.3 MB/s | 766 kB 00:00 2024-09-21T01:34:44.755 INFO:teuthology.orchestra.run.smithi190.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T01:34:44.772 INFO:teuthology.orchestra.run.smithi190.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T01:34:44.797 INFO:teuthology.orchestra.run.smithi190.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-21T01:34:44.803 INFO:teuthology.orchestra.run.smithi066.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T01:34:44.823 INFO:teuthology.orchestra.run.smithi190.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-21T01:34:44.823 INFO:teuthology.orchestra.run.smithi029.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 10 MB/s | 18 MB 00:01 2024-09-21T01:34:44.845 INFO:teuthology.orchestra.run.smithi066.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 3.0 MB/s | 127 kB 00:00 2024-09-21T01:34:44.848 INFO:teuthology.orchestra.run.smithi190.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T01:34:44.849 INFO:teuthology.orchestra.run.smithi029.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 81 kB/s | 27 kB 00:00 2024-09-21T01:34:44.873 INFO:teuthology.orchestra.run.smithi190.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-21T01:34:44.875 INFO:teuthology.orchestra.run.smithi029.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 630 kB/s | 32 kB 00:00 2024-09-21T01:34:44.907 INFO:teuthology.orchestra.run.smithi190.stdout:(148/157): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T01:34:44.950 INFO:teuthology.orchestra.run.smithi029.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T01:34:44.965 INFO:teuthology.orchestra.run.smithi190.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T01:34:44.992 INFO:teuthology.orchestra.run.smithi029.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.2 MB/s | 766 kB 00:00 2024-09-21T01:34:45.026 INFO:teuthology.orchestra.run.smithi029.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.6 MB/s | 127 kB 00:00 2024-09-21T01:34:45.070 INFO:teuthology.orchestra.run.smithi066.stdout:(17/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 18 MB 00:03 2024-09-21T01:34:45.094 INFO:teuthology.orchestra.run.smithi029.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.9 MB/s | 507 kB 00:00 2024-09-21T01:34:45.095 INFO:teuthology.orchestra.run.smithi066.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 2.0 MB/s | 507 kB 00:00 2024-09-21T01:34:45.141 INFO:teuthology.orchestra.run.smithi190.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-09-21T01:34:45.169 INFO:teuthology.orchestra.run.smithi029.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 600 kB/s | 45 kB 00:00 2024-09-21T01:34:45.171 INFO:teuthology.orchestra.run.smithi066.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 601 kB/s | 45 kB 00:00 2024-09-21T01:34:45.216 INFO:teuthology.orchestra.run.smithi190.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-09-21T01:34:45.258 INFO:teuthology.orchestra.run.smithi190.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 986 kB/s | 41 kB 00:00 2024-09-21T01:34:45.261 INFO:teuthology.orchestra.run.smithi029.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.6 MB/s | 148 kB 00:00 2024-09-21T01:34:45.288 INFO:teuthology.orchestra.run.smithi066.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.2 MB/s | 148 kB 00:00 2024-09-21T01:34:45.317 INFO:teuthology.orchestra.run.smithi190.stdout:(153/157): librados2-19.3.0-5120.g18432218.el9. 7.6 MB/s | 3.4 MB 00:00 2024-09-21T01:34:45.337 INFO:teuthology.orchestra.run.smithi029.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T01:34:45.342 INFO:teuthology.orchestra.run.smithi190.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-21T01:34:45.371 INFO:teuthology.orchestra.run.smithi066.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T01:34:45.421 INFO:teuthology.orchestra.run.smithi029.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T01:34:45.426 INFO:teuthology.orchestra.run.smithi190.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-21T01:34:45.513 INFO:teuthology.orchestra.run.smithi066.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 2.2 MB/s | 321 kB 00:00 2024-09-21T01:34:45.513 INFO:teuthology.orchestra.run.smithi029.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.3 MB/s | 307 kB 00:00 2024-09-21T01:34:45.600 INFO:teuthology.orchestra.run.smithi190.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.8 MB/s | 1.4 MB 00:00 2024-09-21T01:34:45.613 INFO:teuthology.orchestra.run.smithi066.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.0 MB/s | 307 kB 00:00 2024-09-21T01:34:45.639 INFO:teuthology.orchestra.run.smithi029.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 8.6 MB/s | 5.3 MB 00:00 2024-09-21T01:34:45.664 INFO:teuthology.orchestra.run.smithi066.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T01:34:45.674 INFO:teuthology.orchestra.run.smithi029.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 618 kB/s | 99 kB 00:00 2024-09-21T01:34:45.700 INFO:teuthology.orchestra.run.smithi029.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T01:34:45.722 INFO:teuthology.orchestra.run.smithi066.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 86 kB 00:00 2024-09-21T01:34:45.751 INFO:teuthology.orchestra.run.smithi029.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T01:34:45.834 INFO:teuthology.orchestra.run.smithi029.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 410 kB/s | 34 kB 00:00 2024-09-21T01:34:45.927 INFO:teuthology.orchestra.run.smithi029.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.6 MB/s | 154 kB 00:00 2024-09-21T01:34:46.151 INFO:teuthology.orchestra.run.smithi190.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 2.5 MB/s | 3.2 MB 00:01 2024-09-21T01:34:46.158 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:34:46.158 INFO:teuthology.orchestra.run.smithi190.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T01:34:46.300 INFO:teuthology.orchestra.run.smithi066.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 5.4 MB/s | 3.1 MB 00:00 2024-09-21T01:34:46.350 INFO:teuthology.orchestra.run.smithi066.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T01:34:46.392 INFO:teuthology.orchestra.run.smithi066.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 823 kB/s | 34 kB 00:00 2024-09-21T01:34:46.467 INFO:teuthology.orchestra.run.smithi066.stdout:(29/157): librgw2-19.3.0-5120.g18432218.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-09-21T01:34:46.493 INFO:teuthology.orchestra.run.smithi066.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.5 MB/s | 154 kB 00:00 2024-09-21T01:34:46.611 INFO:teuthology.orchestra.run.smithi029.stdout:(30/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T01:34:46.694 INFO:teuthology.orchestra.run.smithi029.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 3.0 MB/s | 3.1 MB 00:01 2024-09-21T01:34:46.845 INFO:teuthology.orchestra.run.smithi029.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 1.9 MB/s | 292 kB 00:00 2024-09-21T01:34:47.004 INFO:teuthology.orchestra.run.smithi029.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 19 MB/s | 7.4 MB 00:00 2024-09-21T01:34:47.029 INFO:teuthology.orchestra.run.smithi029.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 268 kB/s | 49 kB 00:00 2024-09-21T01:34:47.055 INFO:teuthology.orchestra.run.smithi029.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 326 kB/s | 16 kB 00:00 2024-09-21T01:34:47.089 INFO:teuthology.orchestra.run.smithi029.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.7 MB/s | 285 kB 00:00 2024-09-21T01:34:47.131 INFO:teuthology.orchestra.run.smithi029.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T01:34:47.156 INFO:teuthology.orchestra.run.smithi029.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 273 kB/s | 18 kB 00:00 2024-09-21T01:34:47.405 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-21T01:34:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 7.4 MB/s | 7.4 MB 00:00 2024-09-21T01:34:47.490 INFO:teuthology.orchestra.run.smithi029.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 9.1 MB/s | 14 MB 00:01 2024-09-21T01:34:47.526 INFO:teuthology.orchestra.run.smithi029.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 202 kB/s | 80 kB 00:00 2024-09-21T01:34:47.558 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-21T01:34:47.558 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-21T01:34:47.559 INFO:teuthology.orchestra.run.smithi066.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.9 MB/s | 292 kB 00:00 2024-09-21T01:34:47.576 INFO:teuthology.orchestra.run.smithi029.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T01:34:47.601 INFO:teuthology.orchestra.run.smithi066.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 1.2 MB/s | 49 kB 00:00 2024-09-21T01:34:47.635 INFO:teuthology.orchestra.run.smithi066.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 494 kB/s | 16 kB 00:00 2024-09-21T01:34:47.677 INFO:teuthology.orchestra.run.smithi029.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 801 kB 00:00 2024-09-21T01:34:47.710 INFO:teuthology.orchestra.run.smithi066.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T01:34:47.711 INFO:teuthology.orchestra.run.smithi029.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T01:34:47.745 INFO:teuthology.orchestra.run.smithi029.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 106 kB 00:00 2024-09-21T01:34:47.779 INFO:teuthology.orchestra.run.smithi029.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T01:34:47.827 INFO:teuthology.orchestra.run.smithi066.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 6.6 MB/s | 784 kB 00:00 2024-09-21T01:34:47.837 INFO:teuthology.orchestra.run.smithi029.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-09-21T01:34:47.863 INFO:teuthology.orchestra.run.smithi029.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T01:34:47.869 INFO:teuthology.orchestra.run.smithi066.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 445 kB/s | 18 kB 00:00 2024-09-21T01:34:47.897 INFO:teuthology.orchestra.run.smithi029.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T01:34:47.978 INFO:teuthology.orchestra.run.smithi066.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-21T01:34:48.039 INFO:teuthology.orchestra.run.smithi029.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-21T01:34:48.069 INFO:teuthology.orchestra.run.smithi066.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 8.6 MB/s | 801 kB 00:00 2024-09-21T01:34:48.098 INFO:teuthology.orchestra.run.smithi029.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-21T01:34:48.128 INFO:teuthology.orchestra.run.smithi066.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T01:34:48.140 INFO:teuthology.orchestra.run.smithi029.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-21T01:34:48.170 INFO:teuthology.orchestra.run.smithi066.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-09-21T01:34:48.208 INFO:teuthology.orchestra.run.smithi029.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T01:34:48.233 INFO:teuthology.orchestra.run.smithi029.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 354 kB/s | 33 kB 00:00 2024-09-21T01:34:48.237 INFO:teuthology.orchestra.run.smithi066.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-09-21T01:34:48.267 INFO:teuthology.orchestra.run.smithi029.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-09-21T01:34:48.279 INFO:teuthology.orchestra.run.smithi066.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T01:34:48.294 INFO:teuthology.orchestra.run.smithi029.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 300 kB/s | 18 kB 00:00 2024-09-21T01:34:48.319 INFO:teuthology.orchestra.run.smithi029.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 213 kB/s | 11 kB 00:00 2024-09-21T01:34:48.321 INFO:teuthology.orchestra.run.smithi066.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-21T01:34:48.363 INFO:teuthology.orchestra.run.smithi066.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-21T01:34:48.378 INFO:teuthology.orchestra.run.smithi029.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T01:34:48.404 INFO:teuthology.orchestra.run.smithi029.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T01:34:48.405 INFO:teuthology.orchestra.run.smithi066.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-09-21T01:34:48.429 INFO:teuthology.orchestra.run.smithi029.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T01:34:48.580 INFO:teuthology.orchestra.run.smithi029.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-21T01:34:48.639 INFO:teuthology.orchestra.run.smithi029.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-21T01:34:48.690 INFO:teuthology.orchestra.run.smithi029.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-21T01:34:48.739 INFO:teuthology.orchestra.run.smithi066.stdout:(47/157): ceph-mgr-dashboard-19.3.0-5120.g18432 6.2 MB/s | 14 MB 00:02 2024-09-21T01:34:48.740 INFO:teuthology.orchestra.run.smithi029.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-21T01:34:48.781 INFO:teuthology.orchestra.run.smithi066.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-09-21T01:34:48.816 INFO:teuthology.orchestra.run.smithi029.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T01:34:48.849 INFO:teuthology.orchestra.run.smithi029.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T01:34:48.875 INFO:teuthology.orchestra.run.smithi029.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 652 kB/s | 38 kB 00:00 2024-09-21T01:34:48.964 INFO:teuthology.orchestra.run.smithi066.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-09-21T01:34:49.009 INFO:teuthology.orchestra.run.smithi029.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-09-21T01:34:49.014 INFO:teuthology.orchestra.run.smithi066.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-21T01:34:49.081 INFO:teuthology.orchestra.run.smithi066.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T01:34:49.363 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-21T01:34:49.363 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-21T01:34:49.385 INFO:teuthology.orchestra.run.smithi029.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-09-21T01:34:49.465 INFO:teuthology.orchestra.run.smithi066.stdout:(52/157): ceph-test-19.3.0-5120.g18432218.el9.x 7.4 MB/s | 40 MB 00:05 2024-09-21T01:34:49.490 INFO:teuthology.orchestra.run.smithi066.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 80 kB/s | 33 kB 00:00 2024-09-21T01:34:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 626 kB/s | 313 kB 00:00 2024-09-21T01:34:49.541 INFO:teuthology.orchestra.run.smithi066.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T01:34:49.566 INFO:teuthology.orchestra.run.smithi066.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T01:34:49.593 INFO:teuthology.orchestra.run.smithi029.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-21T01:34:49.625 INFO:teuthology.orchestra.run.smithi066.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T01:34:49.650 INFO:teuthology.orchestra.run.smithi066.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T01:34:49.654 INFO:teuthology.orchestra.run.smithi029.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.2 MB/s | 137 kB 00:00 2024-09-21T01:34:49.675 INFO:teuthology.orchestra.run.smithi066.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-21T01:34:49.779 INFO:teuthology.orchestra.run.smithi029.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-09-21T01:34:49.817 INFO:teuthology.orchestra.run.smithi066.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-21T01:34:49.843 INFO:teuthology.orchestra.run.smithi066.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-21T01:34:49.868 INFO:teuthology.orchestra.run.smithi066.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-21T01:34:49.893 INFO:teuthology.orchestra.run.smithi066.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T01:34:49.960 INFO:teuthology.orchestra.run.smithi066.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T01:34:49.985 INFO:teuthology.orchestra.run.smithi066.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-21T01:34:49.988 INFO:teuthology.orchestra.run.smithi029.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-21T01:34:50.002 INFO:teuthology.orchestra.run.smithi066.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 922 kB/s | 38 kB 00:00 2024-09-21T01:34:50.056 INFO:teuthology.orchestra.run.smithi029.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T01:34:50.323 INFO:teuthology.orchestra.run.smithi029.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-21T01:34:50.382 INFO:teuthology.orchestra.run.smithi029.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T01:34:50.394 INFO:teuthology.orchestra.run.smithi066.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-21T01:34:50.474 INFO:teuthology.orchestra.run.smithi029.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T01:34:50.566 INFO:teuthology.orchestra.run.smithi029.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-21T01:34:50.625 INFO:teuthology.orchestra.run.smithi029.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-21T01:34:50.684 INFO:teuthology.orchestra.run.smithi029.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-21T01:34:50.809 INFO:teuthology.orchestra.run.smithi029.stdout:(80/157): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-21T01:34:50.835 INFO:teuthology.orchestra.run.smithi029.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-21T01:34:50.869 INFO:teuthology.orchestra.run.smithi066.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-21T01:34:50.877 INFO:teuthology.orchestra.run.smithi029.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 515 kB/s | 35 kB 00:00 2024-09-21T01:34:50.978 INFO:teuthology.orchestra.run.smithi029.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T01:34:50.978 INFO:teuthology.orchestra.run.smithi066.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-09-21T01:34:51.038 INFO:teuthology.orchestra.run.smithi029.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.6 MB/s | 222 kB 00:00 2024-09-21T01:34:51.070 INFO:teuthology.orchestra.run.smithi066.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-21T01:34:51.095 INFO:teuthology.orchestra.run.smithi066.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 3.9 MB/s | 910 kB 00:00 2024-09-21T01:34:51.120 INFO:teuthology.orchestra.run.smithi066.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 965 kB/s | 137 kB 00:00 2024-09-21T01:34:51.156 INFO:teuthology.orchestra.run.smithi029.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-09-21T01:34:51.181 INFO:teuthology.orchestra.run.smithi029.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 541 kB/s | 77 kB 00:00 2024-09-21T01:34:51.207 INFO:teuthology.orchestra.run.smithi029.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 834 kB/s | 42 kB 00:00 2024-09-21T01:34:51.266 INFO:teuthology.orchestra.run.smithi029.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T01:34:51.271 INFO:teuthology.orchestra.run.smithi066.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 582 kB/s | 87 kB 00:00 2024-09-21T01:34:51.299 INFO:teuthology.orchestra.run.smithi029.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-21T01:34:51.342 INFO:teuthology.orchestra.run.smithi029.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-21T01:34:51.367 INFO:teuthology.orchestra.run.smithi029.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 756 kB/s | 51 kB 00:00 2024-09-21T01:34:51.393 INFO:teuthology.orchestra.run.smithi029.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T01:34:51.443 INFO:teuthology.orchestra.run.smithi029.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-21T01:34:51.446 INFO:teuthology.orchestra.run.smithi066.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-21T01:34:51.502 INFO:teuthology.orchestra.run.smithi029.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-21T01:34:51.505 INFO:teuthology.orchestra.run.smithi066.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T01:34:51.570 INFO:teuthology.orchestra.run.smithi029.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T01:34:51.597 INFO:teuthology.orchestra.run.smithi066.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T01:34:51.628 INFO:teuthology.orchestra.run.smithi029.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-21T01:34:51.687 INFO:teuthology.orchestra.run.smithi029.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-21T01:34:51.738 INFO:teuthology.orchestra.run.smithi066.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-21T01:34:51.754 INFO:teuthology.orchestra.run.smithi029.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-21T01:34:51.789 INFO:teuthology.orchestra.run.smithi066.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-21T01:34:51.815 INFO:teuthology.orchestra.run.smithi029.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 961 kB/s | 58 kB 00:00 2024-09-21T01:34:51.839 INFO:teuthology.orchestra.run.smithi066.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T01:34:51.898 INFO:teuthology.orchestra.run.smithi066.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T01:34:51.924 INFO:teuthology.orchestra.run.smithi029.stdout:(100/157): python3-numpy-1.20.1-5.el9.x86_64.rp 4.7 MB/s | 5.1 MB 00:01 2024-09-21T01:34:51.948 INFO:teuthology.orchestra.run.smithi066.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-21T01:34:51.958 INFO:teuthology.orchestra.run.smithi029.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 672 kB/s | 96 kB 00:00 2024-09-21T01:34:51.983 INFO:teuthology.orchestra.run.smithi029.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T01:34:52.017 INFO:teuthology.orchestra.run.smithi029.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 760 kB/s | 45 kB 00:00 2024-09-21T01:34:52.107 INFO:teuthology.orchestra.run.smithi029.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 181 kB/s | 22 kB 00:00 2024-09-21T01:34:52.190 INFO:teuthology.orchestra.run.smithi029.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-21T01:34:52.241 INFO:teuthology.orchestra.run.smithi029.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 6.0 MB/s | 308 kB 00:00 2024-09-21T01:34:52.267 INFO:teuthology.orchestra.run.smithi029.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 768 kB/s | 19 kB 00:00 2024-09-21T01:34:52.376 INFO:teuthology.orchestra.run.smithi029.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-21T01:34:52.423 INFO:teuthology.orchestra.run.smithi066.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-09-21T01:34:52.426 INFO:teuthology.orchestra.run.smithi029.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-09-21T01:34:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:(110/157): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-21T01:34:52.540 INFO:teuthology.orchestra.run.smithi066.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T01:34:52.552 INFO:teuthology.orchestra.run.smithi029.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 387 kB/s | 49 kB 00:00 2024-09-21T01:34:52.578 INFO:teuthology.orchestra.run.smithi029.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T01:34:52.612 INFO:teuthology.orchestra.run.smithi029.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T01:34:52.649 INFO:teuthology.orchestra.run.smithi066.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-21T01:34:52.662 INFO:teuthology.orchestra.run.smithi029.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T01:34:52.705 INFO:teuthology.orchestra.run.smithi029.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-21T01:34:52.730 INFO:teuthology.orchestra.run.smithi029.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-09-21T01:34:52.741 INFO:teuthology.orchestra.run.smithi066.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-21T01:34:52.757 INFO:teuthology.orchestra.run.smithi029.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 528 kB/s | 14 kB 00:00 2024-09-21T01:34:52.766 INFO:teuthology.orchestra.run.smithi066.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-09-21T01:34:52.782 INFO:teuthology.orchestra.run.smithi029.stdout:(118/157): python3-cachetools-4.2.4-1.el9.noarc 615 kB/s | 32 kB 00:00 2024-09-21T01:34:52.791 INFO:teuthology.orchestra.run.smithi066.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-21T01:34:52.816 INFO:teuthology.orchestra.run.smithi029.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-09-21T01:34:52.858 INFO:teuthology.orchestra.run.smithi066.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-21T01:34:52.859 INFO:teuthology.orchestra.run.smithi029.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-21T01:34:52.893 INFO:teuthology.orchestra.run.smithi029.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T01:34:52.927 INFO:teuthology.orchestra.run.smithi029.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-21T01:34:52.933 INFO:teuthology.orchestra.run.smithi066.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-09-21T01:34:52.960 INFO:teuthology.orchestra.run.smithi029.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T01:34:52.994 INFO:teuthology.orchestra.run.smithi029.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 319 kB/s | 11 kB 00:00 2024-09-21T01:34:53.000 INFO:teuthology.orchestra.run.smithi066.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-21T01:34:53.026 INFO:teuthology.orchestra.run.smithi066.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 553 kB/s | 51 kB 00:00 2024-09-21T01:34:53.028 INFO:teuthology.orchestra.run.smithi029.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 529 kB/s | 18 kB 00:00 2024-09-21T01:34:53.062 INFO:teuthology.orchestra.run.smithi029.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-09-21T01:34:53.076 INFO:teuthology.orchestra.run.smithi066.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-09-21T01:34:53.096 INFO:teuthology.orchestra.run.smithi029.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 577 kB/s | 19 kB 00:00 2024-09-21T01:34:53.130 INFO:teuthology.orchestra.run.smithi029.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-09-21T01:34:53.151 INFO:teuthology.orchestra.run.smithi066.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-09-21T01:34:53.206 INFO:teuthology.orchestra.run.smithi029.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.5 MB/s | 2.0 MB 00:00 2024-09-21T01:34:53.235 INFO:teuthology.orchestra.run.smithi066.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-09-21T01:34:53.240 INFO:teuthology.orchestra.run.smithi029.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T01:34:53.283 INFO:teuthology.orchestra.run.smithi029.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-21T01:34:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T01:34:53.309 INFO:teuthology.orchestra.run.smithi029.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T01:34:53.344 INFO:teuthology.orchestra.run.smithi066.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-21T01:34:53.351 INFO:teuthology.orchestra.run.smithi029.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 4.6 MB/s | 1.0 MB 00:00 2024-09-21T01:34:53.377 INFO:teuthology.orchestra.run.smithi029.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 652 kB/s | 16 kB 00:00 2024-09-21T01:34:53.394 INFO:teuthology.orchestra.run.smithi066.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-21T01:34:53.411 INFO:teuthology.orchestra.run.smithi029.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-21T01:34:53.445 INFO:teuthology.orchestra.run.smithi029.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-21T01:34:53.470 INFO:teuthology.orchestra.run.smithi029.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 479 kB/s | 28 kB 00:00 2024-09-21T01:34:53.486 INFO:teuthology.orchestra.run.smithi066.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-21T01:34:53.495 INFO:teuthology.orchestra.run.smithi029.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 609 kB/s | 31 kB 00:00 2024-09-21T01:34:53.521 INFO:teuthology.orchestra.run.smithi029.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:34:53.536 INFO:teuthology.orchestra.run.smithi066.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T01:34:53.547 INFO:teuthology.orchestra.run.smithi029.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T01:34:53.581 INFO:teuthology.orchestra.run.smithi029.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-21T01:34:53.615 INFO:teuthology.orchestra.run.smithi029.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T01:34:53.620 INFO:teuthology.orchestra.run.smithi066.stdout:(100/157): python3-numpy-1.20.1-5.el9.x86_64.rp 3.0 MB/s | 5.1 MB 00:01 2024-09-21T01:34:53.645 INFO:teuthology.orchestra.run.smithi066.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-21T01:34:53.648 INFO:teuthology.orchestra.run.smithi029.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-21T01:34:53.671 INFO:teuthology.orchestra.run.smithi066.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T01:34:53.682 INFO:teuthology.orchestra.run.smithi029.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T01:34:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T01:34:53.716 INFO:teuthology.orchestra.run.smithi029.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 597 kB/s | 20 kB 00:00 2024-09-21T01:34:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-09-21T01:34:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T01:34:53.830 INFO:teuthology.orchestra.run.smithi066.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T01:34:53.847 INFO:teuthology.orchestra.run.smithi066.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T01:34:53.897 INFO:teuthology.orchestra.run.smithi066.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-09-21T01:34:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 628 kB/s | 26 kB 00:00 2024-09-21T01:34:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T01:34:53.989 INFO:teuthology.orchestra.run.smithi066.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T01:34:53.994 INFO:teuthology.orchestra.run.smithi029.stdout:(146/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-21T01:34:54.027 INFO:teuthology.orchestra.run.smithi029.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 616 kB/s | 191 kB 00:00 2024-09-21T01:34:54.057 INFO:teuthology.orchestra.run.smithi066.stdout:(112/157): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-21T01:34:54.070 INFO:teuthology.orchestra.run.smithi029.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 1.0 MB/s | 427 kB 00:00 2024-09-21T01:34:54.082 INFO:teuthology.orchestra.run.smithi066.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-21T01:34:54.116 INFO:teuthology.orchestra.run.smithi066.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T01:34:54.141 INFO:teuthology.orchestra.run.smithi066.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T01:34:54.154 INFO:teuthology.orchestra.run.smithi029.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.9 MB/s | 1.6 MB 00:00 2024-09-21T01:34:54.166 INFO:teuthology.orchestra.run.smithi066.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T01:34:54.191 INFO:teuthology.orchestra.run.smithi066.stdout:(117/157): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-21T01:34:54.217 INFO:teuthology.orchestra.run.smithi066.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-21T01:34:54.242 INFO:teuthology.orchestra.run.smithi066.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-21T01:34:54.276 INFO:teuthology.orchestra.run.smithi066.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T01:34:54.309 INFO:teuthology.orchestra.run.smithi066.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T01:34:54.343 INFO:teuthology.orchestra.run.smithi066.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-21T01:34:54.346 INFO:teuthology.orchestra.run.smithi029.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 11 MB/s | 3.4 MB 00:00 2024-09-21T01:34:54.376 INFO:teuthology.orchestra.run.smithi066.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T01:34:54.402 INFO:teuthology.orchestra.run.smithi066.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T01:34:54.427 INFO:teuthology.orchestra.run.smithi066.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-21T01:34:54.452 INFO:teuthology.orchestra.run.smithi066.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-21T01:34:54.463 INFO:teuthology.orchestra.run.smithi029.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-21T01:34:54.478 INFO:teuthology.orchestra.run.smithi066.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-21T01:34:54.503 INFO:teuthology.orchestra.run.smithi066.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-21T01:34:54.538 INFO:teuthology.orchestra.run.smithi029.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 6.0 MB/s | 2.3 MB 00:00 2024-09-21T01:34:54.553 INFO:teuthology.orchestra.run.smithi066.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-21T01:34:54.578 INFO:teuthology.orchestra.run.smithi066.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:34:54.589 INFO:teuthology.orchestra.run.smithi029.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T01:34:54.604 INFO:teuthology.orchestra.run.smithi066.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T01:34:54.629 INFO:teuthology.orchestra.run.smithi066.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T01:34:54.631 INFO:teuthology.orchestra.run.smithi029.stdout:(154/157): NetworkManager-team-1.51.0-1.el9.x86 244 kB/s | 41 kB 00:00 2024-09-21T01:34:54.663 INFO:teuthology.orchestra.run.smithi066.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-21T01:34:54.682 INFO:teuthology.orchestra.run.smithi029.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-21T01:34:54.688 INFO:teuthology.orchestra.run.smithi066.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-21T01:34:54.713 INFO:teuthology.orchestra.run.smithi066.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T01:34:54.739 INFO:teuthology.orchestra.run.smithi066.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T01:34:54.764 INFO:teuthology.orchestra.run.smithi066.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T01:34:54.789 INFO:teuthology.orchestra.run.smithi066.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T01:34:54.814 INFO:teuthology.orchestra.run.smithi066.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:34:54.839 INFO:teuthology.orchestra.run.smithi066.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T01:34:54.865 INFO:teuthology.orchestra.run.smithi066.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T01:34:54.865 INFO:teuthology.orchestra.run.smithi029.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-21T01:34:54.890 INFO:teuthology.orchestra.run.smithi066.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T01:34:54.915 INFO:teuthology.orchestra.run.smithi066.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T01:34:54.941 INFO:teuthology.orchestra.run.smithi066.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T01:34:54.949 INFO:teuthology.orchestra.run.smithi029.stdout:(157/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-09-21T01:34:54.955 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:34:54.956 INFO:teuthology.orchestra.run.smithi029.stdout:Total 16 MB/s | 228 MB 00:13 2024-09-21T01:34:54.966 INFO:teuthology.orchestra.run.smithi066.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-21T01:34:55.000 INFO:teuthology.orchestra.run.smithi066.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T01:34:55.033 INFO:teuthology.orchestra.run.smithi066.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-21T01:34:55.092 INFO:teuthology.orchestra.run.smithi066.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T01:34:55.501 INFO:teuthology.orchestra.run.smithi066.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 7.3 MB/s | 3.4 MB 00:00 2024-09-21T01:34:55.551 INFO:teuthology.orchestra.run.smithi066.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.9 MB/s | 3.2 MB 00:00 2024-09-21T01:34:55.699 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-21T01:34:55.701 INFO:teuthology.orchestra.run.smithi066.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T01:34:55.743 INFO:teuthology.orchestra.run.smithi066.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 985 kB/s | 41 kB 00:00 2024-09-21T01:34:55.785 INFO:teuthology.orchestra.run.smithi066.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-21T01:34:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:34:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:34:55.827 INFO:teuthology.orchestra.run.smithi066.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.6 MB/s | 1.8 MB 00:00 2024-09-21T01:34:55.869 INFO:teuthology.orchestra.run.smithi066.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-21T01:34:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:34:55.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:34:55.955 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:34:55.994 INFO:teuthology.orchestra.run.smithi066.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-21T01:34:56.007 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:34:56.176 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-21T01:34:56.253 INFO:teuthology.orchestra.run.smithi066.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.0 MB/s | 16 MB 00:03 2024-09-21T01:34:56.258 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:34:56.258 INFO:teuthology.orchestra.run.smithi066.stdout:Total 12 MB/s | 228 MB 00:18 2024-09-21T01:34:56.347 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-21T01:34:56.347 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-21T01:34:56.347 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:34:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:34:56.591 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:34:56.624 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:34:56.715 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:34:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:34:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:34:56.933 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:34:56.984 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:34:57.021 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:34:57.127 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:34:57.155 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:34:57.218 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:34:57.240 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:34:57.260 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:34:57.458 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:34:57.484 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-21T01:34:57.591 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:34:57.657 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-21T01:34:57.657 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-21T01:34:57.742 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:34:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:34:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:34:58.159 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:34:58.273 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:34:58.302 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:34:58.337 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:34:58.411 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:34:58.464 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:34:58.504 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:34:58.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:34:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:34:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:34:58.690 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:34:58.750 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:34:58.780 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:34:58.806 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-21T01:34:58.807 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-21T01:34:58.815 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:34:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:34:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:34:58.978 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:34:59.007 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:34:59.077 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:34:59.122 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:34:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:34:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:34:59.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:34:59.601 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-21T01:34:59.601 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-21T01:34:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:35:00.012 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:35:00.041 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:35:00.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:35:00.122 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:35:00.163 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:35:00.318 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:35:00.383 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:35:00.475 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:35:00.548 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:35:00.614 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:35:00.654 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:35:00.690 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:35:00.737 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:35:00.782 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:35:00.824 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:35:00.867 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:35:01.004 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:35:01.043 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:35:01.085 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:35:01.188 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:35:01.227 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:35:01.267 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:35:01.293 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:35:01.317 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:35:01.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:35:01.519 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:35:01.558 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:35:01.586 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:35:01.620 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:35:01.659 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:35:01.733 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:35:01.898 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:35:02.065 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:35:02.248 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:35:02.620 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:35:02.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:35:02.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:35:02.796 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:35:02.835 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:35:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:35:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:35:03.725 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:35:03.776 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:35:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:35:03.911 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:35:03.965 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:35:04.011 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:35:04.099 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:35:04.138 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:35:04.320 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:35:04.353 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:35:04.429 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:35:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:35:05.184 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:35:05.675 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-21T01:35:05.833 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:35:05.885 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:35:05.951 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:35:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:35:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:35:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:35:06.157 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-21T01:35:06.248 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:35:06.304 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:35:06.375 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:35:06.415 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:35:06.416 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:35:06.440 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:35:06.441 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:35:06.485 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:35:06.523 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:35:06.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:35:06.584 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:35:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:06.787 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:06.817 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:06.843 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:35:06.910 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:06.924 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:35:06.948 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:35:06.967 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:06.981 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:06.988 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:35:07.071 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:07.079 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:07.102 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:35:07.110 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:35:07.188 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:35:07.202 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:07.231 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:35:07.242 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:35:07.260 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:35:07.270 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:07.378 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:07.407 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:35:07.420 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:35:07.470 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:35:07.471 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:35:07.498 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:07.509 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:35:07.531 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:35:07.597 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:07.624 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:35:07.634 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:35:07.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:07.670 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:35:07.700 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:35:07.721 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:35:07.738 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:07.739 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:35:07.760 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:35:07.868 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:35:07.962 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:35:07.994 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:35:08.115 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:35:08.156 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:35:08.204 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:08.270 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:08.277 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:08.411 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:08.508 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:08.623 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:35:08.657 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:35:08.693 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:35:08.767 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:35:08.819 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:35:08.859 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:35:08.902 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:35:08.943 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:35:08.979 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:35:09.038 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:35:09.050 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:35:09.111 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:35:09.140 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:35:09.175 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:35:09.239 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:35:09.259 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:35:09.281 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:35:09.295 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:35:09.328 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:35:09.338 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:35:09.367 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:35:09.412 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:35:09.439 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:35:09.474 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:35:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:35:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:35:09.566 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:35:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:35:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:35:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:35:09.703 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:35:09.749 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:35:09.760 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:35:09.810 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:35:09.869 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:35:09.912 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:35:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:35:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:35:10.092 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:35:10.097 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:10.129 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:35:10.132 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:35:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:35:10.313 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:35:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:35:10.398 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:35:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:35:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:35:10.504 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:35:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:35:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:35:10.569 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:35:10.696 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:10.736 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:10.748 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:35:10.816 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:35:10.826 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:10.826 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:35:10.827 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:35:10.827 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:10.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:35:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:35:11.047 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:35:11.085 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:35:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:35:11.131 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:35:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:35:11.237 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:35:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:35:11.331 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:35:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:35:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:35:11.558 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:35:11.665 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:35:11.759 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:35:11.798 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:35:11.842 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:35:11.850 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:35:11.889 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:35:11.890 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:35:11.931 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:35:11.941 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:35:11.956 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:35:12.037 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:35:12.149 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:35:12.179 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:35:12.221 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:35:12.229 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:35:12.257 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:35:12.300 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:35:12.324 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:35:12.348 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:35:12.388 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:35:12.397 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:35:12.417 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:35:12.452 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:35:12.473 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:35:12.486 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:35:12.519 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:35:12.527 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:35:12.561 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:35:12.564 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:35:12.621 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:35:12.650 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:35:12.689 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:35:12.701 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:35:12.740 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:35:12.742 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:35:12.802 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:35:12.847 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:35:12.900 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:35:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:35:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:35:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:35:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:35:13.201 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:35:13.260 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:35:13.303 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:35:13.336 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:35:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:35:13.598 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:35:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:35:13.655 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:35:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:35:13.704 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:35:13.727 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:35:13.746 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:35:13.767 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:35:13.807 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:35:13.847 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:35:13.858 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:35:13.891 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:35:13.919 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:35:13.976 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:35:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:35:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:35:14.010 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:35:14.055 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:35:14.098 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:35:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:35:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:35:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:35:14.289 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:35:14.321 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:35:14.325 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:35:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:35:14.400 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:35:15.098 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:35:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:35:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:35:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:35:15.256 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:35:15.360 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:35:15.423 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:35:15.477 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:35:15.576 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:35:15.614 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:35:15.805 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:35:15.837 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:35:15.926 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:35:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:35:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:35:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:35:16.640 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:35:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:35:16.772 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:35:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:35:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:35:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:35:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:35:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:35:18.098 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:35:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:35:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:35:18.296 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:35:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:35:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:18.811 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:35:18.851 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:35:18.896 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:35:19.430 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:35:19.467 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:19.522 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:35:19.552 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:35:20.283 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:20.315 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:20.916 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:20.960 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:21.052 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:21.052 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:35:21.052 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:35:21.052 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:21.086 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-09-21T01:35:22.544 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-09-21T01:35:22.545 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:35:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:22.734 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:22.734 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:35:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:35:22.758 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:35:22.758 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:35:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:24.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:24.356 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:24.579 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:24.603 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:24.603 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:24.603 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:35:24.603 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:24.604 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:24.604 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:25.071 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:25.095 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:25.095 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:25.095 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:35:25.095 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:25.096 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:25.096 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:25.202 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:35:25.259 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:35:25.314 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:35:25.503 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:35:25.548 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:35:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:35:25.715 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:35:25.863 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:35:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:35:25.937 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:35:25.965 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:35:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:26.521 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:35:26.588 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:35:26.612 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:29.809 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:29.868 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:30.824 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:30.852 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:30.967 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:31.135 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:35:31.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:31.186 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:31.186 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:31.186 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:35:31.186 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:31.187 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:31.187 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:31.348 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:35:31.395 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:35:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:35:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:32.414 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:32.414 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:32.415 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:35:32.415 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:32.415 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:32.415 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:32.458 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:32.481 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:32.481 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:32.481 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:35:32.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:32.533 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:32.557 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:32.876 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:32.899 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:32.900 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:32.900 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:35:32.900 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:32.900 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:32.900 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-09-21T01:35:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:33.591 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:33.591 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:35:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:35.250 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:35.279 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:35.519 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:35.543 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:35.543 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:35.544 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:35:35.544 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:35.544 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:35.544 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:36.036 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:36.061 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:36.176 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:35:36.234 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:35:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:35:36.494 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:35:36.555 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:35:36.644 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:35:36.732 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:35:36.864 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:35:36.885 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:35:36.904 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:35:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:35:36.945 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:35:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:35:36.978 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:35:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:35:37.005 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:35:37.194 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:35:37.243 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:35:37.265 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:35:37.295 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:35:37.333 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:35:37.374 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:37.404 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:37.436 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:35:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:35:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:37.956 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:37.998 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:38.019 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:38.873 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:35:39.044 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:35:39.086 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:35:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:35:39.214 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:35:39.240 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:35:39.265 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:35:39.266 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:35:39.308 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /root 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2024-09-21T01:35:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:39.450 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:39.475 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:35:39.481 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:39.481 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:39.528 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:35:39.528 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:39.528 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:35:39.528 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:39.660 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:39.661 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:39.791 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:35:40.927 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:40.983 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:41.193 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:35:41.221 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:35:41.485 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:42.000 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:35:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:42.033 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:42.121 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:35:42.157 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:42.179 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:35:42.252 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:35:42.341 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:35:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:42.401 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:42.401 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:42.402 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:35:42.402 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:42.402 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:42.402 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:42.454 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:35:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:35:42.567 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:35:42.595 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:35:42.618 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:35:42.694 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:35:42.856 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:35:42.908 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:35:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:35:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:35:43.507 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:43.533 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:35:43.566 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:35:43.636 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:35:43.639 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:35:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:43.673 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:43.762 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:43.762 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:43.763 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:35:43.763 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:43.822 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:43.847 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:44.190 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:44.214 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:44.214 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:44.214 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:35:44.214 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:44.215 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:35:45.843 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:35:45.843 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:35:45.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:35:45.845 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:35:45.846 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:35:45.846 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:35:45.857 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:35:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:35:45.859 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:35:45.860 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:35:45.861 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:35:45.862 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:35:45.863 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:35:45.864 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:35:45.865 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:35:45.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:35:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:35:47.105 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:48.117 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:35:48.148 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:48.267 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:35:48.428 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:35:48.440 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:35:48.468 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:48.469 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:35:48.493 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:48.519 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:35:48.545 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:48.550 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:35:48.556 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:35:48.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:35:48.660 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:35:48.681 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.682 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.683 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:35:48.684 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:35:48.685 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:35:48.686 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.687 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:35:48.688 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:35:48.689 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:35:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:35:48.691 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:35:48.692 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:35:48.693 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:35:48.694 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:35:48.694 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:35:48.694 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:35:48.694 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-21T01:35:48.739 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:35:48.772 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:35:48.815 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:35:48.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:35:48.875 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:35:48.921 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:35:49.109 DEBUG:teuthology.parallel:result is None 2024-09-21T01:35:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:35:49.801 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:49.825 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:35:49.825 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:49.826 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:35:49.826 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:49.826 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:35:49.826 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:49.877 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:49.901 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:35:49.901 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:49.901 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:35:49.901 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:49.954 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:49.979 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:35:49.979 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:49.980 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:35:49.980 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:49.980 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:35:49.981 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:50.313 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:50.337 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:35:50.338 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:35:50.338 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:35:50.338 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:50.338 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:35:50.338 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:35:50.502 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:35:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:35:50.641 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:35:50.733 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:35:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:35:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:35:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:35:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:50.903 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:50.903 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:50.987 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:51.012 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:35:51.017 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:51.017 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:51.197 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:51.198 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:51.273 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:35:54.615 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:35:54.667 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:35:54.717 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:35:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:35:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:35:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:35:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:35:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:35:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:35:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:35:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:35:56.758 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:35:56.873 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:35:56.924 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:35:57.022 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:35:57.056 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:35:57.090 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:35:57.124 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:35:57.125 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:57.218 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:35:57.218 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:57.303 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:35:57.328 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:35:57.333 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:57.333 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:57.432 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:35:57.433 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:35:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:35:57.435 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:35:57.435 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:35:57.435 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:35:57.435 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:35:57.448 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:35:57.448 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:35:57.449 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:35:57.450 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:35:57.451 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:35:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:35:57.453 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:35:57.454 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:35:57.455 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:35:57.456 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:35:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:35:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:35:57.459 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:35:57.460 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:35:57.524 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:35:57.524 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:57.617 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:35:57.695 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:36:00.146 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:36:00.147 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.148 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:00.150 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:36:00.151 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:36:00.154 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.155 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:36:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:36:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:36:00.158 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.159 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:36:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.161 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:36:00.162 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:36:00.163 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:36:00.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:36:00.163 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-21T01:36:00.598 DEBUG:teuthology.parallel:result is None 2024-09-21T01:36:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:36:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:36:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:36:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:36:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:36:09.500 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:36:09.501 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:36:09.502 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:36:09.503 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:36:09.504 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:36:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:36:09.506 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:36:09.507 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:36:09.509 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:36:09.509 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:36:09.510 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:36:09.511 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:36:09.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:36:09.513 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:36:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:36:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:36:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:36:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:36:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:36:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:36:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:36:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:36:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:36:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:36:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:36:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:36:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:36:12.075 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-21T01:36:12.616 DEBUG:teuthology.parallel:result is None 2024-09-21T01:36:12.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:12.773 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:36:12.796 INFO:teuthology.orchestra.run.smithi029.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:36:12.797 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:36:12.797 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:36:12.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:12.942 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:36:12.970 INFO:teuthology.orchestra.run.smithi066.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:36:12.970 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:36:12.970 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:36:12.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:13.114 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:36:13.140 INFO:teuthology.orchestra.run.smithi190.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:36:13.140 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:36:13.140 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:36:13.142 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T01:36:13.143 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:36:13.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:36:13.174 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:36:13.174 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:36:13.208 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:36:13.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:36:13.240 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T01:36:13.240 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:36:13.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:36:13.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:36:13.337 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:36:13.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:36:13.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:36:13.438 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:36:13.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:36:13.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:36:13.535 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T01:36:13.536 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:36:13.536 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:36:13.564 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:36:13.633 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:36:13.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:36:13.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:36:13.725 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:36:13.726 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:36:13.751 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:36:13.817 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T01:36:13.818 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:36:13.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:36:13.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:36:13.919 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:36:13.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:36:13.944 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:36:14.012 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:36:14.012 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:36:14.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:36:14.101 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T01:36:14.201 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T01:36:14.201 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:14.201 INFO:tasks.cephadm:Cluster fsid is e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:36:14.202 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T01:36:14.202 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.29', 'mon.b': '172.21.15.66', 'mon.c': '172.21.15.190'} 2024-09-21T01:36:14.202 INFO:tasks.cephadm:First mon is mon.a on smithi029 2024-09-21T01:36:14.202 INFO:tasks.cephadm:First mgr is x 2024-09-21T01:36:14.202 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T01:36:14.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname $(hostname -s) 2024-09-21T01:36:14.233 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-09-21T01:36:14.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-09-21T01:36:14.286 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T01:36:14.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:14.430 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T01:36:14.569 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:36:14.570 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:36:14.570 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:36:14.571 DEBUG:teuthology.orchestra.run.smithi029:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:15.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:36 /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:15.108 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:15.651 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:36 /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:15.653 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:16.209 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:36 /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:16.209 DEBUG:teuthology.orchestra.run.smithi029:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:16.230 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:16.250 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:16.286 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T01:36:16.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:36:16.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:36:16.292 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:36:17.768 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:36:17.782 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:36:19.107 INFO:teuthology.orchestra.run.smithi029.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:36:49.265 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:36:49.265 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:36:49.265 INFO:teuthology.orchestra.run.smithi029.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:36:49.266 INFO:teuthology.orchestra.run.smithi029.stdout: "repo_digests": [ 2024-09-21T01:36:49.266 INFO:teuthology.orchestra.run.smithi029.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:36:49.266 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T01:36:49.266 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:37:28.338 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-21T01:37:28.339 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-21T01:37:32.467 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-21T01:37:32.467 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:37:32.468 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:37:32.468 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-09-21T01:37:32.468 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:37:32.468 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-21T01:37:32.468 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-21T01:37:32.492 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph 2024-09-21T01:37:32.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-09-21T01:37:32.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-09-21T01:37:32.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /etc/ceph 2024-09-21T01:37:32.617 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-09-21T01:37:32.647 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-09-21T01:37:32.672 INFO:tasks.cephadm:Writing seed config... 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T01:37:32.673 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T01:37:32.674 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T01:37:32.675 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T01:37:32.675 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:37:32.675 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T01:37:32.693 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e38bc6b6-77b9-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-21T01:37:32.694 DEBUG:teuthology.orchestra.run.smithi029:mon.a> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T01:37:32.737 DEBUG:teuthology.orchestra.run.smithi029:mgr.x> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T01:37:32.781 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T01:37:32.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.29 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:37:32.998 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:37:32.998 INFO:teuthology.orchestra.run.smithi029.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.29', '--skip-admin-label'] 2024-09-21T01:37:32.998 INFO:teuthology.orchestra.run.smithi029.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T01:37:32.999 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying podman|docker is present... 2024-09-21T01:37:33.027 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:37:33.027 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying lvm2 is present... 2024-09-21T01:37:33.028 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying time synchronization is in place... 2024-09-21T01:37:33.036 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:37:33.036 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:37:33.042 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:37:33.043 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:37:33.050 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout enabled 2024-09-21T01:37:33.057 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout active 2024-09-21T01:37:33.057 INFO:teuthology.orchestra.run.smithi029.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:37:33.057 INFO:teuthology.orchestra.run.smithi029.stdout:Repeating the final host check... 2024-09-21T01:37:33.085 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:37:33.086 INFO:teuthology.orchestra.run.smithi029.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T01:37:33.086 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl is present 2024-09-21T01:37:33.086 INFO:teuthology.orchestra.run.smithi029.stdout:lvcreate is present 2024-09-21T01:37:33.093 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:37:33.093 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:37:33.100 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:37:33.100 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:37:33.107 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout enabled 2024-09-21T01:37:33.115 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout active 2024-09-21T01:37:33.115 INFO:teuthology.orchestra.run.smithi029.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:37:33.115 INFO:teuthology.orchestra.run.smithi029.stdout:Host looks OK 2024-09-21T01:37:33.115 INFO:teuthology.orchestra.run.smithi029.stdout:Cluster fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:33.115 INFO:teuthology.orchestra.run.smithi029.stdout:Acquiring lock 139655773112688 on /run/cephadm/e38bc6b6-77b9-11ef-baf5-efdc52797490.lock 2024-09-21T01:37:33.116 INFO:teuthology.orchestra.run.smithi029.stdout:Lock 139655773112688 acquired on /run/cephadm/e38bc6b6-77b9-11ef-baf5-efdc52797490.lock 2024-09-21T01:37:33.116 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying IP 172.21.15.29 port 3300 ... 2024-09-21T01:37:33.116 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying IP 172.21.15.29 port 6789 ... 2024-09-21T01:37:33.116 INFO:teuthology.orchestra.run.smithi029.stdout:Base mon IP(s) is [172.21.15.29:3300, 172.21.15.29:6789], mon addrv is [v2:172.21.15.29:3300,v1:172.21.15.29:6789] 2024-09-21T01:37:33.120 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.29 metric 100 2024-09-21T01:37:33.120 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.29 metric 100 2024-09-21T01:37:33.123 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T01:37:33.123 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T01:37:33.123 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:37:33.127 INFO:teuthology.orchestra.run.smithi029.stdout:Mon IP `172.21.15.29` is in CIDR network `172.21.0.0/20` 2024-09-21T01:37:33.128 INFO:teuthology.orchestra.run.smithi029.stdout:Mon IP `172.21.15.29` is in CIDR network `172.21.0.0/20` 2024-09-21T01:37:33.128 INFO:teuthology.orchestra.run.smithi029.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T01:37:33.128 INFO:teuthology.orchestra.run.smithi029.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T01:37:33.128 INFO:teuthology.orchestra.run.smithi029.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:37:33.690 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:37:33.690 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:37:33.690 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T01:37:33.690 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T01:37:33.691 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T01:37:33.691 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:37:33.691 INFO:teuthology.orchestra.run.smithi029.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T01:37:34.585 INFO:teuthology.orchestra.run.smithi029.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:37:34.585 INFO:teuthology.orchestra.run.smithi029.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:37:34.585 INFO:teuthology.orchestra.run.smithi029.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T01:37:35.598 INFO:teuthology.orchestra.run.smithi029.stdout:stat: stdout 167 167 2024-09-21T01:37:35.598 INFO:teuthology.orchestra.run.smithi029.stdout:Creating initial keys... 2024-09-21T01:37:36.562 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph-authtool: stdout AQBfI+5mSqPYORAA7X8q3nHnuBPSGtdzlhQBzQ== 2024-09-21T01:37:37.540 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph-authtool: stdout AQBgI+5mCGinNxAAmP1zbjkeTFiTMdrGkGSSCQ== 2024-09-21T01:37:38.528 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph-authtool: stdout AQBhI+5mYqMbNxAAI7Jyro4y6du6HKS9OifRJw== 2024-09-21T01:37:38.529 INFO:teuthology.orchestra.run.smithi029.stdout:Creating initial monmap... 2024-09-21T01:37:39.509 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:37:39.509 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T01:37:39.509 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:39.509 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool for a [v2:172.21.15.29:3300,v1:172.21.15.29:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:setting min_mon_release = quincy 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: set fsid to e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:37:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:Creating mon... 2024-09-21T01:37:40.624 INFO:teuthology.orchestra.run.smithi029.stdout:create mon.a on 2024-09-21T01:37:40.839 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T01:37:40.995 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T01:37:41.179 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target. 2024-09-21T01:37:41.179 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target. 2024-09-21T01:37:41.442 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a 2024-09-21T01:37:41.442 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T01:37:41.617 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target.wants/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@.service. 2024-09-21T01:37:42.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:42 smithi029 podman[30844]: 2024-09-21 01:37:42.05984164 +0000 UTC m=+0.311414138 container start 25ea696c4ea0e376fdb265db93a8727631c9b6db708332a99994f233a25676f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T01:37:42.154 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:37:42.154 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout disabled 2024-09-21T01:37:42.161 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:37:42.161 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:37:42.162 INFO:teuthology.orchestra.run.smithi029.stdout:firewalld.service is not enabled 2024-09-21T01:37:42.162 INFO:teuthology.orchestra.run.smithi029.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:37:42.162 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mon to start... 2024-09-21T01:37:42.162 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mon... 2024-09-21T01:37:42.378 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:42 smithi029 ceph-mon[30863]: mkfs e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:42.378 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:42 smithi029 ceph-mon[30863]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:37:42.379 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:42 smithi029 bash[30844]: 25ea696c4ea0e376fdb265db93a8727631c9b6db708332a99994f233a25676f4 2024-09-21T01:37:42.379 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:42 smithi029 systemd[1]: Started Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: monmap epoch 1 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: last_changed 2024-09-21T01:37:38.894365+0000 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: min_mon_release 19 (squid) 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: election_strategy: 1 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: fsmap 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: mgrmap e1: no daemons active 2024-09-21T01:37:43.257 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:43 smithi029 ceph-mon[30863]: from='client.? 172.21.15.29:0/1637941579' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T01:37:43.260 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout id: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout services: 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.546137s) 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout data: 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T01:37:43.261 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T01:37:43.262 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:43.262 INFO:teuthology.orchestra.run.smithi029.stdout:mon is available 2024-09-21T01:37:43.262 INFO:teuthology.orchestra.run.smithi029.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T01:37:44.387 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:44 smithi029 ceph-mon[30863]: from='client.? 172.21.15.29:0/1460029460' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:37:44.387 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:44 smithi029 ceph-mon[30863]: from='client.? 172.21.15.29:0/1460029460' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T01:37:44.392 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:44.392 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout fsid = e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.29:3300,v1:172.21.15.29:6789] 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:37:44.393 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:37:44.394 INFO:teuthology.orchestra.run.smithi029.stdout:Generating new minimal ceph.conf... 2024-09-21T01:37:45.206 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:45 smithi029 ceph-mon[30863]: from='client.? 172.21.15.29:0/4289978755' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:37:45.463 INFO:teuthology.orchestra.run.smithi029.stdout:Restarting the monitor... 2024-09-21T01:37:45.765 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:45 smithi029 systemd[1]: Stopping Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:37:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a[30859]: 2024-09-21T01:37:45.764+0000 7f82ecb40640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:37:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a[30859]: 2024-09-21T01:37:45.764+0000 7f82ecb40640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T01:37:46.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:45 smithi029 podman[31105]: 2024-09-21 01:37:45.88719545 +0000 UTC m=+0.254516667 container died 25ea696c4ea0e376fdb265db93a8727631c9b6db708332a99994f233a25676f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T01:37:46.519 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 podman[31105]: 2024-09-21 01:37:46.290220493 +0000 UTC m=+0.657541710 container cleanup 25ea696c4ea0e376fdb265db93a8727631c9b6db708332a99994f233a25676f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T01:37:46.519 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 bash[31105]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a 2024-09-21T01:37:46.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 podman[31118]: 2024-09-21 01:37:46.518788333 +0000 UTC m=+0.627190945 container remove 25ea696c4ea0e376fdb265db93a8727631c9b6db708332a99994f233a25676f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T01:37:46.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T01:37:46.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 systemd[1]: Stopped Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:37:46.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 systemd[1]: Starting Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 podman[31209]: 2024-09-21 01:37:46.76148069 +0000 UTC m=+0.016104819 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:46 smithi029 podman[31209]: 2024-09-21 01:37:46.862196846 +0000 UTC m=+0.116820999 container create 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 podman[31209]: 2024-09-21 01:37:47.074706864 +0000 UTC m=+0.329330979 container init 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 podman[31209]: 2024-09-21 01:37:47.078067068 +0000 UTC m=+0.332691185 container start 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:37:47.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: pidfile_write: ignore empty --pid-file 2024-09-21T01:37:47.112 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: load: jerasure load: lrc 2024-09-21T01:37:47.163 INFO:teuthology.orchestra.run.smithi029.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T01:37:47.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:37:47.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Git sha 0 2024-09-21T01:37:47.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:37:47.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: DB SUMMARY 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: DB Session ID: MAT7FAV171LJZQDNPA0M 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95160 ; 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:37:47.407 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.env: 0x559435451d60 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.info_log: 0x55943b9f3360 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.statistics: (nil) 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.use_fsync: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:37:47.408 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.db_log_dir: 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.wal_dir: 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.write_buffer_manager: 0x55943b9f7d60 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:37:47.409 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.unordered_write: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.row_cache: None 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.wal_filter: None 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:37:47.410 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.wal_compression: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:37:47.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:37:47.412 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:37:47.412 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:37:47.412 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_open_files: -1 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:37:47.413 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Compression algorithms supported: 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kZSTD supported: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:37:47.414 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.merge_operator: 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_filter: None 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55943b9f2fa0) 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:37:47.415 INFO:journalctl@ceph.mon.a.smithi029.stdout: index_type: 0 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: data_block_index_type: 0 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: index_shortening: 1 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: checksum: 4 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: no_block_cache: 0 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_cache: 0x5594379b7610 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_cache_options: 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: capacity : 536870912 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: num_shard_bits : 4 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: strict_capacity_limit : 0 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_cache_compressed: (nil) 2024-09-21T01:37:47.416 INFO:journalctl@ceph.mon.a.smithi029.stdout: persistent_cache: (nil) 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_size: 4096 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_size_deviation: 10 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_restart_interval: 16 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: index_block_restart_interval: 1 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: metadata_block_size: 4096 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: partition_filters: 0 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: use_delta_encoding: 1 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: filter_policy: bloomfilter 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: whole_key_filtering: 1 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: verify_compression: 0 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: format_version: 5 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: enable_index_compression: 1 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: block_align: 0 2024-09-21T01:37:47.417 INFO:journalctl@ceph.mon.a.smithi029.stdout: max_auto_readahead_size: 262144 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout: prepopulate_block_cache: 0 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression: NoCompression 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.num_levels: 7 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:37:47.418 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:37:47.419 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:37:47.420 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:37:47.421 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:37:47.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.ttl: 2592000 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:37:47.423 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3ed4bd38-9fc6-483a-8e5d-ed5e79de13f9 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882667113422, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882667114605, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91303, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89351, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11700, "raw_average_key_size": 47, "raw_value_size": 82767, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726882667, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3ed4bd38-9fc6-483a-8e5d-ed5e79de13f9", "db_session_id": "MAT7FAV171LJZQDNPA0M", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882667114722, "job": 1, "event": "recovery_finished"} 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55943ba16a80 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: DB pointer 0x55943c7bc000 2024-09-21T01:37:47.424 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: ** DB Stats ** 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: ** Compaction Stats [default] ** 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:37:47.425 INFO:journalctl@ceph.mon.a.smithi029.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: L0 2/0 91.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: Sum 2/0 91.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: ** Compaction Stats [default] ** 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:37:47.426 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Cumulative compaction: 0.00 GB write, 25.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Interval compaction: 0.00 GB write, 25.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Block cache BinnedLRUCache@0x5594379b7610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.427 INFO:journalctl@ceph.mon.a.smithi029.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: starting mon.a rank 0 at public addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] at bind addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???) e1 preinit fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).mds e1 new map 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).mds e1 print_map 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: e1 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: btime 2024-09-21T01:37:42:108364+0000 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: legacy client fscid: -1 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout: No filesystems configured 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:37:47.428 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 bash[31209]: 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 systemd[1]: Started Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: monmap epoch 1 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: last_changed 2024-09-21T01:37:38.894365+0000 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: min_mon_release 19 (squid) 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: election_strategy: 1 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: fsmap 2024-09-21T01:37:47.429 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:37:47.430 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:47 smithi029 ceph-mon[31228]: mgrmap e1: no daemons active 2024-09-21T01:37:48.218 INFO:teuthology.orchestra.run.smithi029.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T01:37:48.239 INFO:teuthology.orchestra.run.smithi029.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:37:48.239 INFO:teuthology.orchestra.run.smithi029.stdout:Creating mgr... 2024-09-21T01:37:48.240 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T01:37:48.240 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T01:37:48.534 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x 2024-09-21T01:37:48.534 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T01:37:48.691 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490.target.wants/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@.service. 2024-09-21T01:37:48.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:48 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/871180384' entity='client.admin' 2024-09-21T01:37:48.845 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:48 smithi029 systemd[1]: Starting Ceph mgr.x for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:37:49.234 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:48 smithi029 podman[31440]: 2024-09-21 01:37:48.826762068 +0000 UTC m=+0.016418172 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:37:49.234 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:48 smithi029 podman[31440]: 2024-09-21 01:37:48.927897686 +0000 UTC m=+0.117553822 container create c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T01:37:49.234 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:49 smithi029 podman[31440]: 2024-09-21 01:37:49.149936208 +0000 UTC m=+0.339592307 container init c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:37:49.234 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:49 smithi029 podman[31440]: 2024-09-21 01:37:49.152655059 +0000 UTC m=+0.342311154 container start c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:37:49.235 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:49 smithi029 bash[31440]: c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b 2024-09-21T01:37:49.249 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:37:49.249 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout disabled 2024-09-21T01:37:49.257 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:37:49.257 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:37:49.257 INFO:teuthology.orchestra.run.smithi029.stdout:firewalld.service is not enabled 2024-09-21T01:37:49.257 INFO:teuthology.orchestra.run.smithi029.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:37:49.267 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:37:49.267 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout disabled 2024-09-21T01:37:49.276 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:37:49.276 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:37:49.276 INFO:teuthology.orchestra.run.smithi029.stdout:firewalld.service is not enabled 2024-09-21T01:37:49.276 INFO:teuthology.orchestra.run.smithi029.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T01:37:49.276 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mgr to start... 2024-09-21T01:37:49.277 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mgr... 2024-09-21T01:37:49.511 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:49 smithi029 systemd[1]: Started Ceph mgr.x for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:37:50.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:49 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2335135182' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:37:50.412 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:37:50.414 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:37:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:37:50.419 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:37:50.419 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:37:42:108364+0000", 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T01:37:50.421 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:37:42.109324+0000", 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:37:50.422 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:37:50.423 INFO:teuthology.orchestra.run.smithi029.stdout:mgr not available, waiting (1/15)... 2024-09-21T01:37:53.295 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:52 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3957459057' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:37:53.809 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:53.809 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:37:53.809 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:37:53.809 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:37:53.809 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:37:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:37:42:108364+0000", 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:37:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:53.813 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:37:42.109324+0000", 2024-09-21T01:37:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:37:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:37:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:mgr not available, waiting (2/15)... 2024-09-21T01:37:54.850 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: Activating manager daemon x 2024-09-21T01:37:54.850 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: mgrmap e2: x(active, starting, since 0.00372779s) 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: Manager daemon x is now available 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' 2024-09-21T01:37:54.851 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' 2024-09-21T01:37:54.852 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:54 smithi029 ceph-mon[31228]: from='mgr.14100 172.21.15.29:0/2763461750' entity='mgr.x' 2024-09-21T01:37:56.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:56 smithi029 ceph-mon[31228]: mgrmap e3: x(active, since 1.00833s) 2024-09-21T01:37:57.117 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:37:57.118 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:37:57.119 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:37:57.122 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:37:42:108364+0000", 2024-09-21T01:37:57.123 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:37:42.109324+0000", 2024-09-21T01:37:57.124 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:37:57.125 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:37:57.125 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:37:57.125 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:37:57.125 INFO:teuthology.orchestra.run.smithi029.stdout:mgr is available 2024-09-21T01:37:57.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:57 smithi029 ceph-mon[31228]: mgrmap e4: x(active, since 2s) 2024-09-21T01:37:57.568 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:57 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/232736230' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:37:58.405 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:58.405 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:37:58.405 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout fsid = e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:37:58.405 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.29:3300,v1:172.21.15.29:6789] 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:37:58.406 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:37:58.407 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:37:58.407 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:37:58.407 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:37:58.407 INFO:teuthology.orchestra.run.smithi029.stdout:Enabling cephadm module... 2024-09-21T01:37:58.652 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1861843002' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:37:59.796 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:37:59 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/969666829' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T01:37:59.796 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ignoring --setuser ceph since I am not root 2024-09-21T01:37:59.796 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:37:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ignoring --setgroup ceph since I am not root 2024-09-21T01:38:00.784 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/969666829' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T01:38:00.784 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:00 smithi029 ceph-mon[31228]: mgrmap e5: x(active, since 5s) 2024-09-21T01:38:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:38:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for the mgr to restart... 2024-09-21T01:38:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mgr epoch 5... 2024-09-21T01:38:01.719 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:01 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1952464961' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:38:05.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: Active manager daemon x restarted 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: Activating manager daemon x 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: mgrmap e6: x(active, starting, since 0.00415934s) 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:38:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:04 smithi029 ceph-mon[31228]: Manager daemon x is now available 2024-09-21T01:38:06.382 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:38:06.382 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T01:38:06.382 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:38:06.383 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:38:06.383 INFO:teuthology.orchestra.run.smithi029.stdout:mgr epoch 5 is available 2024-09-21T01:38:06.383 INFO:teuthology.orchestra.run.smithi029.stdout:Verifying orchestrator module is enabled... 2024-09-21T01:38:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T01:38:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: mgrmap e7: x(active, since 1.00611s) 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:38:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:06 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:38:08.024 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:07 smithi029 ceph-mon[31228]: Found migration_current of "None". Setting to last migration. 2024-09-21T01:38:08.024 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:07 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2317108' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T01:38:08.025 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:07 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2317108' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T01:38:08.025 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:07 smithi029 ceph-mon[31228]: mgrmap e8: x(active, since 2s) 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:38:08.189+0000 7f26a095a640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:38:08] ENGINE Error in HTTPServer.serve 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: Traceback (most recent call last): 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._connections.run(self.expiration_interval) 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._run(expiration_interval) 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:38:08.286 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: s = self.context.wrap_socket( 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: return self.sslsocket_class._create( 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self.do_handshake() 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._sslobj.do_handshake() 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:38:08.287 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:38:08.288 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T01:38:08.288 INFO:teuthology.orchestra.run.smithi029.stdout:Setting orchestrator backend to cephadm... 2024-09-21T01:38:08.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:08] ENGINE Bus STARTING 2024-09-21T01:38:08.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:08] ENGINE Serving on https://172.21.15.29:7150 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:08] ENGINE Error in HTTPServer.serve 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout: Traceback (most recent call last): 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:38:08.845 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._run(expiration_interval) 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: s = self.context.wrap_socket( 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:38:08.846 INFO:journalctl@ceph.mon.a.smithi029.stdout: return self.sslsocket_class._create( 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout: self.do_handshake() 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._sslobj.do_handshake() 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:08] ENGINE Serving on http://172.21.15.29:8765 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:08] ENGINE Bus STARTED 2024-09-21T01:38:08.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:08 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:10.198 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:09 smithi029 ceph-mon[31228]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:10.199 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:09 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:10.199 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:09 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:10.876 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T01:38:10.877 INFO:teuthology.orchestra.run.smithi029.stdout:Generating ssh key... 2024-09-21T01:38:11.126 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:10 smithi029 ceph-mon[31228]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:12.267 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-mon[31228]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:12.267 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-mon[31228]: Generating ssh key... 2024-09-21T01:38:12.267 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: Generating public/private rsa key pair. 2024-09-21T01:38:12.267 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: Your identification has been saved in /tmp/tmpoqli2p39/key 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: Your public key has been saved in /tmp/tmpoqli2p39/key.pub 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: The key fingerprint is: 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: SHA256:+h8mvhN6Bfw6im2lW4wTHnIRQUEWg9d3M84YKG6vID0 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: The key's randomart image is: 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: +---[RSA 3072]----+ 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | +X= . | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | ..ooo o + | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | o.o . * o | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | o.o . o | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | ...+.So | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | . E+ *+ o | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | . o==++o | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | o===o . | 2024-09-21T01:38:12.268 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: | ..+++=. | 2024-09-21T01:38:12.269 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:11 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: +----[SHA256]-----+ 2024-09-21T01:38:13.281 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:12 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:13.281 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:12 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:13.913 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:38:13.914 INFO:teuthology.orchestra.run.smithi029.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:38:13.914 INFO:teuthology.orchestra.run.smithi029.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T01:38:13.922 INFO:teuthology.orchestra.run.smithi029.stdout:Adding host smithi029... 2024-09-21T01:38:14.171 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:13 smithi029 ceph-mon[31228]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:15.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:15 smithi029 ceph-mon[31228]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi029", "addr": "172.21.15.29", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:16.715 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:16 smithi029 ceph-mon[31228]: Deploying cephadm binary to smithi029 2024-09-21T01:38:19.026 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout Added host 'smithi029' with addr '172.21.15.29' 2024-09-21T01:38:19.026 INFO:teuthology.orchestra.run.smithi029.stdout:Deploying unmanaged mon service... 2024-09-21T01:38:19.704 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:19 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:19.704 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:19 smithi029 ceph-mon[31228]: Added host smithi029 2024-09-21T01:38:19.704 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:19 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:20.543 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T01:38:20.544 INFO:teuthology.orchestra.run.smithi029.stdout:Deploying unmanaged mgr service... 2024-09-21T01:38:20.825 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:20 smithi029 ceph-mon[31228]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:20.826 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:20 smithi029 ceph-mon[31228]: Saving service mon spec with placement count:5 2024-09-21T01:38:20.826 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:20 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:21.946 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T01:38:22.593 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:22 smithi029 ceph-mon[31228]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:22.593 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:22 smithi029 ceph-mon[31228]: Saving service mgr spec with placement count:2 2024-09-21T01:38:22.593 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:22 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:24.002 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:23 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/947140378' entity='client.admin' 2024-09-21T01:38:24.002 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:23 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:24.916 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:24 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:25.522 INFO:teuthology.orchestra.run.smithi029.stdout:Enabling the dashboard module... 2024-09-21T01:38:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:25 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1120733982' entity='client.admin' 2024-09-21T01:38:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:25 smithi029 ceph-mon[31228]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:38:26.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:26 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2766061035' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T01:38:26.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:26 smithi029 ceph-mon[31228]: from='mgr.14118 172.21.15.29:0/1282052709' entity='mgr.x' 2024-09-21T01:38:26.845 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ignoring --setuser ceph since I am not root 2024-09-21T01:38:26.845 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ignoring --setgroup ceph since I am not root 2024-09-21T01:38:27.828 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:27 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2766061035' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T01:38:27.828 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:27 smithi029 ceph-mon[31228]: mgrmap e9: x(active, since 21s) 2024-09-21T01:38:28.937 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:28 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4131499812' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:38:28.939 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for the mgr to restart... 2024-09-21T01:38:28.941 INFO:teuthology.orchestra.run.smithi029.stdout:Waiting for mgr epoch 9... 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: Active manager daemon x restarted 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: Activating manager daemon x 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: mgrmap e10: x(active, starting, since 0.00375332s) 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:38:32.298 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:38:32.299 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:32 smithi029 ceph-mon[31228]: Manager daemon x is now available 2024-09-21T01:38:33.341 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:33.341 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:38:33.341 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:38:33.341 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:33 smithi029 ceph-mon[31228]: mgrmap e11: x(active, since 1.00744s) 2024-09-21T01:38:33.622 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:38:33.622 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T01:38:33.622 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:38:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:38:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:mgr epoch 9 is available 2024-09-21T01:38:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T01:38:34.241 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-mon[31228]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T01:38:34.241 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-mon[31228]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T01:38:34.548 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:38:34.390+0000 7f3507260640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:38:34] ENGINE Error in HTTPServer.serve 2024-09-21T01:38:34.548 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: Traceback (most recent call last): 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._connections.run(self.expiration_interval) 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._run(expiration_interval) 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: s = self.context.wrap_socket( 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: return self.sslsocket_class._create( 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:38:34.549 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self.do_handshake() 2024-09-21T01:38:34.550 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:38:34.550 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: self._sslobj.do_handshake() 2024-09-21T01:38:34.550 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:38:34.550 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:38:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:38:35.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: mgrmap e12: x(active, since 2s) 2024-09-21T01:38:35.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:34] ENGINE Bus STARTING 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:34] ENGINE Serving on http://172.21.15.29:8765 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:34] ENGINE Serving on https://172.21.15.29:7150 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:34] ENGINE Bus STARTED 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: [21/Sep/2024:01:38:34] ENGINE Error in HTTPServer.serve 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: Traceback (most recent call last): 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._run(expiration_interval) 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:38:35.345 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: s = self.context.wrap_socket( 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: return self.sslsocket_class._create( 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: self.do_handshake() 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: self._sslobj.do_handshake() 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:35.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:36.178 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUTcXQd0F5vR1WBt5e1uW9A8B8tp4wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAxMzgzNVoXDTM0\nMDkyMjAxMzgzNVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjI5MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA5vb61bQpAvBbIMxoQoNwb3MhzCbg1LlbQmPf\noCqQ/6neoIogX5WdyfJX0CLPYXiSAXCW0cdZEuTHhACnPV3xJBEkAY6mGAJT8ytb\n7Y8Pl2wM6ptgD7wuZ7C8XMQSFP3KbuKE4LuD5OE/ZY1VOsiTtd+/l0nworwo4YO+\n5ggeiEuKvi1iUECjVjc/QwozEAzOv6yYGLzH77vayndqB3Axt3Gn2cTgmRke8Vv9\ndWAZQEzQS5Pr3q/jhVEOfzLiJxgEwfDviTHW1PT/ECM39PuVf8hO9kP7quHAG9t/\nrkfmggvCIBSjSBx4jPI4xIA+JcMcF4TFB7bT3uPnM1kE3KBKcOayn8ItvhW3ze0g\n5Png+lKx3TDsqWyKGbydOU8AfxHtEO9qeCHcEOboOOoz2d36luSTmRDMfSwHjK5F\nPuErUnHPWxAYAigezNWslavj5Hi+hCx5sT2LMzpnaL79xf6VpPPfLKz5iE7XQ7at\nhvY9zHfACNkQayU1TEdljoKAauFM0/I1x5Q4ddfqPf2TguVf6kbzle7bDNa/uwl/\nLZ5klEqGp4BIaS3ahH9LHNgiScM6+fDYQ3YSoGEoSWVL+/8eUXphfgRGsgpI8Hf6\nQYHL+NQfrGrYiMCpGoqWPaKOSoKGsHMkwFPwD+Shdwl5EnYLN6w3sWbt2Cf3lhMS\n/vGOlMsCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDI5ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPHTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAo\n1y1d7PXz6PFnxvm/AnvpVxIVaIkwsDLdaqakbhoCDg3EGlRp4hY0oEVbFTkG9CbU\nwSSMgkguiADuaOnkoHjQyXUB4A/OTsCBIdz+YFidT7NtC0SxeR6V1pKgJzFsiJuS\nd0LIgGtZ8I5882qET75w64O+IubNl7TUtGFP1s5fNwsvxZoJCiBlwe3YQZHRFJNO\njeki+ofUHTZOLWTDnsuRtcUMauDcO8Iq80U8x1p7VmCki3Zxh6fcpiSicQXTpuzB\nftPX5HQS162PxFGAPxkkxD4iwHCIpKDuR4knfEDqUKsnzItt0o3Eytuo6C5hvza8\noxlhdz8XGJ1OMZT8PpmEE6QyEAVwE/LX8/mWI3uaUFqOuDJ26osIAnACcRPTTE1x\nSBb9UJsFQWDv/mpcIIt3vBlypq3IIzsuZgYGwNLUDfMfZA/zPkc6zeY3XpVqgDMK\nbLExSGmw8yX2aqNCCn/4YC9U8a+gsJHLxG0qmYzshaYomY7M5XV1CjBGGWgHBECO\newWB7WNAnnW443KVJD3uN4RzXedqU97qz3wiR4G6LCeN1xs3RWIQHdefBLHX1jPT\nb92z8LNOa92EMmdYdT8t/DEiUPp/BeU2VVUkA8qX9ImmwOVi5LGnYylKOn9fyR7Y\nsstH2yQ/rhCOYlqC5TZhOMSlM0pjpPZYds2/ilKQrQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEA5vb61bQpAvBbIMxoQoNwb3MhzCbg1LlbQmPfoCqQ/6neoIog\nX5WdyfJX0CLPYXiSAXCW0cdZEuTHhACnPV3xJBEkAY6mGAJT8ytb7Y8Pl2wM6ptg\nD7wuZ7C8XMQSFP3KbuKE4LuD5OE/ZY1VOsiTtd+/l0nworwo4YO+5ggeiEuKvi1i\nUECjVjc/QwozEAzOv6yYGLzH77vayndqB3Axt3Gn2cTgmRke8Vv9dWAZQEzQS5Pr\n3q/jhVEOfzLiJxgEwfDviTHW1PT/ECM39PuVf8hO9kP7quHAG9t/rkfmggvCIBSj\nSBx4jPI4xIA+JcMcF4TFB7bT3uPnM1kE3KBKcOayn8ItvhW3ze0g5Png+lKx3TDs\nqWyKGbydOU8AfxHtEO9qeCHcEOboOOoz2d36luSTmRDMfSwHjK5FPuErUnHPWxAY\nAigezNWslavj5Hi+hCx5sT2LMzpnaL79xf6VpPPfLKz5iE7XQ7athvY9zHfACNkQ\nayU1TEdljoKAauFM0/I1x5Q4ddfqPf2TguVf6kbzle7bDNa/uwl/LZ5klEqGp4BI\naS3ahH9LHNgiScM6+fDYQ3YSoGEoSWVL+/8eUXphfgRGsgpI8Hf6QYHL+NQfrGrY\niMCpGoqWPaKOSoKGsHMkwFPwD+Shdwl5EnYLN6w3sWbt2Cf3lhMS/vGOlMsCAwEA\nAQKCAgArUxoDW1u0Eu0tLlrCW1CMf27QElzrWme9gOmSbFmts6owuChRdcnW93A0\neUsocAsHXRzVdg0F58drQgodD1GsH9D+v8j1SP6BZ1lDqynUEC8uFP5EhVq5FaSH\nzDnOGTtPaItfcp1PolcXkBb0Q2q9255fXT3X8O7EOsfk85HypCT+/plqD230PsHN\nrAyfDp/7wX74aPUiSb1FbHgFtCyHSjkftvggj99HjCdzzJaAAsix3KoKqDjIKcfL\n4XodtjoyPSVZuXEgmmJ8Hvcgq87D3Qm+1UrQp683Kon9YbxiGZ2NEgH9BWYxnkA6\n0AmYYMyUttGU3F1K+rpEXgb08dZ0kv9vc9HsEsddOqVF/8UvFFVdS5QELtJFykV2\nbU8sPCh7E5MQFfj8z8TOW5W8BU/Xqn/kiAd7gMudsyMREn5n4deR2itdB9a0LLcP\nI2w84zqjUtILetMvKYgUiz1p1XLpi789VaWDnANJwOXAtFvwGv5K44fVKbBvlb6i\nnFKFeKmMpxH7HFsOsxcwdEEQvlQ26fmFH57eyokLx4CsSZ1QYizZGl1keBYk6ckS\nxOdm3YBiKW/gUcNxO6IS/J/NO14ccaV0Vm0ZPw32I+eKUVFt2ZLbkMBdmJKJ08mq\nwjXeBqJ1eWRbTMrPfVUs+nsQhNfYlCbAE23KjvK5xdepgH0SJQKCAQEA9OXystP9\nGT4aJ2ZgAbxV/C51LkIETCUk64LoxcT85YrfQKHAgSBqaaujilKgu3vYg4PvSx3Y\nNjBEzzYlAYAJEEtCRZzbhmBKNa4iYPcFV0hzCbvOUqT5R7E7YD2HCZBXxiccg37X\nK9HoHuIQ7wAIavK+h0m5jOD+uP7h6WVY+zKjGHtFQYpWlNAAIBpWHLMlonEKNwDs\nX39Q/n/H/mta44upB4Ri0akh5XOWkRdFyUeesiVDPO9iGBX8eRLJFDKqOipkX8mk\nDlSdB/FnPV5dHehmgIcV5x7oT2YhRgnpXoiW9QDIK+jlZGzVvL2gba98bdUpyJqW\nImLVOSxyNTPPVQKCAQEA8W9VWrR0ftXD2Srx+2tKYLqJAdu+G7sPsHS1Hux9qHgo\nlN2cYP/lJ1shMxsx7b2Eoy/F7YoA/Hs3x4206uodgo3moZ6u07ozDDYjBuKpT830\nHknD9HmWdvW7BNAcvwItkH3judhKYVFuyfBZQfDyX9xROe2UG5dnEoOyC1/wcPTW\nHidKOfq422FizuATrB2ZFdM/R9xTWeykoVvBfpSCSVp8tn+kajHX4hymENr6tjHL\nOCjhWUN/8SVnbXYvXRdok1ty5ZCa0v/pEkVM0rlSI/bsi/n1sDY3DjqgPg8yo/ou\nsdbyn94tsb4Sjuuh2LSUJYufCAUwXlO8dLUojweTnwKCAQAmOyubPTra0PYWRoYg\nXz1xI/OJKCcxl+7ekU4zaaAtWXOmzFmYDu5t1qyuMF4FRr0zMsqNiHgzZW+ohYRb\naph/obAT2/vtS2TqYmfdiVKdIKNM1rPL9RMxXPqu+Iv3M0Bwn7R01r1Siu9YuuP3\n26J0x7Z0uk6sLoe1a5zq6bZtaKdLcBsMs0L28k3CtTnSOylZ5Zm5onxEIEUNpfcb\nwbiroybdFI60ulmfT18nAWIUnQiqOOT5Q0mmBvCmGgeoDozCjdGjnn/s+MqW7dm3\nG98fhyNRV0VVEl8j3ga9v1heOKaQUvHs4OvGtLLdRyEfXnDnWIqwVWmwroso5MZj\nhdYNAoIBAQCAjDS3/MjiF1MVt+XrZa7OXcNA+Az6mti5rQoakvQ81O1WhcATbWxH\nTiAZI+url3KdqHwMA6SOydRjLbIcOdbIgmE/wEt5c/RmFhUEP37CfPfzgHcTHgHd\nCO0wnNYRZUl5W236r4y60C7a2lvL3kxOefHNlA99uVkp5HDR2Z0dp16x412EDViL\nhgij+W6//2pY0oco7ch7YqItP7uAd/X+tG0ehku2Nx+oxnQtGUrP4yc+mZWSf3wJ\nm068W2bQe/w0KUa9sAafPZ6NGdTThdSUTY6Y4NfNoLAjpVdYc6MU58og6j94m0Vd\nE8Sx0Af2Z/9RFkvwa0Lbll4LDcGwleolAoIBAQDNk15VcbCz8QaQ8LWA6qR81lo9\nH3Ptit4NDSTn7YEmzYtMofH5BjQYJPh0ahfFnTF3offLhpfhcw3LzolbE6tJYGHl\nlBEqRtfs2NcAMrAA2/J7hQM0r+7xYNYOUf0k00IIBg3MsU+8X8W5nK0D3GpfpQ2d\nvymSiiFEGcAL25tJ5QOLW66vIVvdLSIKpkVGj+FMfS+i3s/nJwOYB2WP7jT6opJO\ngCIaG+aexiE/PkBlHd0ftBCToC4ykGVYeOrjFagCj5pYVAs69V1Cvcx3nmow3ZuX\nxyuzypO7fYS9uOaCzYktvfMA8KRbnt1CYftnoEbVUq8lE9ZcvMChm4qpMFVR\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T01:38:36.430 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:37 smithi029 ceph-mon[31228]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:38.448 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T01:38:39.678 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T01:38:39.678 INFO:teuthology.orchestra.run.smithi029.stdout:Creating initial admin user... 2024-09-21T01:38:40.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:40 smithi029 ceph-mon[31228]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:40.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:40 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:41.236 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ycx0WdHlJuylPn8q2IolXePHkojwXFgOnaerbOBw4JW134xM3YqQ2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726882720, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T01:38:41.236 INFO:teuthology.orchestra.run.smithi029.stdout:Fetching dashboard port number... 2024-09-21T01:38:41.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:41 smithi029 ceph-mon[31228]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:42.557 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T01:38:42.567 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:38:42.567 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout disabled 2024-09-21T01:38:42.574 INFO:teuthology.orchestra.run.smithi029.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:38:42.574 INFO:teuthology.orchestra.run.smithi029.stdout:systemctl: stdout inactive 2024-09-21T01:38:42.574 INFO:teuthology.orchestra.run.smithi029.stdout:firewalld.service is not enabled 2024-09-21T01:38:42.574 INFO:teuthology.orchestra.run.smithi029.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout:Ceph Dashboard is now available at: 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: URL: https://smithi029.front.sepia.ceph.com:8443/ 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: User: admin 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: Password: 69gfp2cfqt 2024-09-21T01:38:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:42.577 INFO:teuthology.orchestra.run.smithi029.stdout:Saving cluster configuration to /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config directory 2024-09-21T01:38:42.809 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:42 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2786063197' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.957 INFO:teuthology.orchestra.run.smithi029.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: ceph telemetry on 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout:For more information see: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:43.958 INFO:teuthology.orchestra.run.smithi029.stdout:Bootstrap complete. 2024-09-21T01:38:44.003 INFO:tasks.cephadm:Fetching config... 2024-09-21T01:38:44.003 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:38:44.003 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T01:38:44.020 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T01:38:44.020 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:38:44.020 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T01:38:44.076 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T01:38:44.076 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:38:44.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T01:38:44.142 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T01:38:44.142 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:38:44.142 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T01:38:44.198 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T01:38:44.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:38:44.273 INFO:teuthology.orchestra.run.smithi029.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:38:44.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:38:44.328 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:38:44.341 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:38:44.382 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeWE5FSfMZD1EBnGxPWFOqiSv3ipKN1Tk74nT536SMXuxme1YyLALjRRJ1XgghYUpZTFd95RlBoTZXVym085AWr8/tbFIkz0o1mUd/+lCp0DkELcCSzXKsu47N3/sz7Ke6EMy/ds16gMi5F5En/ABZGfJ1CnKn2/PNBZ6HstDX8A6exgHA6RhRpva96Jo/ZALk5Jx0qFn/NcgX7IBhEYyjpSgTYQhy3I0Dx88CZ0/kfPmX3ZjPSq/h0845kIXh17r4ApS8Z0p6cfMqrGArSOgOO+WXwX4GeLIVuVij7YRj/0inFHHGIU7SSY8mcE8AX8yOHw8Ydi1UISy4Q66NJRIEHocXVY/9C5Xm6kyI9kA6+Az8xgS2TgBLUE5jYR+BQ18uzqqXWevUM7QPnxQt0MkpJONw+W1cTAyaS5FT0e//WVTWfsKSOgfWm2n3PMd7TXV+HZVTvxAvA55e2G6LTSBAhTfp3grS35dsS6zNK/UwjIt3LhVNmyIqokuLVKwDQAU= ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:38:44.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T01:38:44.546 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:44 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1802941654' entity='client.admin' 2024-09-21T01:38:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:38:45.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:45 smithi029 ceph-mon[31228]: mgrmap e13: x(active, since 12s) 2024-09-21T01:38:46.157 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T01:38:46.158 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T01:38:46.647 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:46 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4107333323' entity='client.admin' 2024-09-21T01:38:46.743 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:38:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"} : dispatch 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:38:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:38:48.236 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi066 2024-09-21T01:38:48.236 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:38:48.236 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:38:48.257 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:38:48.257 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:38:48.315 INFO:tasks.cephadm:Adding host smithi066 to orchestrator... 2024-09-21T01:38:48.315 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch host add smithi066 2024-09-21T01:38:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: Updating smithi029:/etc/ceph/ceph.conf 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: Updating smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:49.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:49 smithi029 ceph-mon[31228]: Updating smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:38:51.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:50 smithi029 ceph-mon[31228]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:51 smithi029 ceph-mon[31228]: Deploying cephadm binary to smithi066 2024-09-21T01:38:52.990 INFO:teuthology.orchestra.run.smithi029.stdout:Added host 'smithi066' with addr '172.21.15.66' 2024-09-21T01:38:53.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:38:53.902 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:38:54.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:53 smithi029 ceph-mon[31228]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:38:54.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:54.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:38:54.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:54.024 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:38:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[{"addr": "172.21.15.29", "hostname": "smithi029", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}] 2024-09-21T01:38:54.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:54 smithi029 ceph-mon[31228]: Added host smithi066 2024-09-21T01:38:54.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:54 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:55.247 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-09-21T01:38:55.247 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:38:55.247 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:38:55.266 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:38:55.266 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:38:55.325 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-09-21T01:38:55.325 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch host add smithi190 2024-09-21T01:38:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:38:55.953 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:55 smithi029 ceph-mon[31228]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:38:55.953 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:55 smithi029 ceph-mon[31228]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:38:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:38:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:38:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"} : dispatch 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:38:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:58 smithi029 ceph-mon[31228]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-21T01:38:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:58 smithi029 ceph-mon[31228]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:38:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:58 smithi029 ceph-mon[31228]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:38:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:58 smithi029 ceph-mon[31228]: Deploying cephadm binary to smithi190 2024-09-21T01:38:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:58 smithi029 ceph-mon[31228]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:39:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:38:59 smithi029 ceph-mon[31228]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:00.128 INFO:teuthology.orchestra.run.smithi029.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-09-21T01:39:00.907 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:39:01.166 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:01.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:01.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:01.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: Added host smithi190 2024-09-21T01:39:01.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:01.426 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:01.426 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:39:01.856 INFO:teuthology.orchestra.run.smithi029.stdout:[{"addr": "172.21.15.29", "hostname": "smithi029", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-09-21T01:39:02.538 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:02.569 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T01:39:02.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T01:39:02.822 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:03.565 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:03 smithi029 ceph-mon[31228]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:39:03.565 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:03 smithi029 ceph-mon[31228]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:04.448 INFO:teuthology.orchestra.run.smithi029.stderr:adjusted tunables profile to default 2024-09-21T01:39:04.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2747301248' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:39:04.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"} : dispatch 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:04.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:05.186 INFO:tasks.cephadm:Adding mon.a on smithi029 2024-09-21T01:39:05.187 INFO:tasks.cephadm:Adding mon.b on smithi066 2024-09-21T01:39:05.187 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-09-21T01:39:05.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c' 2024-09-21T01:39:05.424 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-21T01:39:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2747301248' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:06.058 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-09-21T01:39:06.677 DEBUG:teuthology.orchestra.run.smithi066:mon.b> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T01:39:06.679 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T01:39:06.681 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:39:06.682 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:39:07.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: Saving service mon spec with placement smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c;count:3 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:07.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:07 smithi029 ceph-mon[31228]: Deploying daemon mon.c on smithi190 2024-09-21T01:39:07.589 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:09.161 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:39:09.161 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","modified":"2024-09-21T01:37:38.894365Z","created":"2024-09-21T01:37:38.894365Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T01:39:09.161 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-09-21T01:39:09.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:09 smithi029 ceph-mon[31228]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:10.303 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:10 smithi190 ceph-mon[34368]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T01:39:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:10 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/1509037377' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:10.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:10 smithi190 bash[34349]: 1a326e76f88cb62738d5477853e6138c0d9da2d6057128fddbce6692c8f26ced 2024-09-21T01:39:10.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:10 smithi190 systemd[1]: Started Ceph mon.c for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:39:10.908 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:39:10.908 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:39:11.162 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:39:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: mon.a calling monitor election 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: mon.c calling monitor election 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: monmap epoch 2 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: last_changed 2024-09-21T01:39:10.286757+0000 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: min_mon_release 19 (squid) 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: election_strategy: 1 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: fsmap 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: mgrmap e13: x(active, since 43s) 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: overall HEALTH_OK 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:15.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: mon.a calling monitor election 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: mon.c calling monitor election 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:15.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: monmap epoch 2 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: last_changed 2024-09-21T01:39:10.286757+0000 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: min_mon_release 19 (squid) 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: election_strategy: 1 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: fsmap 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:15.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: mgrmap e13: x(active, since 43s) 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: overall HEALTH_OK 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:15.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:39:16.266 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","modified":"2024-09-21T01:39:10.286757Z","created":"2024-09-21T01:37:38.894365Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:39:16.266 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-09-21T01:39:16.558 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:16 smithi190 ceph-mon[34368]: Deploying daemon mon.b on smithi066 2024-09-21T01:39:16.558 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/1654136943' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:16.558 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:16.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:16 smithi029 ceph-mon[31228]: Deploying daemon mon.b on smithi066 2024-09-21T01:39:16.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/1654136943' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:16.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:17 smithi190 ceph-mon[34368]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:17.827 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:39:17.828 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:39:17.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:17 smithi029 ceph-mon[31228]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:18.069 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:39:18.749 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:39:18.750 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","modified":"2024-09-21T01:39:10.286757Z","created":"2024-09-21T01:37:38.894365Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:39:18.750 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-09-21T01:39:19.136 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 podman[34982]: 2024-09-21 01:39:19.136480967 +0000 UTC m=+0.177787841 container create 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T01:39:19.391 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 podman[34982]: 2024-09-21 01:39:19.355475037 +0000 UTC m=+0.396781909 container init 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:39:19.391 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 podman[34982]: 2024-09-21 01:39:19.358684975 +0000 UTC m=+0.399991847 container start 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T01:39:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:19 smithi190 ceph-mon[34368]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:19 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/1913738343' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:39:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:39:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pidfile_write: ignore empty --pid-file 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: load: jerasure load: lrc 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Git sha 0 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: DB SUMMARY 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: DB Session ID: M2LFU9L0L7U7RRQIDO6U 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T01:39:19.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.env: 0x562859aefd60 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.info_log: 0x56285ed33360 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.statistics: (nil) 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.use_fsync: 0 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:39:19.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.db_log_dir: 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.wal_dir: 2024-09-21T01:39:19.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:39:19.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.write_buffer_manager: 0x56285ed37d60 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.unordered_write: 0 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:39:19.805 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.row_cache: None 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.wal_filter: None 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.wal_compression: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:39:19.806 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:39:19.807 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_open_files: -1 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Compression algorithms supported: 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kZSTD supported: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:39:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.merge_operator: 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_filter: None 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:39:19.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56285ed32fa0) 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_type: 0 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: data_block_index_type: 0 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_shortening: 1 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: checksum: 4 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: no_block_cache: 0 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache: 0x56285acf7610 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_options: 2024-09-21T01:39:19.810 INFO:journalctl@ceph.mon.b.smithi066.stdout: capacity : 536870912 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: num_shard_bits : 4 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: strict_capacity_limit : 0 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_compressed: (nil) 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: persistent_cache: (nil) 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_size: 4096 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_size_deviation: 10 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_restart_interval: 16 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_block_restart_interval: 1 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: metadata_block_size: 4096 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: partition_filters: 0 2024-09-21T01:39:19.811 INFO:journalctl@ceph.mon.b.smithi066.stdout: use_delta_encoding: 1 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: filter_policy: bloomfilter 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: whole_key_filtering: 1 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: verify_compression: 0 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: format_version: 5 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: enable_index_compression: 1 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_align: 0 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: max_auto_readahead_size: 262144 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: prepopulate_block_cache: 0 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:39:19.812 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression: NoCompression 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.num_levels: 7 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:39:19.813 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:39:19.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:39:19.815 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:39:19.816 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:39:19.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.ttl: 2592000 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:39:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 68b02bb1-c090-4f54-a203-5e86fd9616de 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882759397268, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882759397888, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726882759, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "68b02bb1-c090-4f54-a203-5e86fd9616de", "db_session_id": "M2LFU9L0L7U7RRQIDO6U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726882759398029, "job": 1, "event": "recovery_finished"} 2024-09-21T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56285ed56a80 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: DB pointer 0x56285fafc000 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** DB Stats ** 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-21T01:39:19.820 INFO:journalctl@ceph.mon.b.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:39:19.821 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: Block cache BinnedLRUCache@0x56285acf7610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T01:39:19.822 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: using public_addr v2:172.21.15.66:0/0 -> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: starting mon.b rank -1 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(???) e0 preinit fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: e1 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: btime 2024-09-21T01:37:42:108364+0000 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: legacy client fscid: -1 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout: No filesystems configured 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T01:39:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Deploying cephadm binary to smithi066 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Added host smithi066 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"} : dispatch 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-21T01:39:19.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Deploying cephadm binary to smithi190 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Added host smithi190 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2747301248' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:39:19.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"} : dispatch 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2747301248' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Saving service mon spec with placement smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c;count:3 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Deploying daemon mon.c on smithi190 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/1509037377' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.a calling monitor election 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.c calling monitor election 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: monmap epoch 2 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: last_changed 2024-09-21T01:39:10.286757+0000 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:19.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: min_mon_release 19 (squid) 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: election_strategy: 1 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: fsmap 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mgrmap e13: x(active, since 43s) 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: overall HEALTH_OK 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: Deploying daemon mon.b on smithi066 2024-09-21T01:39:19.830 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/1654136943' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/1913738343' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 ceph-mon[35001]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 bash[34982]: 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 2024-09-21T01:39:19.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:19 smithi066 systemd[1]: Started Ceph mon.b for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:39:19.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:19 smithi029 ceph-mon[31228]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:19.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:19 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/1913738343' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:20.326 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:39:20.326 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:39:20.575 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:39:24.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: mon.a calling monitor election 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: mon.c calling monitor election 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: mon.b calling monitor election 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: monmap epoch 3 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: last_changed 2024-09-21T01:39:19.428549+0000 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: min_mon_release 19 (squid) 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: election_strategy: 1 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: fsmap 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: mgrmap e13: x(active, since 52s) 2024-09-21T01:39:24.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: overall HEALTH_OK 2024-09-21T01:39:24.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.802 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: mon.a calling monitor election 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: mon.c calling monitor election 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: mon.b calling monitor election 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: monmap epoch 3 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: last_changed 2024-09-21T01:39:19.428549+0000 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: min_mon_release 19 (squid) 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: election_strategy: 1 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: fsmap 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: mgrmap e13: x(active, since 52s) 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: overall HEALTH_OK 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.847 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.931 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:39:24.931 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","modified":"2024-09-21T01:39:19.428549Z","created":"2024-09-21T01:37:38.894365Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T01:39:24.931 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: mon.a calling monitor election 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: mon.c calling monitor election 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: mon.b calling monitor election 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:24.932 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: monmap epoch 3 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: last_changed 2024-09-21T01:39:19.428549+0000 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: created 2024-09-21T01:37:38.894365+0000 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: min_mon_release 19 (squid) 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: election_strategy: 1 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: 0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: fsmap 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: mgrmap e13: x(active, since 52s) 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: overall HEALTH_OK 2024-09-21T01:39:24.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.934 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.934 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:24.934 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/966996269' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.654 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/966996269' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:25.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:25.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/966996269' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:39:25.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:39:25.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:25.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:26.237 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T01:39:26.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T01:39:26.501 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi029:/etc/ceph/ceph.conf 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:27.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Updating smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.117 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:39:27.118 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:27.118 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:27.118 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.118 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:26 smithi029 ceph-mon[31228]: Reconfiguring daemon mon.a on smithi029 2024-09-21T01:39:27.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi029:/etc/ceph/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Updating smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.170 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:26 smithi066 ceph-mon[35001]: Reconfiguring daemon mon.a on smithi029 2024-09-21T01:39:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi029:/etc/ceph/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Updating smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:27.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:26 smithi190 ceph-mon[34368]: Reconfiguring daemon mon.a on smithi029 2024-09-21T01:39:27.854 INFO:teuthology.orchestra.run.smithi029.stdout:# minimal ceph.conf for e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:27.854 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-09-21T01:39:27.855 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:39:27.855 INFO:teuthology.orchestra.run.smithi029.stdout: mon_host = [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/581624644' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:28.702 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/581624644' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:28.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:28.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/581624644' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:28.730 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T01:39:28.730 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:39:28.730 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:39:28.758 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:39:28.758 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:28.824 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:39:28.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:39:28.852 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:39:28.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:28.918 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:39:28.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:39:28.951 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:39:28.951 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:39:29.019 INFO:tasks.cephadm:Adding mgr.x on smithi029 2024-09-21T01:39:29.020 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-09-21T01:39:29.020 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-09-21T01:39:29.020 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi029=x;smithi066=y;smithi190=z' 2024-09-21T01:39:29.300 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:39:29.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:29 smithi190 ceph-mon[34368]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:29.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:29 smithi190 ceph-mon[34368]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:39:29.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:29 smithi190 ceph-mon[34368]: Reconfiguring daemon mon.b on smithi066 2024-09-21T01:39:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:29 smithi066 ceph-mon[35001]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:29 smithi066 ceph-mon[35001]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:39:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:29 smithi066 ceph-mon[35001]: Reconfiguring daemon mon.b on smithi066 2024-09-21T01:39:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:29 smithi029 ceph-mon[31228]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:29 smithi029 ceph-mon[31228]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:39:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:29 smithi029 ceph-mon[31228]: Reconfiguring daemon mon.b on smithi066 2024-09-21T01:39:29.957 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: Reconfiguring daemon mon.c on smithi190 2024-09-21T01:39:30.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: Reconfiguring daemon mon.c on smithi190 2024-09-21T01:39:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: Reconfiguring daemon mon.c on smithi190 2024-09-21T01:39:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.239 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T01:39:31.242 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T01:39:31.244 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T01:39:31.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T01:39:31.247 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T01:39:31.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T01:39:31.290 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T01:39:31.290 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T01:39:31.334 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T01:39:31.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T01:39:31.378 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T01:39:31.378 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T01:39:31.421 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T01:39:31.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T01:39:31.465 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T01:39:31.465 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T01:39:31.509 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T01:39:31.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T01:39:31.548 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi029=x;smithi066=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:31.548 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: Saving service mgr spec with placement smithi029=x;smithi066=y;smithi190=z;count:3 2024-09-21T01:39:31.548 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:31.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi029=x;smithi066=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: Saving service mgr spec with placement smithi029=x;smithi066=y;smithi190=z;count:3 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:31.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.552 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T01:39:31.552 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T01:39:31.579 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:31.582 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T01:39:31.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T01:39:31.633 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T01:39:31.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T01:39:31.676 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T01:39:31.676 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T01:39:31.719 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T01:39:31.720 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T01:39:31.764 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T01:39:31.764 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T01:39:31.812 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T01:39:31.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T01:39:31.856 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T01:39:31.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi029=x;smithi066=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: Saving service mgr spec with placement smithi029=x;smithi066=y;smithi190=z;count:3 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:31.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:31.903 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T01:39:31.903 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T01:39:31.953 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T01:39:31.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T01:39:32.000 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T01:39:32.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T01:39:32.043 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T01:39:32.043 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T01:39:32.097 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T01:39:32.097 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T01:39:32.156 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T01:39:32.157 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T01:39:32.218 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T01:39:32.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T01:39:32.270 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T01:39:32.270 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T01:39:32.313 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T01:39:32.313 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T01:39:32.366 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T01:39:32.366 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T01:39:32.416 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T01:39:32.416 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi029') 2024-09-21T01:39:32.500 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:32.665 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:32.715 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:32 smithi029 ceph-mon[31228]: Deploying daemon mgr.z on smithi190 2024-09-21T01:39:32.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:32 smithi190 ceph-mon[34368]: Deploying daemon mgr.z on smithi190 2024-09-21T01:39:32.728 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:32 smithi190 systemd[1]: Starting Ceph mgr.z for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:39:32.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:32 smithi066 ceph-mon[35001]: Deploying daemon mgr.z on smithi190 2024-09-21T01:39:33.042 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:32 smithi190 podman[35610]: 2024-09-21 01:39:32.694384434 +0000 UTC m=+0.015342067 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:39:33.042 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:32 smithi190 podman[35610]: 2024-09-21 01:39:32.819181277 +0000 UTC m=+0.140138941 container create 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T01:39:33.042 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:33 smithi190 podman[35610]: 2024-09-21 01:39:33.004688646 +0000 UTC m=+0.325646272 container init 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T01:39:33.042 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:33 smithi190 podman[35610]: 2024-09-21 01:39:33.00765863 +0000 UTC m=+0.328616262 container start 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:39:33.043 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mgr[35629]: -- 172.21.15.190:0/2015758380 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558bc63e50e0 con 0x558bcb188480 2024-09-21T01:39:33.478 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:33 smithi190 bash[35610]: 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 2024-09-21T01:39:33.478 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 01:39:33 smithi190 systemd[1]: Started Ceph mgr.z for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='client.? ' entity='client.admin' 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:33 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4051041290' entity='client.admin' 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='client.? ' entity='client.admin' 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:33.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:33.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:33.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:33.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:33.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:33 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4051041290' entity='client.admin' 2024-09-21T01:39:34.129 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.148 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.148 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.148 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.149 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.149 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.149 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.154 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.154 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.154 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.155 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.160 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.165 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.196 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='client.? ' entity='client.admin' 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:33 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4051041290' entity='client.admin' 2024-09-21T01:39:34.911 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:34 smithi029 ceph-mon[31228]: Deploying daemon mgr.y on smithi066 2024-09-21T01:39:34.911 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:34 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4138629192' entity='client.admin' 2024-09-21T01:39:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:34 smithi066 ceph-mon[35001]: Deploying daemon mgr.y on smithi066 2024-09-21T01:39:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:34 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4138629192' entity='client.admin' 2024-09-21T01:39:35.050 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:34 smithi066 systemd[1]: Starting Ceph mgr.y for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:39:35.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:34 smithi190 ceph-mon[34368]: Deploying daemon mgr.y on smithi066 2024-09-21T01:39:35.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:34 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4138629192' entity='client.admin' 2024-09-21T01:39:35.338 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 podman[35804]: 2024-09-21 01:39:34.988578965 +0000 UTC m=+0.022421293 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:39:35.338 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 podman[35804]: 2024-09-21 01:39:35.089442814 +0000 UTC m=+0.123285160 container create 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:39:35.338 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 podman[35804]: 2024-09-21 01:39:35.300020636 +0000 UTC m=+0.333862941 container init 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T01:39:35.339 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 podman[35804]: 2024-09-21 01:39:35.30305396 +0000 UTC m=+0.336896273 container start 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T01:39:35.765 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mgr[35823]: -- 172.21.15.66:0/1025402185 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a364130e0 con 0x561a3b1b6000 2024-09-21T01:39:35.765 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 bash[35804]: 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e 2024-09-21T01:39:35.765 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 01:39:35 smithi066 systemd[1]: Started Ceph mgr.y for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:39:35.967 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:35.968 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:35 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/167167655' entity='client.admin' 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:35 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/167167655' entity='client.admin' 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:36.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:35 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/167167655' entity='client.admin' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: Reconfiguring daemon mgr.x on smithi029 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/825552764' entity='client.admin' 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:37 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2458841802' entity='client.admin' 2024-09-21T01:39:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: Reconfiguring daemon mgr.x on smithi029 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/825552764' entity='client.admin' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:37 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2458841802' entity='client.admin' 2024-09-21T01:39:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: Reconfiguring daemon mgr.x on smithi029 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/825552764' entity='client.admin' 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:37.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:37 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2458841802' entity='client.admin' 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: Standby manager daemon z started 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:38 smithi066 ceph-mon[35001]: from='client.? ' entity='client.admin' 2024-09-21T01:39:38.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: Standby manager daemon z started 2024-09-21T01:39:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:39:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:39:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:38 smithi029 ceph-mon[31228]: from='client.? ' entity='client.admin' 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: Standby manager daemon z started 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.190:0/4075926041' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:38 smithi190 ceph-mon[34368]: from='client.? ' entity='client.admin' 2024-09-21T01:39:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:39 smithi066 ceph-mon[35001]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:39 smithi066 ceph-mon[35001]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T01:39:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:39:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:39 smithi029 ceph-mon[31228]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:39 smithi029 ceph-mon[31228]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T01:39:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:39:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:39 smithi190 ceph-mon[34368]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:39 smithi190 ceph-mon[34368]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T01:39:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:39:41.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.044+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fb950005180 con 0x7fb96c06b920 2024-09-21T01:39:41.045 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2728010769' entity='client.admin' 2024-09-21T01:39:41.045 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:41.045 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/629927733' entity='client.admin' 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1248537171' entity='client.admin' 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: Standby manager daemon y started 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:39:41.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:40 smithi029 ceph-mon[31228]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.048+0000 7fb95b7fe640 1 -- 172.21.15.29:0/4036692384 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7fb95c06ebd0 con 0x7fb96c06b920 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb95407afd0 msgr2=0x7fb95407d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb95407afd0 0x7fb95407d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb960000c00 tx=0x7fb960005730 comp rx=0 tx=0).stop 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb96c06b920 msgr2=0x7fb96c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb96c06b920 0x7fb96c199eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00ea60 tx=0x7fb95c00ef20 comp rx=0 tx=0).stop 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 shutdown_connections 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb95407afd0 0x7fb95407d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96c109470 0x7fb96c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb96c106930 0x7fb96c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 --2- 172.21.15.29:0/4036692384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb96c06b920 0x7fb96c199eb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 >> 172.21.15.29:0/4036692384 conn(0x7fb96c0fdfb0 msgr2=0x7fb96c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 shutdown_connections 2024-09-21T01:39:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:41.051+0000 7fb970e80640 1 -- 172.21.15.29:0/4036692384 wait complete. 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2728010769' entity='client.admin' 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/629927733' entity='client.admin' 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1248537171' entity='client.admin' 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: Standby manager daemon y started 2024-09-21T01:39:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:39:41.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:41.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:39:41.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:40 smithi190 ceph-mon[34368]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2728010769' entity='client.admin' 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/629927733' entity='client.admin' 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1248537171' entity='client.admin' 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: Standby manager daemon y started 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:39:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:40 smithi066 ceph-mon[35001]: from='mgr.? 172.21.15.66:0/3909199445' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:39:42.193 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:41 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2004779301' entity='client.admin' 2024-09-21T01:39:42.193 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:41 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4036692384' entity='client.admin' 2024-09-21T01:39:42.193 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:41 smithi029 ceph-mon[31228]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-21T01:39:42.193 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:39:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:41 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2004779301' entity='client.admin' 2024-09-21T01:39:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:41 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4036692384' entity='client.admin' 2024-09-21T01:39:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:41 smithi190 ceph-mon[34368]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-21T01:39:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:39:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:41 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2004779301' entity='client.admin' 2024-09-21T01:39:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:41 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4036692384' entity='client.admin' 2024-09-21T01:39:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:41 smithi066 ceph-mon[35001]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-21T01:39:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:39:43.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:42 smithi029 ceph-mon[31228]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:43.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:42 smithi190 ceph-mon[34368]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:43.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:42 smithi066 ceph-mon[35001]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:43.860 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi029') 2024-09-21T01:39:43.860 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi029') 2024-09-21T01:39:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:44 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3917414708' entity='client.admin' 2024-09-21T01:39:44.539 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:44 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3917414708' entity='client.admin' 2024-09-21T01:39:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:44 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3917414708' entity='client.admin' 2024-09-21T01:39:44.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 -- 172.21.15.29:0/785569346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f262406c6a0 msgr2=0x7f26241063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 --2- 172.21.15.29:0/785569346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f262406c6a0 0x7f26241063c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2614009900 tx=0x7f261402a410 comp rx=0 tx=0).stop 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 -- 172.21.15.29:0/785569346 shutdown_connections 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 --2- 172.21.15.29:0/785569346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26241094a0 0x7f262410b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 --2- 172.21.15.29:0/785569346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2624106900 0x7f2624108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 --2- 172.21.15.29:0/785569346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f262406c6a0 0x7f26241063c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 -- 172.21.15.29:0/785569346 >> 172.21.15.29:0/785569346 conn(0x7f26240fe000 msgr2=0x7f2624100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 -- 172.21.15.29:0/785569346 shutdown_connections 2024-09-21T01:39:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.995+0000 7f2629abb640 1 -- 172.21.15.29:0/785569346 wait complete. 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 Processor -- start 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 -- start start 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 0x7f2624199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26241094a0 0x7f262419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f262410f3e0 con 0x7f2624106900 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f262410f260 con 0x7f262406c6a0 2024-09-21T01:39:44.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.996+0000 7f2629abb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f262410f560 con 0x7f26241094a0 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53736/0 (socket says 172.21.15.29:53736) 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 -- 172.21.15.29:0/2748050842 learned_addr learned my addr 172.21.15.29:0/2748050842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f26237fe640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 0x7f2624199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2623fff640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26241094a0 0x7f262419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 -- 172.21.15.29:0/2748050842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26241094a0 msgr2=0x7f262419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26241094a0 0x7f262419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 -- 172.21.15.29:0/2748050842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 msgr2=0x7f2624199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 0x7f2624199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 -- 172.21.15.29:0/2748050842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262419ba00 con 0x7f2624106900 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f26237fe640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 0x7f2624199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2622ffd640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f261800d950 tx=0x7f261800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618017030 con 0x7f2624106900 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618004020 con 0x7f2624106900 2024-09-21T01:39:44.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262419bcf0 con 0x7f2624106900 2024-09-21T01:39:45.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.997+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180047a0 con 0x7f2624106900 2024-09-21T01:39:45.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.998+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a23f0 con 0x7f2624106900 2024-09-21T01:39:45.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:44.999+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26240ffb20 con 0x7f2624106900 2024-09-21T01:39:45.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.000+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f2618009a30 con 0x7f2624106900 2024-09-21T01:39:45.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.001+0000 7f2620ff9640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 0x7f260c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:45.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.001+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f26180e55c0 con 0x7f2624106900 2024-09-21T01:39:45.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.001+0000 7f26237fe640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 0x7f260c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:45.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.002+0000 7f26237fe640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 0x7f260c0b8a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2614004770 tx=0x7f26140314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:45.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.004+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26180adba0 con 0x7f2624106900 2024-09-21T01:39:45.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.193+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f262402e640 con 0x7f2624106900 2024-09-21T01:39:45.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.197+0000 7f2620ff9640 1 -- 172.21.15.29:0/2748050842 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f26180b1f80 con 0x7f2624106900 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 msgr2=0x7f260c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 0x7f260c0b8a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2614004770 tx=0x7f26140314a0 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 msgr2=0x7f262419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f261800d950 tx=0x7f261800de10 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 shutdown_connections 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f260c0b6500 0x7f260c0b8a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26241094a0 0x7f262419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2624106900 0x7f262419a640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 --2- 172.21.15.29:0/2748050842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262406c6a0 0x7f2624199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 >> 172.21.15.29:0/2748050842 conn(0x7f26240fe000 msgr2=0x7f2624104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 shutdown_connections 2024-09-21T01:39:45.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:45.199+0000 7f2629abb640 1 -- 172.21.15.29:0/2748050842 wait complete. 2024-09-21T01:39:45.450 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:45 smithi066 ceph-mon[35001]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:45.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:45 smithi190 ceph-mon[34368]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:45 smithi029 ceph-mon[31228]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:46.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:46 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2748050842' entity='client.admin' 2024-09-21T01:39:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:46 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2748050842' entity='client.admin' 2024-09-21T01:39:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:46 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2748050842' entity='client.admin' 2024-09-21T01:39:47.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:47 smithi029 ceph-mon[31228]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:47.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:47 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/344800156' entity='client.admin' 2024-09-21T01:39:47.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:47 smithi190 ceph-mon[34368]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:47.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:47 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/344800156' entity='client.admin' 2024-09-21T01:39:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:47 smithi066 ceph-mon[35001]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:47 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/344800156' entity='client.admin' 2024-09-21T01:39:47.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 -- 172.21.15.29:0/3887646245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 msgr2=0x7fe70c101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 --2- 172.21.15.29:0/3887646245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c101820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02a410 comp rx=0 tx=0).stop 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 -- 172.21.15.29:0/3887646245 shutdown_connections 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 --2- 172.21.15.29:0/3887646245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 0x7fe70c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 --2- 172.21.15.29:0/3887646245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 --2- 172.21.15.29:0/3887646245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c101820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 -- 172.21.15.29:0/3887646245 >> 172.21.15.29:0/3887646245 conn(0x7fe70c0fd310 msgr2=0x7fe70c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 -- 172.21.15.29:0/3887646245 shutdown_connections 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.604+0000 7fe711169640 1 -- 172.21.15.29:0/3887646245 wait complete. 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.605+0000 7fe711169640 1 Processor -- start 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.605+0000 7fe711169640 1 -- start start 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:47.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 0x7fe70c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10f3d0 con 0x7fe70c101410 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10f250 con 0x7fe70c101d60 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe711169640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10f550 con 0x7fe70c109520 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53774/0 (socket says 172.21.15.29:53774) 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 -- 172.21.15.29:0/428329042 learned_addr learned my addr 172.21.15.29:0/428329042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70a59c640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70b59e640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 0x7fe70c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 -- 172.21.15.29:0/428329042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 msgr2=0x7fe70c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 0x7fe70c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 -- 172.21.15.29:0/428329042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 msgr2=0x7fe70c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 -- 172.21.15.29:0/428329042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19b9f0 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70a59c640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe70ad9d640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc031440 tx=0x7fe6fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.606+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc03f070 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.607+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6fc031c50 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.607+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc035030 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.607+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19bc80 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.607+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a2210 con 0x7fe70c101410 2024-09-21T01:39:47.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.608+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c107df0 con 0x7fe70c101410 2024-09-21T01:39:47.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.609+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fe6fc042070 con 0x7fe70c101410 2024-09-21T01:39:47.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.611+0000 7fe6f3fff640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 0x7fe6f40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:47.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.611+0000 7fe70a59c640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 0x7fe6f40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:47.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.611+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe6fc106560 con 0x7fe70c101410 2024-09-21T01:39:47.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.611+0000 7fe70a59c640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 0x7fe6f40b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe700005d00 tx=0x7fe70000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:47.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.614+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6fc0d2ea0 con 0x7fe70c101410 2024-09-21T01:39:47.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.802+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe70c02e640 con 0x7fe70c101410 2024-09-21T01:39:47.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.807+0000 7fe6f3fff640 1 -- 172.21.15.29:0/428329042 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7fe6fc0d7de0 con 0x7fe70c101410 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 msgr2=0x7fe6f40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 0x7fe6f40b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe700005d00 tx=0x7fe70000b410 comp rx=0 tx=0).stop 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 msgr2=0x7fe70c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc031440 tx=0x7fe6fc003c10 comp rx=0 tx=0).stop 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 shutdown_connections 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6f40b6480 0x7fe6f40b8990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c109520 0x7fe70c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70c101d60 0x7fe70c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 --2- 172.21.15.29:0/428329042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe70c101410 0x7fe70c199fc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:47.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.809+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 >> 172.21.15.29:0/428329042 conn(0x7fe70c0fd310 msgr2=0x7fe70c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:47.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.810+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 shutdown_connections 2024-09-21T01:39:47.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.810+0000 7fe711169640 1 -- 172.21.15.29:0/428329042 wait complete. 2024-09-21T01:39:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea6007afd0 msgr2=0x7fea6007d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea6007afd0 0x7fea6007d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea6400abe0 tx=0x7fea6400a2d0 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea741047e0 msgr2=0x7fea7419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea741047e0 0x7fea7419a540 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea5c006010 tx=0x7fea5c003c10 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 shutdown_connections 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea6007afd0 0x7fea6007d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea74105180 0x7fea7419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea741047e0 0x7fea7419a540 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 --2- 172.21.15.29:0/2004779301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea74103750 0x7fea74199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:48.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 >> 172.21.15.29:0/2004779301 conn(0x7fea740ff0c0 msgr2=0x7fea74107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:48.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 shutdown_connections 2024-09-21T01:39:48.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:40.942+0000 7fea78bb4640 1 -- 172.21.15.29:0/2004779301 wait complete. 2024-09-21T01:39:49.002 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:48 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/428329042' entity='client.admin' 2024-09-21T01:39:49.002 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:48 smithi029 ceph-mon[31228]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.037 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi029') 2024-09-21T01:39:49.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:48 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/428329042' entity='client.admin' 2024-09-21T01:39:49.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:48 smithi190 ceph-mon[34368]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:49.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:48 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/428329042' entity='client.admin' 2024-09-21T01:39:49.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:48 smithi066 ceph-mon[35001]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:49.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 -- 172.21.15.29:0/1212287436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c10b6b0 msgr2=0x7f643c10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:49.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 --2- 172.21.15.29:0/1212287436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c10b6b0 0x7f643c10dbc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6434009ca0 tx=0x7f643402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 -- 172.21.15.29:0/1212287436 shutdown_connections 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 --2- 172.21.15.29:0/1212287436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c10b6b0 0x7f643c10dbc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 --2- 172.21.15.29:0/1212287436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c108d00 0x7f643c10b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 --2- 172.21.15.29:0/1212287436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c06c490 0x7f643c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 -- 172.21.15.29:0/1212287436 >> 172.21.15.29:0/1212287436 conn(0x7f643c0fdfe0 msgr2=0x7f643c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 -- 172.21.15.29:0/1212287436 shutdown_connections 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.865+0000 7f6443eab640 1 -- 172.21.15.29:0/1212287436 wait complete. 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 Processor -- start 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 -- start start 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c108d00 0x7f643c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c10b6b0 0x7f643c19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:49.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1115c0 con 0x7f643c06c490 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f643c111440 con 0x7f643c108d00 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6443eab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f643c111740 con 0x7f643c10b6b0 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6441c47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6441c47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53762/0 (socket says 172.21.15.29:53762) 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.866+0000 7f6441c47640 1 -- 172.21.15.29:0/344800156 learned_addr learned my addr 172.21.15.29:0/344800156 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441446640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c108d00 0x7f643c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6442448640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c10b6b0 0x7f643c19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 -- 172.21.15.29:0/344800156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c10b6b0 msgr2=0x7f643c19d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c10b6b0 0x7f643c19d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 -- 172.21.15.29:0/344800156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c108d00 msgr2=0x7f643c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c108d00 0x7f643c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 -- 172.21.15.29:0/344800156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19dbb0 con 0x7f643c06c490 2024-09-21T01:39:49.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6441c47640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f643800ea60 tx=0x7f643800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643800cde0 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64380035f0 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19dea0 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438010690 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.867+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a45a0 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.868+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c0ffb00 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.870+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f64380037a0 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.871+0000 7f642effd640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 0x7f64280b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.871+0000 7f6441446640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 0x7f64280b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.871+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f64380e5b00 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.871+0000 7f6441446640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 0x7f64280b8a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6430009bd0 tx=0x7f6430005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:46.874+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6438056710 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.068+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f643c02e640 con 0x7f643c06c490 2024-09-21T01:39:49.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.072+0000 7f642effd640 1 -- 172.21.15.29:0/344800156 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f64380adfc0 con 0x7f643c06c490 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 msgr2=0x7f64280b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 0x7f64280b8a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6430009bd0 tx=0x7f6430005f90 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 msgr2=0x7f643c19c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f643800ea60 tx=0x7f643800ef20 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 shutdown_connections 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64280b6580 0x7f64280b8a90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c10b6b0 0x7f643c19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f643c108d00 0x7f643c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 --2- 172.21.15.29:0/344800156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f643c06c490 0x7f643c19c180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 >> 172.21.15.29:0/344800156 conn(0x7f643c0fdfe0 msgr2=0x7f643c109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 shutdown_connections 2024-09-21T01:39:49.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:47.074+0000 7f6443eab640 1 -- 172.21.15.29:0/344800156 wait complete. 2024-09-21T01:39:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.572+0000 7f0de0439640 1 -- 172.21.15.29:0/3268162583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd8109470 msgr2=0x7f0dd810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.572+0000 7f0de0439640 1 --2- 172.21.15.29:0/3268162583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd8109470 0x7f0dd810b980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009ca0 tx=0x7f0dd002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 -- 172.21.15.29:0/3268162583 shutdown_connections 2024-09-21T01:39:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 --2- 172.21.15.29:0/3268162583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd8109470 0x7f0dd810b980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 --2- 172.21.15.29:0/3268162583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd81068d0 0x7f0dd8108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 --2- 172.21.15.29:0/3268162583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd806b980 0x7f0dd8106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 -- 172.21.15.29:0/3268162583 >> 172.21.15.29:0/3268162583 conn(0x7f0dd80fdfb0 msgr2=0x7f0dd8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 -- 172.21.15.29:0/3268162583 shutdown_connections 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 -- 172.21.15.29:0/3268162583 wait complete. 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 Processor -- start 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.573+0000 7f0de0439640 1 -- start start 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 0x7f0dd819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd8109470 0x7f0dd819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd810f390 con 0x7f0dd806b980 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd810f210 con 0x7f0dd8109470 2024-09-21T01:39:50.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0de0439640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd810f510 con 0x7f0dd81068d0 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53790/0 (socket says 172.21.15.29:53790) 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 -- 172.21.15.29:0/2152329195 learned_addr learned my addr 172.21.15.29:0/2152329195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde9d6640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd8109470 0x7f0dd819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0ddd9d4640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 0x7f0dd819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 -- 172.21.15.29:0/2152329195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 msgr2=0x7f0dd819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 0x7f0dd819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 -- 172.21.15.29:0/2152329195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd8109470 msgr2=0x7f0dd819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd8109470 0x7f0dd819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 -- 172.21.15.29:0/2152329195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd819b980 con 0x7f0dd806b980 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0ddd9d4640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 0x7f0dd819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dde1d5640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400e9f0 tx=0x7f0dd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.574+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400cd80 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.575+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd819bc70 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.575+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd40054d0 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.575+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4010690 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.575+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81a2370 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.576+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd80ffad0 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.577+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f0dd4004e30 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.578+0000 7f0dcf7fe640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 0x7f0dc40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.578+0000 7f0ddd9d4640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 0x7f0dc40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.578+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0dd40e5c90 con 0x7f0dd806b980 2024-09-21T01:39:50.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.579+0000 7f0ddd9d4640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 0x7f0dc40b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009b50 tx=0x7f0dc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:50.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.581+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0dd4056710 con 0x7f0dd806b980 2024-09-21T01:39:50.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.771+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f0dd802e640 con 0x7f0dd806b980 2024-09-21T01:39:50.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.775+0000 7f0dcf7fe640 1 -- 172.21.15.29:0/2152329195 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v26)=0 v26) ==== 116+0+0 (secure 0 0 0) 0x7f0dd40ae1f0 con 0x7f0dd806b980 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 msgr2=0x7f0dc40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 0x7f0dc40b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009b50 tx=0x7f0dc8005f90 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 msgr2=0x7f0dd8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400e9f0 tx=0x7f0dd400edf0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 shutdown_connections 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dc40b6480 0x7f0dc40b8990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd8109470 0x7f0dd819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd81068d0 0x7f0dd819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 --2- 172.21.15.29:0/2152329195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dd806b980 0x7f0dd8199f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.777+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 >> 172.21.15.29:0/2152329195 conn(0x7f0dd80fdfb0 msgr2=0x7f0dd8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.778+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 shutdown_connections 2024-09-21T01:39:50.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.778+0000 7f0de0439640 1 -- 172.21.15.29:0/2152329195 wait complete. 2024-09-21T01:39:50.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 -- 172.21.15.29:0/3715467962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 msgr2=0x7f0640102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 --2- 172.21.15.29:0/3715467962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640102af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0630009900 tx=0x7f063002a480 comp rx=0 tx=0).stop 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 -- 172.21.15.29:0/3715467962 shutdown_connections 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 --2- 172.21.15.29:0/3715467962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 0x7f064010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 --2- 172.21.15.29:0/3715467962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 0x7f0640103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 --2- 172.21.15.29:0/3715467962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640102af0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.894+0000 7f0645fc5640 1 -- 172.21.15.29:0/3715467962 >> 172.21.15.29:0/3715467962 conn(0x7f06400fe010 msgr2=0x7f06401004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.895+0000 7f0645fc5640 1 -- 172.21.15.29:0/3715467962 shutdown_connections 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.895+0000 7f0645fc5640 1 -- 172.21.15.29:0/3715467962 wait complete. 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.895+0000 7f0645fc5640 1 Processor -- start 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 -- start start 2024-09-21T01:39:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 0x7f0640104b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 0x7f0640105080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f064010e2c0 con 0x7f0640102680 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f064010e140 con 0x7f06401040b0 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f0645fc5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f064010e440 con 0x7f0640103710 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53798/0 (socket says 172.21.15.29:53798) 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 -- 172.21.15.29:0/4292178875 learned_addr learned my addr 172.21.15.29:0/4292178875 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063effd640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 0x7f0640104b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063ffff640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 0x7f0640105080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 -- 172.21.15.29:0/4292178875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 msgr2=0x7f0640104b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 0x7f0640104b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 -- 172.21.15.29:0/4292178875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 msgr2=0x7f0640105080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 0x7f0640105080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.896+0000 7f063f7fe640 1 -- 172.21.15.29:0/4292178875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0640105720 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f063f7fe640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063003f030 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401a40f0 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0630003cb0 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630035070 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.897+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401a4490 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.898+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06401a46f0 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.900+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f0630042030 con 0x7f0640102680 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.900+0000 7f063cff9640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 0x7f06280b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.900+0000 7f063effd640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 0x7f06280b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:50.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.900+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0630106380 con 0x7f0640102680 2024-09-21T01:39:50.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.901+0000 7f063effd640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 0x7f06280b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f063400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:50.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:50.903+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0630003e60 con 0x7f0640102680 2024-09-21T01:39:50.913 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:50.913 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:50.914 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:51.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.103+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f06401a48c0 con 0x7f0640102680 2024-09-21T01:39:51.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.108+0000 7f063cff9640 1 -- 172.21.15.29:0/4292178875 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f06300d2cc0 con 0x7f0640102680 2024-09-21T01:39:51.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 msgr2=0x7f06280b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:51.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 0x7f06280b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f063400b390 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 msgr2=0x7f0640106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630004340 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 shutdown_connections 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06280b6480 0x7f06280b8990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06401040b0 0x7f0640105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0640103710 0x7f0640104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 --2- 172.21.15.29:0/4292178875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0640102680 0x7f0640106200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 >> 172.21.15.29:0/4292178875 conn(0x7f06400fe010 msgr2=0x7f064006bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 shutdown_connections 2024-09-21T01:39:51.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:51.110+0000 7f0645fc5640 1 -- 172.21.15.29:0/4292178875 wait complete. 2024-09-21T01:39:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:51 smithi066 ceph-mon[35001]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:51 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2152329195' entity='client.admin' 2024-09-21T01:39:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:51 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4292178875' entity='client.admin' 2024-09-21T01:39:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:51 smithi029 ceph-mon[31228]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:51 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2152329195' entity='client.admin' 2024-09-21T01:39:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:51 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4292178875' entity='client.admin' 2024-09-21T01:39:52.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.095+0000 7f29c0517640 1 -- 172.21.15.29:0/3960692828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b81040a0 msgr2=0x7f29b810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.095+0000 7f29c0517640 1 --2- 172.21.15.29:0/3960692828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b81040a0 0x7f29b810aa40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ca0 tx=0x7f29b002a950 comp rx=0 tx=0).stop 2024-09-21T01:39:52.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 -- 172.21.15.29:0/3960692828 shutdown_connections 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 --2- 172.21.15.29:0/3960692828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b81040a0 0x7f29b810aa40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 --2- 172.21.15.29:0/3960692828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8103700 0x7f29b8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 --2- 172.21.15.29:0/3960692828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b8102670 0x7f29b8102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 -- 172.21.15.29:0/3960692828 >> 172.21.15.29:0/3960692828 conn(0x7f29b80fdfe0 msgr2=0x7f29b8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 -- 172.21.15.29:0/3960692828 shutdown_connections 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 -- 172.21.15.29:0/3960692828 wait complete. 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 Processor -- start 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.096+0000 7f29c0517640 1 -- start start 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8102670 0x7f29b8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b81040a0 0x7f29b819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e2c0 con 0x7f29b8103700 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e140 con 0x7f29b81040a0 2024-09-21T01:39:52.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29c0517640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e440 con 0x7f29b8102670 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57158/0 (socket says 172.21.15.29:57158) 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 -- 172.21.15.29:0/321438264 learned_addr learned my addr 172.21.15.29:0/321438264 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29be2b3640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8102670 0x7f29b8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29beab4640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b81040a0 0x7f29b819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 -- 172.21.15.29:0/321438264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8102670 msgr2=0x7f29b8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8102670 0x7f29b8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 -- 172.21.15.29:0/321438264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b81040a0 msgr2=0x7f29b819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b81040a0 0x7f29b819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 -- 172.21.15.29:0/321438264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b819b9a0 con 0x7f29b8103700 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.097+0000 7f29bdab2640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.098+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800cd80 con 0x7f29b8103700 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.098+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a80054d0 con 0x7f29b8103700 2024-09-21T01:39:52.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.098+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8010690 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.098+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b819bc90 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.098+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a2390 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.099+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b81a2560 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.101+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f29a8004e30 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.101+0000 7f29af7fe640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 0x7f29a40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.101+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f29a80e5b90 con 0x7f29b8103700 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.101+0000 7f29be2b3640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 0x7f29a40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.102+0000 7f29be2b3640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 0x7f29a40b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29b4005d00 tx=0x7f29b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:52.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.104+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29a80ae0f0 con 0x7f29b8103700 2024-09-21T01:39:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:51 smithi190 ceph-mon[34368]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:51 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2152329195' entity='client.admin' 2024-09-21T01:39:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:51 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4292178875' entity='client.admin' 2024-09-21T01:39:52.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.952+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/402319541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4105180 msgr2=0x7fa1b410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.952+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/402319541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4105180 0x7fa1b410bb20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009900 tx=0x7fa1ac02a480 comp rx=0 tx=0).stop 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/402319541 shutdown_connections 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/402319541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4105180 0x7fa1b410bb20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/402319541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b41047e0 0x7fa1b4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/402319541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4103750 0x7fa1b4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/402319541 >> 172.21.15.29:0/402319541 conn(0x7fa1b40ff0e0 msgr2=0x7fa1b4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/402319541 shutdown_connections 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/402319541 wait complete. 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 Processor -- start 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.953+0000 7fa1bc7b2640 1 -- start start 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b4103750 0x7fa1b4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4105180 0x7fa1b419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b410f3a0 con 0x7fa1b41047e0 2024-09-21T01:39:52.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b410f220 con 0x7fa1b4105180 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bc7b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b410f520 con 0x7fa1b4103750 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53706/0 (socket says 172.21.15.29:53706) 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 -- 172.21.15.29:0/3917414708 learned_addr learned my addr 172.21.15.29:0/3917414708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1bad4f640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4105180 0x7fa1b419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1ba54e640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b4103750 0x7fa1b4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 -- 172.21.15.29:0/3917414708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b4103750 msgr2=0x7fa1b4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b4103750 0x7fa1b4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 -- 172.21.15.29:0/3917414708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4105180 msgr2=0x7fa1b419b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4105180 0x7fa1b419b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 -- 172.21.15.29:0/3917414708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b419b9c0 con 0x7fa1b41047e0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b9d4d640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9f0 tx=0x7fa1a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800cd80 con 0x7fa1b41047e0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.954+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a80054d0 con 0x7fa1b41047e0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.955+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b419bcb0 con 0x7fa1b41047e0 2024-09-21T01:39:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.955+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8010690 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.955+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b41a23b0 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.956+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b410a910 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.957+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fa1a8004e90 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.958+0000 7fa1b37fe640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 0x7fa19c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.958+0000 7fa1ba54e640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 0x7fa19c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.958+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa1a80e5ac0 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.959+0000 7fa1ba54e640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 0x7fa19c0b8a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005fd0 tx=0x7fa1a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:42.963+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1a80adf80 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.153+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fa1b402e640 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.157+0000 7fa1b37fe640 1 -- 172.21.15.29:0/3917414708 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v22)=0 v22) ==== 116+0+0 (secure 0 0 0) 0x7fa1a80b2360 con 0x7fa1b41047e0 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 msgr2=0x7fa19c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 0x7fa19c0b8a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005fd0 tx=0x7fa1a4009500 comp rx=0 tx=0).stop 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 msgr2=0x7fa1b419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9f0 tx=0x7fa1a800edf0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 shutdown_connections 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa19c0b6580 0x7fa19c0b8a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1b4105180 0x7fa1b419b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b41047e0 0x7fa1b419a600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 --2- 172.21.15.29:0/3917414708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b4103750 0x7fa1b4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 >> 172.21.15.29:0/3917414708 conn(0x7fa1b40ff0e0 msgr2=0x7fa1b4107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 shutdown_connections 2024-09-21T01:39:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:43.160+0000 7fa1bc7b2640 1 -- 172.21.15.29:0/3917414708 wait complete. 2024-09-21T01:39:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.294+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f29b81a2be0 con 0x7f29b8103700 2024-09-21T01:39:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.298+0000 7f29af7fe640 1 -- 172.21.15.29:0/321438264 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v28)=0 v28) ==== 127+0+0 (secure 0 0 0) 0x7f29a80b24d0 con 0x7f29b8103700 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 msgr2=0x7f29a40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 0x7f29a40b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29b4005d00 tx=0x7f29b400b410 comp rx=0 tx=0).stop 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 msgr2=0x7f29b819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800edf0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 shutdown_connections 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f29a40b6480 0x7f29a40b8990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b81040a0 0x7f29b819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29b8103700 0x7f29b819a5e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 --2- 172.21.15.29:0/321438264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b8102670 0x7f29b8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:52.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.300+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 >> 172.21.15.29:0/321438264 conn(0x7f29b80fdfe0 msgr2=0x7f29b806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:52.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.301+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 shutdown_connections 2024-09-21T01:39:52.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:52.301+0000 7f29c0517640 1 -- 172.21.15.29:0/321438264 wait complete. 2024-09-21T01:39:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:53 smithi066 ceph-mon[35001]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:53 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/321438264' entity='client.admin' 2024-09-21T01:39:53.563 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:53 smithi029 ceph-mon[31228]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:53.563 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:53 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/321438264' entity='client.admin' 2024-09-21T01:39:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:53 smithi190 ceph-mon[34368]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:53 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/321438264' entity='client.admin' 2024-09-21T01:39:54.212 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:55.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:55.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:55.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:55.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:56.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:56.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/29261105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f681036d0 msgr2=0x7f6f68103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/29261105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f681036d0 0x7f6f68103ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009900 tx=0x7f6f5c02a480 comp rx=0 tx=0).stop 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/29261105 shutdown_connections 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/29261105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f68104070 0x7f6f6810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/29261105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f681036d0 0x7f6f68103ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/29261105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68102640 0x7f6f68102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/29261105 >> 172.21.15.29:0/29261105 conn(0x7f6f680fdfb0 msgr2=0x7f6f68100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/29261105 shutdown_connections 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.006+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/29261105 wait complete. 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.007+0000 7f6f6ce2e640 1 Processor -- start 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 -- start start 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:56.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f681036d0 0x7f6f6819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 0x7f6f681a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6810e270 con 0x7f6f68102640 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6810e0f0 con 0x7f6f681036d0 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6ce2e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6810e3f0 con 0x7f6f68104070 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57166/0 (socket says 172.21.15.29:57166) 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f6659c640 1 -- 172.21.15.29:0/199357760 learned_addr learned my addr 172.21.15.29:0/199357760 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f66d9d640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 0x7f6f681a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.008+0000 7f6f65d9b640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f681036d0 0x7f6f6819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 -- 172.21.15.29:0/199357760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 msgr2=0x7f6f681a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 0x7f6f681a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 -- 172.21.15.29:0/199357760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f681036d0 msgr2=0x7f6f6819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f681036d0 0x7f6f6819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 -- 172.21.15.29:0/199357760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f681a1a10 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f66d9d640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 0x7f6f681a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6659c640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d950 tx=0x7f6f5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58017070 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f681a1d00 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58009c30 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800f620 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.009+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a21d0 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.010+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f68102ab0 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.012+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f6f58010460 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.013+0000 7f6f577fe640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 0x7f6f500b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.013+0000 7f6f65d9b640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 0x7f6f500b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.013+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f580e51b0 con 0x7f6f68102640 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.013+0000 7f6f65d9b640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 0x7f6f500b8990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:56.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.016+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6f58009de0 con 0x7f6f68102640 2024-09-21T01:39:56.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.208+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f6f68109800 con 0x7f6f68102640 2024-09-21T01:39:56.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.212+0000 7f6f577fe640 1 -- 172.21.15.29:0/199357760 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v29)=0 v29) ==== 125+0+0 (secure 0 0 0) 0x7f6f580b1af0 con 0x7f6f68102640 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 msgr2=0x7f6f500b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 0x7f6f500b8990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c031400 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 msgr2=0x7f6f6806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d950 tx=0x7f6f5800de10 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 shutdown_connections 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6f500b6480 0x7f6f500b8990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f68104070 0x7f6f681a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f681036d0 0x7f6f6819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 --2- 172.21.15.29:0/199357760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f68102640 0x7f6f6806b630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:56.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 >> 172.21.15.29:0/199357760 conn(0x7f6f680fdfb0 msgr2=0x7f6f681069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:56.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 shutdown_connections 2024-09-21T01:39:56.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:56.215+0000 7f6f6ce2e640 1 -- 172.21.15.29:0/199357760 wait complete. 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.225+0000 7fb8e63ab640 1 -- 172.21.15.29:0/1899851799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e01069a0 msgr2=0x7fb8e0108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.225+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/1899851799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e01069a0 0x7fb8e0108dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009900 tx=0x7fb8d402a410 comp rx=0 tx=0).stop 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 -- 172.21.15.29:0/1899851799 shutdown_connections 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/1899851799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e0109480 0x7fb8e010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/1899851799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e01069a0 0x7fb8e0108dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/1899851799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 -- 172.21.15.29:0/1899851799 >> 172.21.15.29:0/1899851799 conn(0x7fb8e00fdfe0 msgr2=0x7fb8e0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 -- 172.21.15.29:0/1899851799 shutdown_connections 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 -- 172.21.15.29:0/1899851799 wait complete. 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.226+0000 7fb8e63ab640 1 Processor -- start 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 -- start start 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e01069a0 0x7fb8e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e0109480 0x7fb8e019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010f390 con 0x7fb8e006c5a0 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010f210 con 0x7fb8e01069a0 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e63ab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010f510 con 0x7fb8e0109480 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:57.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57178/0 (socket says 172.21.15.29:57178) 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8dffff640 1 -- 172.21.15.29:0/2421790323 learned_addr learned my addr 172.21.15.29:0/2421790323 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8df7fe640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e01069a0 0x7fb8e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.227+0000 7fb8e4948640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e0109480 0x7fb8e019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 -- 172.21.15.29:0/2421790323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e0109480 msgr2=0x7fb8e019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e0109480 0x7fb8e019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 -- 172.21.15.29:0/2421790323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e01069a0 msgr2=0x7fb8e019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e01069a0 0x7fb8e019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 -- 172.21.15.29:0/2421790323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e019ba20 con 0x7fb8e006c5a0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dffff640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000d770 tx=0x7fb8d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0017070 con 0x7fb8e006c5a0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d00054d0 con 0x7fb8e006c5a0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e019bd10 con 0x7fb8e006c5a0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004c30 con 0x7fb8e006c5a0 2024-09-21T01:39:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.228+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a2410 con 0x7fb8e006c5a0 2024-09-21T01:39:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.229+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e0106db0 con 0x7fb8e006c5a0 2024-09-21T01:39:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.231+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb8d0004dd0 con 0x7fb8e006c5a0 2024-09-21T01:39:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.232+0000 7fb8dd7fa640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 0x7fb8c80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.232+0000 7fb8df7fe640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 0x7fb8c80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.232+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8d00e5500 con 0x7fb8e006c5a0 2024-09-21T01:39:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.233+0000 7fb8df7fe640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 0x7fb8c80b8990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000c00 tx=0x7fb8d4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:57.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.236+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8d00ada60 con 0x7fb8e006c5a0 2024-09-21T01:39:57.239 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:39:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/199357760' entity='client.admin' 2024-09-21T01:39:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:57.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:57.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:57.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:57.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.437+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fb8e00ffb00 con 0x7fb8e006c5a0 2024-09-21T01:39:57.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.441+0000 7fb8dd7fa640 1 -- 172.21.15.29:0/2421790323 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7fb8e00ffb00 con 0x7fb8e006c5a0 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 msgr2=0x7fb8c80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 0x7fb8c80b8990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000c00 tx=0x7fb8d4031400 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 msgr2=0x7fb8e0199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000d770 tx=0x7fb8d000dc30 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 shutdown_connections 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8c80b6480 0x7fb8c80b8990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e0109480 0x7fb8e019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e01069a0 0x7fb8e019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 --2- 172.21.15.29:0/2421790323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8e006c5a0 0x7fb8e0199ff0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 >> 172.21.15.29:0/2421790323 conn(0x7fb8e00fdfe0 msgr2=0x7fb8e0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 shutdown_connections 2024-09-21T01:39:57.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:57.444+0000 7fb8e63ab640 1 -- 172.21.15.29:0/2421790323 wait complete. 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/199357760' entity='client.admin' 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/199357760' entity='client.admin' 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:57.699 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.699 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.699 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi029') 2024-09-21T01:39:57.700 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi029') 2024-09-21T01:39:58.612 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2421790323' entity='client.admin' 2024-09-21T01:39:58.612 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:58.612 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:58.612 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:58.612 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:58 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2421790323' entity='client.admin' 2024-09-21T01:39:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:58 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2421790323' entity='client.admin' 2024-09-21T01:39:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:39:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:39:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:39:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:39:58.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 -- 172.21.15.29:0/3192935788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce81047e0 msgr2=0x7f7ce8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3192935788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce81047e0 0x7f7ce8104bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009900 tx=0x7f7cd002a480 comp rx=0 tx=0).stop 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 -- 172.21.15.29:0/3192935788 shutdown_connections 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3192935788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8105180 0x7f7ce810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3192935788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce81047e0 0x7f7ce8104bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3192935788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8103750 0x7f7ce8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 -- 172.21.15.29:0/3192935788 >> 172.21.15.29:0/3192935788 conn(0x7f7ce80ff0e0 msgr2=0x7f7ce8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 -- 172.21.15.29:0/3192935788 shutdown_connections 2024-09-21T01:39:58.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.831+0000 7f7ced83a640 1 -- 172.21.15.29:0/3192935788 wait complete. 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 Processor -- start 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 -- start start 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce81047e0 0x7f7ce819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8105180 0x7f7ce819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810f450 con 0x7f7ce8103750 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810f2d0 con 0x7f7ce8105180 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ced83a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810f5d0 con 0x7f7ce81047e0 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ce6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.832+0000 7f7ce6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57194/0 (socket says 172.21.15.29:57194) 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 -- 172.21.15.29:0/3281642280 learned_addr learned my addr 172.21.15.29:0/3281642280 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce77fe640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8105180 0x7f7ce819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce67fc640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce81047e0 0x7f7ce819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 -- 172.21.15.29:0/3281642280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce81047e0 msgr2=0x7f7ce819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce81047e0 0x7f7ce819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 -- 172.21.15.29:0/3281642280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8105180 msgr2=0x7f7ce819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8105180 0x7f7ce819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 -- 172.21.15.29:0/3281642280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce819ba60 con 0x7f7ce8103750 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.833+0000 7f7ce6ffd640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800ebc0 tx=0x7f7cd800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.834+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8010630 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.834+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd8010c80 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.834+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce819bd50 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.834+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8018b60 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.834+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81a2450 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.835+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce8103bc0 con 0x7f7ce8103750 2024-09-21T01:39:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.836+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f7cd8018d00 con 0x7f7ce8103750 2024-09-21T01:39:58.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.837+0000 7f7cec838640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 0x7f7cd40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:58.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.837+0000 7f7ce67fc640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 0x7f7cd40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.837+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7cd80e5bb0 con 0x7f7ce8103750 2024-09-21T01:39:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.838+0000 7f7ce67fc640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 0x7f7cd40b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0004770 tx=0x7f7cd00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:58.842+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cd80b24f0 con 0x7f7ce8103750 2024-09-21T01:39:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.034+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f7ce810a910 con 0x7f7ce8103750 2024-09-21T01:39:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.038+0000 7f7cec838640 1 -- 172.21.15.29:0/3281642280 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f7cd80b7430 con 0x7f7ce8103750 2024-09-21T01:39:59.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 msgr2=0x7f7cd40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 0x7f7cd40b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0004770 tx=0x7f7cd00314a0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 msgr2=0x7f7ce819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800ebc0 tx=0x7f7cd800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 shutdown_connections 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7cd40b6480 0x7f7cd40b8990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce8105180 0x7f7ce819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce81047e0 0x7f7ce819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 --2- 172.21.15.29:0/3281642280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce8103750 0x7f7ce819a0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 >> 172.21.15.29:0/3281642280 conn(0x7f7ce80ff0e0 msgr2=0x7f7ce8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 shutdown_connections 2024-09-21T01:39:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.042+0000 7f7ced83a640 1 -- 172.21.15.29:0/3281642280 wait complete. 2024-09-21T01:39:59.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.067+0000 7fca31f80640 1 -- 172.21.15.29:0/3946290928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 msgr2=0x7fca2c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.067+0000 7fca31f80640 1 --2- 172.21.15.29:0/3946290928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c102ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009900 tx=0x7fca1c02a480 comp rx=0 tx=0).stop 2024-09-21T01:39:59.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.067+0000 7fca31f80640 1 -- 172.21.15.29:0/3946290928 shutdown_connections 2024-09-21T01:39:59.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 --2- 172.21.15.29:0/3946290928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 --2- 172.21.15.29:0/3946290928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 0x7fca2c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 --2- 172.21.15.29:0/3946290928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c102ab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 -- 172.21.15.29:0/3946290928 >> 172.21.15.29:0/3946290928 conn(0x7fca2c0fdfb0 msgr2=0x7fca2c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:59.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 -- 172.21.15.29:0/3946290928 shutdown_connections 2024-09-21T01:39:59.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.068+0000 7fca31f80640 1 -- 172.21.15.29:0/3946290928 wait complete. 2024-09-21T01:39:59.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.069+0000 7fca31f80640 1 Processor -- start 2024-09-21T01:39:59.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 -- start start 2024-09-21T01:39:59.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 0x7fca2c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10e2b0 con 0x7fca2c102640 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10e130 con 0x7fca2c104070 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.070+0000 7fca31f80640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10e430 con 0x7fca2c1036d0 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57216/0 (socket says 172.21.15.29:57216) 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2b7fe640 1 -- 172.21.15.29:0/2323435787 learned_addr learned my addr 172.21.15.29:0/2323435787 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2affd640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 0x7fca2c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 -- 172.21.15.29:0/2323435787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 msgr2=0x7fca2c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 0x7fca2c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 -- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 msgr2=0x7fca2c06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.071+0000 7fca2bfff640 1 -- 172.21.15.29:0/2323435787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c1a1a40 con 0x7fca2c104070 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca2b7fe640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca2bfff640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c1a1240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca24009fd0 tx=0x7fca2400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400ce20 con 0x7fca2c104070 2024-09-21T01:39:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca24005020 con 0x7fca2c104070 2024-09-21T01:39:59.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24010690 con 0x7fca2c104070 2024-09-21T01:39:59.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c1a1d30 con 0x7fca2c104070 2024-09-21T01:39:59.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.072+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c1a2200 con 0x7fca2c104070 2024-09-21T01:39:59.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.074+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c109820 con 0x7fca2c104070 2024-09-21T01:39:59.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.076+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fca240054d0 con 0x7fca2c104070 2024-09-21T01:39:59.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.077+0000 7fca28ff9640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 0x7fca140b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.077+0000 7fca2b7fe640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 0x7fca140b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.077+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca240e5c90 con 0x7fca2c104070 2024-09-21T01:39:59.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.078+0000 7fca2b7fe640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 0x7fca140b8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005710 tx=0x7fca1c0043f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:59.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.080+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca240ae150 con 0x7fca2c104070 2024-09-21T01:39:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.268+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fca2c02e640 con 0x7fca2c104070 2024-09-21T01:39:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.270+0000 7fca28ff9640 1 -- 172.21.15.29:0/2323435787 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v31)=0 v31) ==== 108+0+0 (secure 0 0 0) 0x7fca240b2530 con 0x7fca2c104070 2024-09-21T01:39:59.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 msgr2=0x7fca140b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 0x7fca140b8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005710 tx=0x7fca1c0043f0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 msgr2=0x7fca2c1a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c1a1240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca24009fd0 tx=0x7fca2400ef10 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 shutdown_connections 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fca140b65c0 0x7fca140b8ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca2c104070 0x7fca2c1a1240 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c1036d0 0x7fca2c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 --2- 172.21.15.29:0/2323435787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca2c102640 0x7fca2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 >> 172.21.15.29:0/2323435787 conn(0x7fca2c0fdfb0 msgr2=0x7fca2c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 shutdown_connections 2024-09-21T01:39:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.273+0000 7fca31f80640 1 -- 172.21.15.29:0/2323435787 wait complete. 2024-09-21T01:39:59.563 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:59 smithi029 ceph-mon[31228]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:59.563 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:39:59 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3281642280' entity='client.admin' 2024-09-21T01:39:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:59 smithi190 ceph-mon[34368]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:39:59 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3281642280' entity='client.admin' 2024-09-21T01:39:59.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 -- 172.21.15.29:0/1127372550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c102670 msgr2=0x7f101c102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 --2- 172.21.15.29:0/1127372550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c102670 0x7f101c102ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1018009900 tx=0x7f101802a480 comp rx=0 tx=0).stop 2024-09-21T01:39:59.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 -- 172.21.15.29:0/1127372550 shutdown_connections 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 --2- 172.21.15.29:0/1127372550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c1040a0 0x7f101c10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 --2- 172.21.15.29:0/1127372550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c103700 0x7f101c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 --2- 172.21.15.29:0/1127372550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c102670 0x7f101c102ae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 -- 172.21.15.29:0/1127372550 >> 172.21.15.29:0/1127372550 conn(0x7f101c0fdfe0 msgr2=0x7f101c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 -- 172.21.15.29:0/1127372550 shutdown_connections 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.779+0000 7f1024626640 1 -- 172.21.15.29:0/1127372550 wait complete. 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 Processor -- start 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 -- start start 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 0x7f101c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 0x7f101c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10e310 con 0x7f101c103700 2024-09-21T01:39:59.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10e190 con 0x7f101c102670 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1024626640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10e490 con 0x7f101c1040a0 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1021bc1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1021bc1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57236/0 (socket says 172.21.15.29:57236) 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.780+0000 7f1021bc1640 1 -- 172.21.15.29:0/3408805930 learned_addr learned my addr 172.21.15.29:0/3408805930 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f10223c2640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 0x7f101c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1022bc3640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 0x7f101c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 -- 172.21.15.29:0/3408805930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 msgr2=0x7f101c19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 0x7f101c19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 -- 172.21.15.29:0/3408805930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 msgr2=0x7f101c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 0x7f101c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 -- 172.21.15.29:0/3408805930 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c19bac0 con 0x7f101c103700 2024-09-21T01:39:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1022bc3640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 0x7f101c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f10223c2640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 0x7f101c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1021bc1640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f100c00b4a0 tx=0x7f100c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c014070 con 0x7f101c103700 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c19bdb0 con 0x7f101c103700 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100c010ac0 con 0x7f101c103700 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c017c30 con 0x7f101c103700 2024-09-21T01:39:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.781+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c1a24b0 con 0x7f101c103700 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.782+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c1a2680 con 0x7f101c103700 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.784+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f100c017070 con 0x7f101c103700 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.785+0000 7f10137fe640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 0x7f10080b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.785+0000 7f10223c2640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 0x7f10080b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.785+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f100c0e4d50 con 0x7f101c103700 2024-09-21T01:39:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.786+0000 7f10223c2640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 0x7f10080b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1018031440 tx=0x7f1018005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:39:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.788+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f100c0ad2b0 con 0x7f101c103700 2024-09-21T01:39:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:59 smithi066 ceph-mon[35001]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:39:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:39:59 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3281642280' entity='client.admin' 2024-09-21T01:39:59.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.985+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f101c1a2850 con 0x7f101c103700 2024-09-21T01:39:59.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.990+0000 7f10137fe640 1 -- 172.21.15.29:0/3408805930 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v32)=0 v32) ==== 111+0+0 (secure 0 0 0) 0x7f100c0b1690 con 0x7f101c103700 2024-09-21T01:39:59.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.992+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 msgr2=0x7f10080b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.992+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 0x7f10080b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1018031440 tx=0x7f1018005d50 comp rx=0 tx=0).stop 2024-09-21T01:39:59.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.992+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 msgr2=0x7f101c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:39:59.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.992+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f100c00b4a0 tx=0x7f100c00b960 comp rx=0 tx=0).stop 2024-09-21T01:39:59.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 shutdown_connections 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f10080b6480 0x7f10080b8990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101c1040a0 0x7f101c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f101c103700 0x7f101c19a700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 --2- 172.21.15.29:0/3408805930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f101c102670 0x7f101c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 >> 172.21.15.29:0/3408805930 conn(0x7f101c0fdfe0 msgr2=0x7f101c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 shutdown_connections 2024-09-21T01:39:59.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:39:59.993+0000 7f1024626640 1 -- 172.21.15.29:0/3408805930 wait complete. 2024-09-21T01:40:00.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 -- 172.21.15.29:0/440790637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d4102640 msgr2=0x7f05d4102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:00.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 --2- 172.21.15.29:0/440790637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d4102640 0x7f05d4102ab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009900 tx=0x7f05cc02a480 comp rx=0 tx=0).stop 2024-09-21T01:40:00.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 -- 172.21.15.29:0/440790637 shutdown_connections 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 --2- 172.21.15.29:0/440790637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 0x7f05d410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 --2- 172.21.15.29:0/440790637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d41036d0 0x7f05d4103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 --2- 172.21.15.29:0/440790637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d4102640 0x7f05d4102ab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 -- 172.21.15.29:0/440790637 >> 172.21.15.29:0/440790637 conn(0x7f05d40fdfb0 msgr2=0x7f05d4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 -- 172.21.15.29:0/440790637 shutdown_connections 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.546+0000 7f05db4b3640 1 -- 172.21.15.29:0/440790637 wait complete. 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.547+0000 7f05db4b3640 1 Processor -- start 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.547+0000 7f05db4b3640 1 -- start start 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.547+0000 7f05db4b3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d4102640 0x7f05d4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.547+0000 7f05db4b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.547+0000 7f05db4b3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 0x7f05d419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:00.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05db4b3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05d410e280 con 0x7f05d41036d0 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05db4b3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05d410e100 con 0x7f05d4102640 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05db4b3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05d410e400 con 0x7f05d4104070 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57258/0 (socket says 172.21.15.29:57258) 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 -- 172.21.15.29:0/1053321485 learned_addr learned my addr 172.21.15.29:0/1053321485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d924f640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d4102640 0x7f05d4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 -- 172.21.15.29:0/1053321485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 msgr2=0x7f05d419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d9a50640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 0x7f05d419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 0x7f05d419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 -- 172.21.15.29:0/1053321485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d4102640 msgr2=0x7f05d4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d4102640 0x7f05d4199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 -- 172.21.15.29:0/1053321485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d419ba00 con 0x7f05d41036d0 2024-09-21T01:40:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d8a4e640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05c800e9f0 tx=0x7f05c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:00.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c800cd80 con 0x7f05d41036d0 2024-09-21T01:40:00.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05c80108e0 con 0x7f05d41036d0 2024-09-21T01:40:00.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d419bcf0 con 0x7f05d41036d0 2024-09-21T01:40:00.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8018980 con 0x7f05d41036d0 2024-09-21T01:40:00.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.548+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d41a23f0 con 0x7f05d41036d0 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.550+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d41a25c0 con 0x7f05d41036d0 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.552+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f05c8010430 con 0x7f05d41036d0 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.553+0000 7f05d27fc640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 0x7f05bc0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.553+0000 7f05d924f640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 0x7f05bc0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.553+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f05c80e5a00 con 0x7f05d41036d0 2024-09-21T01:40:00.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.554+0000 7f05d924f640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 0x7f05bc0b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:00.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.558+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f05c80adec0 con 0x7f05d41036d0 2024-09-21T01:40:00.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.747+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f05d41a2790 con 0x7f05d41036d0 2024-09-21T01:40:00.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.752+0000 7f05d27fc640 1 -- 172.21.15.29:0/1053321485 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7f05c80b22a0 con 0x7f05d41036d0 2024-09-21T01:40:00.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 msgr2=0x7f05bc0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 0x7f05bc0b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc031400 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 msgr2=0x7f05d419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05c800e9f0 tx=0x7f05c800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 shutdown_connections 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05bc0b6500 0x7f05bc0b8a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4104070 0x7f05d419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05d41036d0 0x7f05d419a640 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 --2- 172.21.15.29:0/1053321485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05d4102640 0x7f05d4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 >> 172.21.15.29:0/1053321485 conn(0x7f05d40fdfb0 msgr2=0x7f05d40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 shutdown_connections 2024-09-21T01:40:00.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:00.754+0000 7f05db4b3640 1 -- 172.21.15.29:0/1053321485 wait complete. 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3408805930' entity='client.admin' 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: overall HEALTH_OK 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:01.265 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1053321485' entity='client.admin' 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3408805930' entity='client.admin' 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: overall HEALTH_OK 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1053321485' entity='client.admin' 2024-09-21T01:40:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3408805930' entity='client.admin' 2024-09-21T01:40:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: overall HEALTH_OK 2024-09-21T01:40:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1053321485' entity='client.admin' 2024-09-21T01:40:02.163 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi029') 2024-09-21T01:40:02.163 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi029') 2024-09-21T01:40:02.164 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi029') 2024-09-21T01:40:02.405 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:40:03.273 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:03 smithi029 ceph-mon[31228]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:03.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:03 smithi066 ceph-mon[35001]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:03.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:03 smithi190 ceph-mon[34368]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.521+0000 7fb61b638640 1 -- 172.21.15.29:0/2266712239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614102670 msgr2=0x7fb614102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.521+0000 7fb61b638640 1 --2- 172.21.15.29:0/2266712239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614102670 0x7fb614102ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb610009900 tx=0x7fb61002a480 comp rx=0 tx=0).stop 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 -- 172.21.15.29:0/2266712239 shutdown_connections 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 --2- 172.21.15.29:0/2266712239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6141040a0 0x7fb61410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 --2- 172.21.15.29:0/2266712239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb614103700 0x7fb614103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 --2- 172.21.15.29:0/2266712239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614102670 0x7fb614102ae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 -- 172.21.15.29:0/2266712239 >> 172.21.15.29:0/2266712239 conn(0x7fb6140fdfe0 msgr2=0x7fb614100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 -- 172.21.15.29:0/2266712239 shutdown_connections 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 -- 172.21.15.29:0/2266712239 wait complete. 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 Processor -- start 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.522+0000 7fb61b638640 1 -- start start 2024-09-21T01:40:03.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb614102670 0x7fb61419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 0x7fb61419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e2b0 con 0x7fb614103700 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e130 con 0x7fb6141040a0 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb61b638640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e430 con 0x7fb614102670 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36826/0 (socket says 172.21.15.29:36826) 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb6193d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb614102670 0x7fb61419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 -- 172.21.15.29:0/299333955 learned_addr learned my addr 172.21.15.29:0/299333955 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb619bd5640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 0x7fb61419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 -- 172.21.15.29:0/299333955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb614102670 msgr2=0x7fb61419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb614102670 0x7fb61419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 -- 172.21.15.29:0/299333955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 msgr2=0x7fb61419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:03.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 0x7fb61419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb618bd3640 1 -- 172.21.15.29:0/299333955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61419ba30 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.523+0000 7fb619bd5640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 0x7fb61419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb618bd3640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb60400d950 tx=0x7fb60400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604017030 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb604004020 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6040047a0 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61419bd20 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.524+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6141a2420 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.525+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6141a25f0 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.526+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb604009a30 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.528+0000 7fb60a7fc640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 0x7fb6000b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.528+0000 7fb6193d4640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 0x7fb6000b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.528+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb6040e5740 con 0x7fb614103700 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.528+0000 7fb6193d4640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 0x7fb6000b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:03.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.531+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6040adc80 con 0x7fb614103700 2024-09-21T01:40:03.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.719+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fb6141a27c0 con 0x7fb614103700 2024-09-21T01:40:03.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.719+0000 7fb60a7fc640 1 -- 172.21.15.29:0/299333955 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v33)=0 v33) ==== 106+0+0 (secure 0 0 0) 0x7fb6040b2060 con 0x7fb614103700 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 msgr2=0x7fb6000b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 0x7fb6000b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610031400 comp rx=0 tx=0).stop 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 msgr2=0x7fb61419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb60400d950 tx=0x7fb60400de10 comp rx=0 tx=0).stop 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 shutdown_connections 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6000b6480 0x7fb6000b8990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6141040a0 0x7fb61419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614103700 0x7fb61419a670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 --2- 172.21.15.29:0/299333955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb614102670 0x7fb61419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 >> 172.21.15.29:0/299333955 conn(0x7fb6140fdfe0 msgr2=0x7fb6140ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 shutdown_connections 2024-09-21T01:40:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:03.722+0000 7fb61b638640 1 -- 172.21.15.29:0/299333955 wait complete. 2024-09-21T01:40:04.428 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi029') 2024-09-21T01:40:04.429 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi029') 2024-09-21T01:40:04.429 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e38bc6b6-77b9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi029') 2024-09-21T01:40:04.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T01:40:04.685 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:40:05.059 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:05 smithi029 ceph-mon[31228]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 -- 172.21.15.29:0/4164602984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 msgr2=0x7fc71c10b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 --2- 172.21.15.29:0/4164602984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c10b990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc714009ca0 tx=0x7fc71402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 -- 172.21.15.29:0/4164602984 shutdown_connections 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 --2- 172.21.15.29:0/4164602984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c10b990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 --2- 172.21.15.29:0/4164602984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c106940 0x7fc71c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 --2- 172.21.15.29:0/4164602984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06c600 0x7fc71c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 -- 172.21.15.29:0/4164602984 >> 172.21.15.29:0/4164602984 conn(0x7fc71c0fdfe0 msgr2=0x7fc71c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 -- 172.21.15.29:0/4164602984 shutdown_connections 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.150+0000 7fc722b2d640 1 -- 172.21.15.29:0/4164602984 wait complete. 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 Processor -- start 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 -- start start 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c06c600 0x7fc71c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c106940 0x7fc71c102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10f370 con 0x7fc71c109480 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10f1f0 con 0x7fc71c106940 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc722b2d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10f4f0 con 0x7fc71c06c600 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:05.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36838/0 (socket says 172.21.15.29:36838) 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 -- 172.21.15.29:0/489751397 learned_addr learned my addr 172.21.15.29:0/489751397 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7208c9640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c06c600 0x7fc71c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc71bfff640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c106940 0x7fc71c102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 -- 172.21.15.29:0/489751397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c06c600 msgr2=0x7fc71c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c06c600 0x7fc71c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 -- 172.21.15.29:0/489751397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c106940 msgr2=0x7fc71c102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c106940 0x7fc71c102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.151+0000 7fc7210ca640 1 -- 172.21.15.29:0/489751397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c102f40 con 0x7fc71c109480 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc7210ca640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7140097b0 tx=0x7fc714003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714004240 con 0x7fc71c109480 2024-09-21T01:40:05.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc71402f380 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c1a3ff0 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71402fc30 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.152+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a4390 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.153+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c106d50 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.155+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fc7140043e0 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.156+0000 7fc719ffb640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 0x7fc7040b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.156+0000 7fc7208c9640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 0x7fc7040b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.156+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc714106ba0 con 0x7fc71c109480 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.157+0000 7fc7208c9640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 0x7fc7040b8a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00bbe0 tx=0x7fc70c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:05.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.160+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc714106e20 con 0x7fc71c109480 2024-09-21T01:40:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:05 smithi066 ceph-mon[35001]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.351+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc71c0ffb00 con 0x7fc71c109480 2024-09-21T01:40:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.352+0000 7fc719ffb640 1 -- 172.21.15.29:0/489751397 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+21518 (secure 0 0 0) 0x7fc71c0ffb00 con 0x7fc71c109480 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global dev auth_debug true 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global basic log_to_file true 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global basic log_to_journald false 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global basic log_to_stderr false 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_allow_pool_delete true 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T01:40:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global dev ms_die_on_bug true 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global dev ms_die_on_old_message true 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global dev ms_die_on_skipped_message false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T01:40:05.355 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced debug_mon 20/20 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced debug_ms 1/1 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced debug_paxos 20/20 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T01:40:05.356 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced client_mount_timeout 30 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced debug_client 20/20 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced debug_mgr 20/20 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced debug_ms 1/1 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:40:05.357 INFO:teuthology.orchestra.run.smithi029.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev bdev_debug_aio true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced bdev_enable_discard true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced debug_bluefs 1/20 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced debug_bluestore 1/20 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced debug_ms 1/1 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced debug_osd 20/20 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_op_order true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T01:40:05.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_shutdown true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T01:40:05.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T01:40:05.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T01:40:05.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T01:40:05.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced debug_mds 20/20 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced debug_ms 2/2 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T01:40:05.361 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_bal_split_size 100 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds dev mds_debug_frag true 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds dev mds_debug_scatterstat true 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds dev mds_verify_scatter true 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:client advanced client_mount_timeout 600 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:client advanced debug_client 20/20 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:client advanced debug_ms 1/1 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T01:40:05.362 INFO:teuthology.orchestra.run.smithi029.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T01:40:05.363 INFO:teuthology.orchestra.run.smithi029.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T01:40:05.363 INFO:teuthology.orchestra.run.smithi029.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T01:40:05.363 INFO:teuthology.orchestra.run.smithi029.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.355+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 msgr2=0x7fc7040b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.355+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 0x7fc7040b8a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00bbe0 tx=0x7fc70c00b390 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.355+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 msgr2=0x7fc71c102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.355+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7140097b0 tx=0x7fc714003c20 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 shutdown_connections 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7040b6500 0x7fc7040b8a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc71c109480 0x7fc71c102840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c106940 0x7fc71c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 --2- 172.21.15.29:0/489751397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c06c600 0x7fc71c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 >> 172.21.15.29:0/489751397 conn(0x7fc71c0fdfe0 msgr2=0x7fc71c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 shutdown_connections 2024-09-21T01:40:05.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:05.356+0000 7fc722b2d640 1 -- 172.21.15.29:0/489751397 wait complete. 2024-09-21T01:40:05.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:05 smithi190 ceph-mon[34368]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:06.091 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/489751397' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:40:06.092 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T01:40:06.092 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:40:06.092 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:40:06.109 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:40:06.109 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-09-21T01:40:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:40:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-21 01:38:44.869118632 +0000 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-21 01:33:46.954756856 +0000 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-21 01:33:46.954756856 +0000 2024-09-21T01:40:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-21 01:33:46.954756856 +0000 2024-09-21T01:40:06.166 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:40:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-21T01:40:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-21T01:40:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.00011304 s, 4.5 MB/s 2024-09-21T01:40:06.233 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:40:06.290 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-09-21T01:40:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/489751397' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-21 01:38:44.906118060 +0000 2024-09-21T01:40:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-21 01:33:47.269751992 +0000 2024-09-21T01:40:06.347 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-21 01:33:47.269751992 +0000 2024-09-21T01:40:06.347 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-21 01:33:47.269751992 +0000 2024-09-21T01:40:06.347 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:40:06.410 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-21T01:40:06.410 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-21T01:40:06.411 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000112141 s, 4.6 MB/s 2024-09-21T01:40:06.412 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:40:06.469 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-09-21T01:40:06.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/489751397' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:06.525 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-21 01:38:44.944117474 +0000 2024-09-21T01:40:06.526 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-21 01:33:47.597746928 +0000 2024-09-21T01:40:06.526 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-21 01:33:47.597746928 +0000 2024-09-21T01:40:06.526 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-21 01:33:47.597746928 +0000 2024-09-21T01:40:06.526 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:40:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-21T01:40:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-21T01:40:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000109291 s, 4.7 MB/s 2024-09-21T01:40:06.590 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:40:06.648 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-21 01:38:44.986116825 +0000 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-21 01:33:47.927741832 +0000 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-21 01:33:47.927741832 +0000 2024-09-21T01:40:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-21 01:33:47.927741832 +0000 2024-09-21T01:40:06.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:40:06.767 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-21T01:40:06.767 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-21T01:40:06.767 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000104265 s, 4.9 MB/s 2024-09-21T01:40:06.769 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:40:06.826 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-21T01:40:06.826 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:40:06.846 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:40:06.847 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-21 01:39:34.801100530 +0000 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-21 01:33:54.609583333 +0000 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-21 01:33:54.609583333 +0000 2024-09-21T01:40:06.906 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-21 01:33:54.609583333 +0000 2024-09-21T01:40:06.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:40:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-21T01:40:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-21T01:40:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000283715 s, 1.8 MB/s 2024-09-21T01:40:06.978 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:40:07.038 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-09-21T01:40:07.095 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:40:07.095 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:07.095 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-21 01:39:34.801100530 +0000 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-21 01:33:54.934581210 +0000 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-21 01:33:54.934581210 +0000 2024-09-21T01:40:07.096 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-21 01:33:54.934581210 +0000 2024-09-21T01:40:07.096 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:40:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-21T01:40:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-21T01:40:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000237004 s, 2.2 MB/s 2024-09-21T01:40:07.164 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:40:07.221 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-21 01:39:34.802100523 +0000 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-21 01:33:55.269579022 +0000 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-21 01:33:55.269579022 +0000 2024-09-21T01:40:07.279 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-21 01:33:55.269579022 +0000 2024-09-21T01:40:07.280 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:40:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:07 smithi066 ceph-mon[35001]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-21T01:40:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-21T01:40:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00019274 s, 2.7 MB/s 2024-09-21T01:40:07.325 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:40:07.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:07 smithi029 ceph-mon[31228]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:07.383 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-09-21T01:40:07.440 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-21 01:39:34.802100523 +0000 2024-09-21T01:40:07.441 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-21 01:33:55.606576821 +0000 2024-09-21T01:40:07.442 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-21 01:33:55.606576821 +0000 2024-09-21T01:40:07.442 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-21 01:33:55.606576821 +0000 2024-09-21T01:40:07.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:40:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:07 smithi190 ceph-mon[34368]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-21T01:40:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-21T01:40:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000181897 s, 2.8 MB/s 2024-09-21T01:40:07.507 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:40:07.565 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-21T01:40:07.565 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:40:07.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:40:07.585 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-09-21T01:40:07.644 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:40:07.644 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:07.644 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-21 01:39:32.518978671 +0000 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-21 01:33:43.181800618 +0000 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-21 01:33:43.181800618 +0000 2024-09-21T01:40:07.645 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-21 01:33:43.181800618 +0000 2024-09-21T01:40:07.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:40:07.715 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-21T01:40:07.716 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-21T01:40:07.716 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000235833 s, 2.2 MB/s 2024-09-21T01:40:07.717 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:40:07.776 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-09-21T01:40:07.833 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-21 01:39:32.519978643 +0000 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-21 01:33:43.503791317 +0000 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-21 01:33:43.503791317 +0000 2024-09-21T01:40:07.834 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-21 01:33:43.503791317 +0000 2024-09-21T01:40:07.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:40:07.899 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-21T01:40:07.899 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-21T01:40:07.899 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000210818 s, 2.4 MB/s 2024-09-21T01:40:07.900 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:40:07.957 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-09-21T01:40:08.013 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:40:08.013 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:08.013 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-21 01:39:32.519978643 +0000 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-21 01:33:43.838781641 +0000 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-21 01:33:43.838781641 +0000 2024-09-21T01:40:08.014 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-21 01:33:43.838781641 +0000 2024-09-21T01:40:08.014 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:40:08.077 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-21T01:40:08.078 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-21T01:40:08.078 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000228242 s, 2.2 MB/s 2024-09-21T01:40:08.079 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:40:08.135 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-09-21T01:40:08.190 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-21 01:39:32.520978615 +0000 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-21 01:33:44.164772225 +0000 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-21 01:33:44.164772225 +0000 2024-09-21T01:40:08.191 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-21 01:33:44.164772225 +0000 2024-09-21T01:40:08.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:40:08.253 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-21T01:40:08.253 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-21T01:40:08.254 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000229496 s, 2.2 MB/s 2024-09-21T01:40:08.255 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:40:08.311 INFO:tasks.cephadm:Deploying osd.0 on smithi029 with /dev/vg_nvme/lv_4... 2024-09-21T01:40:08.311 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:40:08.574 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:40:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:09 smithi066 ceph-mon[35001]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:09.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:09 smithi029 ceph-mon[31228]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:09.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:09 smithi190 ceph-mon[34368]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:11.234 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:11 smithi029 ceph-mon[31228]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:11.237 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:40:11.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi029:vg_nvme/lv_4 2024-09-21T01:40:11.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:11 smithi066 ceph-mon[35001]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:11.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:11 smithi190 ceph-mon[34368]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:11.521 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:40:11.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 -- 172.21.15.29:0/74366788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a41036d0 msgr2=0x7f51a4103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:11.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 --2- 172.21.15.29:0/74366788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a41036d0 0x7f51a4103ae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402a480 comp rx=0 tx=0).stop 2024-09-21T01:40:11.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 -- 172.21.15.29:0/74366788 shutdown_connections 2024-09-21T01:40:11.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 --2- 172.21.15.29:0/74366788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a4104070 0x7f51a410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:11.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 --2- 172.21.15.29:0/74366788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a41036d0 0x7f51a4103ae0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:11.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 --2- 172.21.15.29:0/74366788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4102640 0x7f51a4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 -- 172.21.15.29:0/74366788 >> 172.21.15.29:0/74366788 conn(0x7f51a40fdfb0 msgr2=0x7f51a4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 -- 172.21.15.29:0/74366788 shutdown_connections 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.977+0000 7f51ab89c640 1 -- 172.21.15.29:0/74366788 wait complete. 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 Processor -- start 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 -- start start 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a41036d0 0x7f51a419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 0x7f51a419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410e270 con 0x7f51a4102640 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410e0f0 con 0x7f51a41036d0 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51ab89c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410e3f0 con 0x7f51a4104070 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.978+0000 7f51a9638640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:11.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40154/0 (socket says 172.21.15.29:40154) 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 -- 172.21.15.29:0/2114155867 learned_addr learned my addr 172.21.15.29:0/2114155867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a8e37640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a41036d0 0x7f51a419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9e39640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 0x7f51a419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 -- 172.21.15.29:0/2114155867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 msgr2=0x7f51a419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 0x7f51a419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 -- 172.21.15.29:0/2114155867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a41036d0 msgr2=0x7f51a419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a41036d0 0x7f51a419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 -- 172.21.15.29:0/2114155867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a419b9b0 con 0x7f51a4102640 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9e39640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 0x7f51a419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51a9638640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51a000d950 tx=0x7f51a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0017030 con 0x7f51a4102640 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a0004020 con 0x7f51a4102640 2024-09-21T01:40:11.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a419bca0 con 0x7f51a4102640 2024-09-21T01:40:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00047a0 con 0x7f51a4102640 2024-09-21T01:40:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.979+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a2430 con 0x7f51a4102640 2024-09-21T01:40:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.980+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a41a2600 con 0x7f51a4102640 2024-09-21T01:40:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.982+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f51a0009a30 con 0x7f51a4102640 2024-09-21T01:40:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.983+0000 7f519a7fc640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 0x7f51900b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:11.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.983+0000 7f51a8e37640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 0x7f51900b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:11.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.983+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51a00e5790 con 0x7f51a4102640 2024-09-21T01:40:11.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.984+0000 7f51a8e37640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 0x7f51900b8a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5194000c00 tx=0x7f5194031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:11.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:11.986+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51a00adcd0 con 0x7f51a4102640 2024-09-21T01:40:12.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:12.191+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f51a40fffd0 con 0x7f51900b6500 2024-09-21T01:40:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:13 smithi066 ceph-mon[35001]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:13 smithi066 ceph-mon[35001]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:13.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:13 smithi029 ceph-mon[31228]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:13.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:13 smithi029 ceph-mon[31228]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:13.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:13.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:13.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:13 smithi190 ceph-mon[34368]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:13 smithi190 ceph-mon[34368]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:15.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:15 smithi029 ceph-mon[31228]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:15.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:15 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"} : dispatch 2024-09-21T01:40:15.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:15 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"}]': finished 2024-09-21T01:40:15.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:15 smithi029 ceph-mon[31228]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:40:15.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:15 smithi066 ceph-mon[35001]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:15 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"} : dispatch 2024-09-21T01:40:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:15 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"}]': finished 2024-09-21T01:40:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:15 smithi066 ceph-mon[35001]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:40:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:15 smithi190 ceph-mon[34368]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:15 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"} : dispatch 2024-09-21T01:40:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:15 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/275457575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c63b551-55db-4486-b93e-1f5773c9cdaf"}]': finished 2024-09-21T01:40:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:15 smithi190 ceph-mon[34368]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:40:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1841560884' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1841560884' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1841560884' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:17.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:17 smithi066 ceph-mon[35001]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:17 smithi029 ceph-mon[31228]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:17 smithi190 ceph-mon[34368]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:19.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:19 smithi066 ceph-mon[35001]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:19.314 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:19 smithi029 ceph-mon[31228]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:19 smithi190 ceph-mon[34368]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:21.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:21 smithi066 ceph-mon[35001]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:21 smithi029 ceph-mon[31228]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:21 smithi190 ceph-mon[34368]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:23.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:23 smithi066 ceph-mon[35001]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:23.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:40:23.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:23.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:23 smithi066 ceph-mon[35001]: Deploying daemon osd.0 on smithi029 2024-09-21T01:40:23.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:23 smithi029 ceph-mon[31228]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:23.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:40:23.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:23.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:23 smithi029 ceph-mon[31228]: Deploying daemon osd.0 on smithi029 2024-09-21T01:40:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:23 smithi190 ceph-mon[34368]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:40:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:23 smithi190 ceph-mon[34368]: Deploying daemon osd.0 on smithi029 2024-09-21T01:40:25.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:25 smithi029 ceph-mon[31228]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:25.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:25 smithi190 ceph-mon[34368]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:25.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:25 smithi066 ceph-mon[35001]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:27.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:27 smithi029 ceph-mon[31228]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:27.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:27 smithi190 ceph-mon[34368]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:27 smithi066 ceph-mon[35001]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:29.316 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:29 smithi029 ceph-mon[31228]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:29.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:29 smithi190 ceph-mon[34368]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:29.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:29 smithi066 ceph-mon[35001]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:30.267 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:30.267 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:30.267 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:30.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:30.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:30.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:31 smithi029 ceph-mon[31228]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:31 smithi190 ceph-mon[34368]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:31 smithi066 ceph-mon[35001]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:33.766 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:33.809 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 0 on host 'smithi029' 2024-09-21T01:40:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.808+0000 7f519a7fc640 1 -- 172.21.15.29:0/2114155867 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51a40fffd0 con 0x7f51900b6500 2024-09-21T01:40:33.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 msgr2=0x7f51900b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:33.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 0x7f51900b8a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5194000c00 tx=0x7f5194031400 comp rx=0 tx=0).stop 2024-09-21T01:40:33.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 msgr2=0x7f51a419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:33.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51a000d950 tx=0x7f51a000de10 comp rx=0 tx=0).stop 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 shutdown_connections 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51900b6500 0x7f51900b8a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4104070 0x7f51a419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51a41036d0 0x7f51a419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 --2- 172.21.15.29:0/2114155867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51a4102640 0x7f51a419a010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 >> 172.21.15.29:0/2114155867 conn(0x7f51a40fdfb0 msgr2=0x7f51a40ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 shutdown_connections 2024-09-21T01:40:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:40:33.811+0000 7f51ab89c640 1 -- 172.21.15.29:0/2114155867 wait complete. 2024-09-21T01:40:34.507 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:34.507 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.507 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.539 DEBUG:teuthology.orchestra.run.smithi029:osd.0> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T01:40:34.541 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/vg_nvme/lv_4... 2024-09-21T01:40:34.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:40:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:34.808 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:40:35.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:35 smithi066 ceph-mon[35001]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:35.639 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:35 smithi029 ceph-mon[31228]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:35 smithi190 ceph-mon[34368]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:35.919 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:40:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T01:40:35.638+0000 7f28ad5ab640 -1 osd.0 0 log_to_monitors true 2024-09-21T01:40:36.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:36 smithi066 ceph-mon[35001]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:40:36.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:36 smithi190 ceph-mon[34368]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:40:36.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:36 smithi029 ceph-mon[31228]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:40:37.248 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:40:37.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-09-21T01:40:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:40:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:40:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:40:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:37.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:37.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.594 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T01:40:37.410+0000 7f28a952d640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T01:40:37.595 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T01:40:37.417+0000 7f28a532b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:37.596 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:37.596 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:40:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:40:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:40:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:37.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:38.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 -- 172.21.15.66:0/2119223188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 msgr2=0x7f1220102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:38.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 --2- 172.21.15.66:0/2119223188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f1220102ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c02a480 comp rx=0 tx=0).stop 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 -- 172.21.15.66:0/2119223188 shutdown_connections 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 --2- 172.21.15.66:0/2119223188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1220104070 0x7f122010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 --2- 172.21.15.66:0/2119223188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12201036d0 0x7f1220103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 --2- 172.21.15.66:0/2119223188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f1220102ab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.078+0000 7f1227988640 1 -- 172.21.15.66:0/2119223188 >> 172.21.15.66:0/2119223188 conn(0x7f12200fdfb0 msgr2=0x7f1220100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.079+0000 7f1227988640 1 -- 172.21.15.66:0/2119223188 shutdown_connections 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.079+0000 7f1227988640 1 -- 172.21.15.66:0/2119223188 wait complete. 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.079+0000 7f1227988640 1 Processor -- start 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.079+0000 7f1227988640 1 -- start start 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:38.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12201036d0 0x7f122019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 0x7f12201a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f122010e280 con 0x7f12201036d0 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f122010e100 con 0x7f1220102640 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1227988640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f122010e400 con 0x7f1220104070 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41672/0 (socket says 172.21.15.66:41672) 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 -- 172.21.15.66:0/2611065780 learned_addr learned my addr 172.21.15.66:0/2611065780 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1224f23640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12201036d0 0x7f122019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225f25640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 0x7f12201a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 -- 172.21.15.66:0/2611065780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 msgr2=0x7f12201a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 0x7f12201a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 -- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12201036d0 msgr2=0x7f122019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12201036d0 0x7f122019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:38.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 -- 172.21.15.66:0/2611065780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12201a1a90 con 0x7f1220102640 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225f25640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 0x7f12201a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f1225724640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f121c006010 tx=0x7f121c031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.080+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c03f030 con 0x7f1220102640 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.081+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1220198ab0 con 0x7f1220102640 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.081+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c003aa0 con 0x7f1220102640 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.081+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c035070 con 0x7f1220102640 2024-09-21T01:40:38.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.081+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1220198f90 con 0x7f1220102640 2024-09-21T01:40:38.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.083+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220102ab0 con 0x7f1220102640 2024-09-21T01:40:38.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.084+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f121c042030 con 0x7f1220102640 2024-09-21T01:40:38.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.085+0000 7f12127fc640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 0x7f120c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:40:38.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.085+0000 7f1224f23640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 0x7f120c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:40:38.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.085+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f121c105a00 con 0x7f1220102640 2024-09-21T01:40:38.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.086+0000 7f1224f23640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 0x7f120c0b8990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1214009b60 tx=0x7f1214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:40:38.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.089+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f121c0cde00 con 0x7f1220102640 2024-09-21T01:40:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:38.288+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1220063800 con 0x7f120c0b6480 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi029 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi029 to 17092M 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi029 2024-09-21T01:40:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi029 to 17092M 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:38.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi029 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi029 to 17092M 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:40:38.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:39.546 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] boot 2024-09-21T01:40:39.561 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:40:39.562 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] boot 2024-09-21T01:40:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:40:39.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:40:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: osd.0 [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] boot 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:40:39.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:40:40.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:40 smithi190 ceph-mon[34368]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:40:40.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:40 smithi066 ceph-mon[35001]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:40:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:40 smithi029 ceph-mon[31228]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/2198095091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"}]': finished 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:40:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:40:41.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:41 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/906940362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/2198095091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"}]': finished 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:40:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:41 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/906940362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:41.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:41.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/2198095091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"} : dispatch 2024-09-21T01:40:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f29e6aa-4705-4810-9e3a-0118bec1dd83"}]': finished 2024-09-21T01:40:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:40:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:40:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:41 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/906940362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:40:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:43 smithi190 ceph-mon[34368]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:43 smithi066 ceph-mon[35001]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:43.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:43 smithi029 ceph-mon[31228]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:45.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:45 smithi066 ceph-mon[35001]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:45 smithi190 ceph-mon[34368]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:45 smithi029 ceph-mon[31228]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:47.609 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:47 smithi066 ceph-mon[35001]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:47.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:47 smithi190 ceph-mon[34368]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:47.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:47 smithi029 ceph-mon[31228]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:49.685 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:49 smithi066 ceph-mon[35001]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:49.685 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:40:49.685 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:49 smithi190 ceph-mon[34368]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:40:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:49 smithi029 ceph-mon[31228]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:40:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:50.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:50 smithi066 ceph-mon[35001]: Deploying daemon osd.1 on smithi066 2024-09-21T01:40:50.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:50 smithi190 ceph-mon[34368]: Deploying daemon osd.1 on smithi066 2024-09-21T01:40:50.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:50 smithi029 ceph-mon[31228]: Deploying daemon osd.1 on smithi066 2024-09-21T01:40:51.448 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:51 smithi066 ceph-mon[35001]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:51 smithi190 ceph-mon[34368]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:51 smithi029 ceph-mon[31228]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:53 smithi190 ceph-mon[34368]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:53.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:53 smithi066 ceph-mon[35001]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:53.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:53 smithi029 ceph-mon[31228]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:55 smithi190 ceph-mon[34368]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:55.766 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:55 smithi066 ceph-mon[35001]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:55.766 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:55.766 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:55.766 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:55.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:55 smithi029 ceph-mon[31228]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:55.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:40:55.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:55.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:57 smithi190 ceph-mon[34368]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:57.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:57 smithi066 ceph-mon[35001]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:57.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:57 smithi029 ceph-mon[31228]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:40:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:59.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:40:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:40:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:40:59.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.370+0000 7f12127fc640 1 -- 172.21.15.66:0/2611065780 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1220063800 con 0x7f120c0b6480 2024-09-21T01:40:59.371 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-09-21T01:40:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 msgr2=0x7f120c0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 0x7f120c0b8990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1214009b60 tx=0x7f1214005f90 comp rx=0 tx=0).stop 2024-09-21T01:40:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 msgr2=0x7f122006b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f121c006010 tx=0x7f121c031a50 comp rx=0 tx=0).stop 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 shutdown_connections 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f120c0b6480 0x7f120c0b8990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1220104070 0x7f12201a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12201036d0 0x7f122019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 --2- 172.21.15.66:0/2611065780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1220102640 0x7f122006b700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 >> 172.21.15.66:0/2611065780 conn(0x7f12200fdfb0 msgr2=0x7f12201069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 shutdown_connections 2024-09-21T01:40:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:40:59.372+0000 7f1227988640 1 -- 172.21.15.66:0/2611065780 wait complete. 2024-09-21T01:40:59.974 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T01:40:59.976 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2024-09-21T01:40:59.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:41:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:59 smithi066 ceph-mon[35001]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:00.224 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:59 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:00.224 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:59 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.224 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:40:59 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.224 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:59 smithi029 ceph-mon[31228]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:00.225 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:59 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:00.225 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:59 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.225 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:40:59 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.227 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:59 smithi190 ceph-mon[34368]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:59 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:59 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:40:59 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:00.229 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:41:01.867 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:01 smithi190 ceph-mon[34368]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:01.867 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:01 smithi190 ceph-mon[34368]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:01.867 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:01 smithi190 ceph-mon[34368]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:01 smithi066 ceph-mon[35001]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:01 smithi066 ceph-mon[35001]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:01 smithi066 ceph-mon[35001]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:02.050 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T01:41:01.580+0000 7f04522e2640 -1 osd.1 0 log_to_monitors true 2024-09-21T01:41:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:01 smithi029 ceph-mon[31228]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:01 smithi029 ceph-mon[31228]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:01 smithi029 ceph-mon[31228]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:41:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:41:02.417 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-09-21T01:41:02.671 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:41:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:02.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:03.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:03.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:03.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:03.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:03.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 -- 172.21.15.190:0/761281375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410102cf0 msgr2=0x7f2410105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:03.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 --2- 172.21.15.190:0/761281375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410102cf0 0x7f2410105110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2400009900 tx=0x7f240002a410 comp rx=0 tx=0).stop 2024-09-21T01:41:03.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 -- 172.21.15.190:0/761281375 shutdown_connections 2024-09-21T01:41:03.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 --2- 172.21.15.190:0/761281375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410108370 0x7f241010a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:03.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 --2- 172.21.15.190:0/761281375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410105750 0x7f2410107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:03.125 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 --2- 172.21.15.190:0/761281375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410102cf0 0x7f2410105110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:03.125 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 -- 172.21.15.190:0/761281375 >> 172.21.15.190:0/761281375 conn(0x7f24100fd770 msgr2=0x7f24100ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:41:03.125 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.123+0000 7f241629d640 1 -- 172.21.15.190:0/761281375 shutdown_connections 2024-09-21T01:41:03.125 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 -- 172.21.15.190:0/761281375 wait complete. 2024-09-21T01:41:03.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 Processor -- start 2024-09-21T01:41:03.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 -- start start 2024-09-21T01:41:03.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410102cf0 0x7f2410199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410108370 0x7f241019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f241010e270 con 0x7f2410108370 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f241010e0f0 con 0x7f2410102cf0 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.124+0000 7f241629d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f241010e3f0 con 0x7f2410105750 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39976/0 (socket says 172.21.15.190:39976) 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 -- 172.21.15.190:0/2699778851 learned_addr learned my addr 172.21.15.190:0/2699778851 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f240ffff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410102cf0 0x7f2410199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f241483a640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410108370 0x7f241019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 -- 172.21.15.190:0/2699778851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410102cf0 msgr2=0x7f2410199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410102cf0 0x7f2410199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:03.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 -- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410108370 msgr2=0x7f241019b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410108370 0x7f241019b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 -- 172.21.15.190:0/2699778851 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019b8f0 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f2407fff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c920 tx=0x7f23f800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8007c70 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f8007e10 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019bbe0 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8018820 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.125+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a2370 con 0x7f2410105750 2024-09-21T01:41:03.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.126+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24101a2540 con 0x7f2410105750 2024-09-21T01:41:03.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.129+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f23f800f450 con 0x7f2410105750 2024-09-21T01:41:03.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.130+0000 7f240dffb640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 0x7f23fc0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:03.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.130+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f23f80e5cf0 con 0x7f2410105750 2024-09-21T01:41:03.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.130+0000 7f240ffff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 0x7f23fc0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:03.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.131+0000 7f240ffff640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 0x7f23fc0b8a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f2400031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:03.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.132+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23f8014030 con 0x7f2410105750 2024-09-21T01:41:03.326 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:03.325+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f24100feb90 con 0x7f23fc0b6500 2024-09-21T01:41:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi066 2024-09-21T01:41:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi066 to 17088M 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:03.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:03 smithi190 ceph-mon[34368]: from='osd.1 ' entity='osd.1' 2024-09-21T01:41:04.049 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T01:41:03.745+0000 7f044e264640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T01:41:04.049 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T01:41:03.859+0000 7f044605a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi066 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi066 to 17088M 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:41:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:04.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:04.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:04.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:04.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:04.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:03 smithi066 ceph-mon[35001]: from='osd.1 ' entity='osd.1' 2024-09-21T01:41:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi066 2024-09-21T01:41:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi066 to 17088M 2024-09-21T01:41:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:04.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:03 smithi029 ceph-mon[31228]: from='osd.1 ' entity='osd.1' 2024-09-21T01:41:05.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:04 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:41:05.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:04 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:41:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:04 smithi029 ceph-mon[31228]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:05.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:04 smithi029 ceph-mon[31228]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:05.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:04 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:41:05.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:04 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:41:05.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:04 smithi190 ceph-mon[34368]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:05.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:05.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:04 smithi190 ceph-mon[34368]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:04 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:41:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:04 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:41:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:04 smithi066 ceph-mon[35001]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:05.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:04 smithi066 ceph-mon[35001]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:41:06.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] boot 2024-09-21T01:41:06.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:41:06.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:06.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/3322932689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"}]': finished 2024-09-21T01:41:06.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:41:06.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] boot 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/3322932689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"}]': finished 2024-09-21T01:41:06.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:41:06.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: osd.1 [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] boot 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/3322932689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"} : dispatch 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe1d952-803c-4222-88cd-cff3887aaecc"}]': finished 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:41:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:06 smithi029 ceph-mon[31228]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/2646457603' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:06 smithi029 ceph-mon[31228]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:41:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:06 smithi190 ceph-mon[34368]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/2646457603' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:06 smithi190 ceph-mon[34368]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:41:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:06 smithi066 ceph-mon[35001]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/2646457603' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:06 smithi066 ceph-mon[35001]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:41:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:09.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:09 smithi029 ceph-mon[31228]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:09.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:09 smithi190 ceph-mon[34368]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:09 smithi066 ceph-mon[35001]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:11.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:11 smithi190 ceph-mon[34368]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:11.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:11 smithi029 ceph-mon[31228]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:11 smithi066 ceph-mon[35001]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:13.315 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:13 smithi190 ceph-mon[34368]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:13.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:13 smithi029 ceph-mon[31228]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:13 smithi066 ceph-mon[35001]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:15.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:15 smithi029 ceph-mon[31228]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:15.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:41:15.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:15.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:15 smithi029 ceph-mon[31228]: Deploying daemon osd.2 on smithi190 2024-09-21T01:41:15.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:15 smithi066 ceph-mon[35001]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:15.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:41:15.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:15.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:15 smithi066 ceph-mon[35001]: Deploying daemon osd.2 on smithi190 2024-09-21T01:41:15.465 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:15 smithi190 ceph-mon[34368]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:15.465 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:41:15.465 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:15.465 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:15 smithi190 ceph-mon[34368]: Deploying daemon osd.2 on smithi190 2024-09-21T01:41:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:17 smithi029 ceph-mon[31228]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:17.355 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:17 smithi190 ceph-mon[34368]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:17 smithi066 ceph-mon[35001]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:19.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:19 smithi190 ceph-mon[34368]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:19 smithi029 ceph-mon[31228]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:19 smithi066 ceph-mon[35001]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:21 smithi029 ceph-mon[31228]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:21.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:21 smithi190 ceph-mon[34368]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:21 smithi066 ceph-mon[35001]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:23.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:24.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.539+0000 7f240dffb640 1 -- 172.21.15.190:0/2699778851 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f24100feb90 con 0x7f23fc0b6500 2024-09-21T01:41:24.540 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-09-21T01:41:24.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 msgr2=0x7f23fc0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:24.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 0x7f23fc0b8a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f2400031400 comp rx=0 tx=0).stop 2024-09-21T01:41:24.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 msgr2=0x7f241019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:24.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c920 tx=0x7f23f800cde0 comp rx=0 tx=0).stop 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 shutdown_connections 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23fc0b6500 0x7f23fc0b8a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2410108370 0x7f241019b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2410105750 0x7f241019a5c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 --2- 172.21.15.190:0/2699778851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2410102cf0 0x7f2410199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.541+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 >> 172.21.15.190:0/2699778851 conn(0x7f24100fd770 msgr2=0x7f2410107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.542+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 shutdown_connections 2024-09-21T01:41:24.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:41:24.542+0000 7f241629d640 1 -- 172.21.15.190:0/2699778851 wait complete. 2024-09-21T01:41:25.201 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T01:41:25.204 INFO:tasks.cephadm:Deploying osd.3 on smithi029 with /dev/vg_nvme/lv_3... 2024-09-21T01:41:25.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:41:25.471 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:41:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:25 smithi190 ceph-mon[34368]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:25 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:25 smithi066 ceph-mon[35001]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:25 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:25.827 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:25 smithi029 ceph-mon[31228]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:25.827 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:25.827 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:25.827 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:25 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:26.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:26 smithi190 ceph-mon[34368]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:26.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:26 smithi190 ceph-mon[34368]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:26.729 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T01:41:26.411+0000 7f2bb008e640 -1 osd.2 0 log_to_monitors true 2024-09-21T01:41:26.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:26 smithi066 ceph-mon[35001]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:26.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:26 smithi066 ceph-mon[35001]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:26.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:26 smithi029 ceph-mon[31228]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:26.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:26 smithi029 ceph-mon[31228]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:27.704 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:27 smithi190 ceph-mon[34368]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:27 smithi066 ceph-mon[35001]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:41:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: from='osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:41:27.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:27.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:27 smithi029 ceph-mon[31228]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:41:28.066 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:41:28.092 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi029:vg_nvme/lv_3 2024-09-21T01:41:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:41:28.728 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T01:41:28.421+0000 7f2bac010640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T01:41:28.728 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T01:41:28.597+0000 7f2ba3e06640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:41:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:41:28.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi190 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:28 smithi190 ceph-mon[34368]: from='osd.2 ' entity='osd.2' 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi190 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.736 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:28.737 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.737 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:28.737 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:28.737 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.737 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:28 smithi029 ceph-mon[31228]: from='osd.2 ' entity='osd.2' 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi190 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:28.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:28.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:28.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:28 smithi066 ceph-mon[35001]: from='osd.2 ' entity='osd.2' 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 -- 172.21.15.29:0/121090177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c104070 msgr2=0x7f661c10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 --2- 172.21.15.29:0/121090177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c104070 0x7f661c10aa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6614009ca0 tx=0x7f661402a950 comp rx=0 tx=0).stop 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 -- 172.21.15.29:0/121090177 shutdown_connections 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 --2- 172.21.15.29:0/121090177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c104070 0x7f661c10aa10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 --2- 172.21.15.29:0/121090177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c1036d0 0x7f661c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 --2- 172.21.15.29:0/121090177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c102640 0x7f661c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 -- 172.21.15.29:0/121090177 >> 172.21.15.29:0/121090177 conn(0x7f661c0fdfb0 msgr2=0x7f661c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 -- 172.21.15.29:0/121090177 shutdown_connections 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.831+0000 7f6621eda640 1 -- 172.21.15.29:0/121090177 wait complete. 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 Processor -- start 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 -- start start 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c1036d0 0x7f661c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c104070 0x7f661c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f661c10f410 con 0x7f661c102640 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f661c10f290 con 0x7f661c1036d0 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f6621eda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f661c10f590 con 0x7f661c104070 2024-09-21T01:41:28.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f661b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c1036d0 0x7f661c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.832+0000 7f661b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57384/0 (socket says 172.21.15.29:57384) 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 -- 172.21.15.29:0/920651997 learned_addr learned my addr 172.21.15.29:0/920651997 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661bfff640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c104070 0x7f661c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 -- 172.21.15.29:0/920651997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c104070 msgr2=0x7f661c19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c104070 0x7f661c19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 -- 172.21.15.29:0/920651997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c1036d0 msgr2=0x7f661c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c1036d0 0x7f661c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 -- 172.21.15.29:0/920651997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c19b980 con 0x7f661c102640 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f661b7fe640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f660c00d950 tx=0x7f660c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c017070 con 0x7f661c102640 2024-09-21T01:41:28.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c009c30 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c19bc70 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c00f620 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.833+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c1a2370 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.835+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c109800 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.836+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f660c010460 con 0x7f661c102640 2024-09-21T01:41:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.837+0000 7f6618ff9640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 0x7f66080b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:28.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.838+0000 7f661affd640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 0x7f66080b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:28.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.838+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f660c0e5730 con 0x7f661c102640 2024-09-21T01:41:28.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.838+0000 7f661affd640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 0x7f66080b8990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d00 tx=0x7f660400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:28.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:28.841+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f660c009de0 con 0x7f661c102640 2024-09-21T01:41:29.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:29.040+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f661c063800 con 0x7f66080b6480 2024-09-21T01:41:29.716 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:41:29.716 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:29.717 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:29.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:41:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:41:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-21T01:41:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:30.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:30 smithi066 ceph-mon[35001]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:30.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:30 smithi066 ceph-mon[35001]: osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] boot 2024-09-21T01:41:30.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:30 smithi066 ceph-mon[35001]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:41:30.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:30.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:30 smithi029 ceph-mon[31228]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:30 smithi029 ceph-mon[31228]: osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] boot 2024-09-21T01:41:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:30 smithi029 ceph-mon[31228]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:41:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:30.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:30.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:30 smithi190 ceph-mon[34368]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:30.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:30 smithi190 ceph-mon[34368]: osd.2 [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] boot 2024-09-21T01:41:30.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:30 smithi190 ceph-mon[34368]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:41:30.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:41:30.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:31.838 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 ceph-mon[31228]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:31.838 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:31.839 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 ceph-mon[31228]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:41:31.839 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48758]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48758]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48758]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48758]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48765]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48765]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.839 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48765]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:31.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 ceph-mon[34368]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:31.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:31.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 ceph-mon[34368]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:41:31.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42294]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42294]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42294]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.881 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42297]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:31.922 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 ceph-mon[35001]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:41:31.922 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:31.922 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 ceph-mon[35001]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:41:31.922 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42834]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42834]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42837]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42837]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42837]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:31.923 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42837]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48770]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48770]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48774]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48774]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48774]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:31 smithi029 sudo[48774]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42300]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42300]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:31 smithi190 sudo[42303]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:41:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42840]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42840]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:41:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:41:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42843]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:41:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:31 smithi066 sudo[42843]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:41:32.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:32.839+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f660c0a9330 con 0x7f661c102640 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"} : dispatch 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"}]': finished 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.841 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1228240354' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:32.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:32.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"} : dispatch 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"}]': finished 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:32.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:32.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:32.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:32.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:32.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1228240354' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:41:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:41:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"} : dispatch 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/56062700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f101639-e43c-4377-8de2-63ffbe3ce76d"}]': finished 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:33.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:41:33.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1228240354' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:33.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:33 smithi029 ceph-mon[31228]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:33.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:33 smithi029 ceph-mon[31228]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T01:41:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:33 smithi190 ceph-mon[34368]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:33 smithi190 ceph-mon[34368]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T01:41:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:33 smithi066 ceph-mon[35001]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:33 smithi066 ceph-mon[35001]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T01:41:35.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:35 smithi029 ceph-mon[31228]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:35 smithi190 ceph-mon[34368]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:35 smithi066 ceph-mon[35001]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:37 smithi029 ceph-mon[31228]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:37 smithi190 ceph-mon[34368]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:37 smithi066 ceph-mon[35001]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:39 smithi029 ceph-mon[31228]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:39 smithi190 ceph-mon[34368]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:39 smithi066 ceph-mon[35001]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:40 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:41:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:40 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:40 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:41:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:40 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:41.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:40 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:41:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:40 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:41.763 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:41 smithi029 ceph-mon[31228]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:41.763 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:41 smithi029 ceph-mon[31228]: Deploying daemon osd.3 on smithi029 2024-09-21T01:41:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:41 smithi190 ceph-mon[34368]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:41 smithi190 ceph-mon[34368]: Deploying daemon osd.3 on smithi029 2024-09-21T01:41:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:41 smithi066 ceph-mon[35001]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:41 smithi066 ceph-mon[35001]: Deploying daemon osd.3 on smithi029 2024-09-21T01:41:43.814 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:43 smithi029 ceph-mon[31228]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:43 smithi190 ceph-mon[34368]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:43 smithi066 ceph-mon[35001]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:45.647 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:45 smithi029 ceph-mon[31228]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:45.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:45 smithi066 ceph-mon[35001]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:45 smithi190 ceph-mon[34368]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:47.700 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:47 smithi029 ceph-mon[31228]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:47.700 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:47.700 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:47.701 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:47 smithi190 ceph-mon[34368]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:47 smithi066 ceph-mon[35001]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:49 smithi029 ceph-mon[31228]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:49 smithi190 ceph-mon[34368]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:49 smithi066 ceph-mon[35001]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:50 smithi190 ceph-mon[34368]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:50 smithi029 ceph-mon[31228]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:50 smithi066 ceph-mon[35001]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:51.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.308+0000 7f6618ff9640 1 -- 172.21.15.29:0/920651997 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f661c063800 con 0x7f66080b6480 2024-09-21T01:41:51.309 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 3 on host 'smithi029' 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 msgr2=0x7f66080b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 0x7f66080b8990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d00 tx=0x7f660400b410 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 msgr2=0x7f661c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f660c00d950 tx=0x7f660c00de10 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 shutdown_connections 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66080b6480 0x7f66080b8990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c104070 0x7f661c19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f661c1036d0 0x7f661c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 --2- 172.21.15.29:0/920651997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f661c102640 0x7f661c199f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.311+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 >> 172.21.15.29:0/920651997 conn(0x7f661c0fdfb0 msgr2=0x7f661c1069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.312+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 shutdown_connections 2024-09-21T01:41:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:41:51.312+0000 7f6621eda640 1 -- 172.21.15.29:0/920651997 wait complete. 2024-09-21T01:41:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:41:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:52.334 DEBUG:teuthology.orchestra.run.smithi029:osd.3> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T01:41:52.336 INFO:tasks.cephadm:Deploying osd.4 on smithi066 with /dev/vg_nvme/lv_3... 2024-09-21T01:41:52.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:41:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:41:52.994 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:52 smithi066 ceph-mon[35001]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:52.995 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:52 smithi066 ceph-mon[35001]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:41:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:52 smithi029 ceph-mon[31228]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:52 smithi029 ceph-mon[31228]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:41:53.095 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 01:41:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T01:41:52.822+0000 7feb176ac640 -1 osd.3 0 log_to_monitors true 2024-09-21T01:41:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:52 smithi190 ceph-mon[34368]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:52 smithi190 ceph-mon[34368]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:41:54.188 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:53 smithi066 ceph-mon[35001]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:41:54.188 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:53 smithi066 ceph-mon[35001]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:41:54.188 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:53 smithi066 ceph-mon[35001]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:41:54.189 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:53 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:53 smithi190 ceph-mon[34368]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:41:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:53 smithi190 ceph-mon[34368]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:41:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:53 smithi190 ceph-mon[34368]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:41:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:53 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:54.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:53 smithi029 ceph-mon[31228]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:41:54.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:53 smithi029 ceph-mon[31228]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:41:54.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:53 smithi029 ceph-mon[31228]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:41:54.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:54.824 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:41:54.850 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-09-21T01:41:55.079 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T01:41:54.717+0000 7feb1362e640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T01:41:55.080 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T01:41:54.724+0000 7feb0b424640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:55.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:54 smithi029 ceph-mon[31228]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' 2024-09-21T01:41:55.117 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:54 smithi190 ceph-mon[34368]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' 2024-09-21T01:41:55.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:41:55.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:41:55.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:55.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:41:55.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:54 smithi066 ceph-mon[35001]: from='osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638]' entity='osd.3' 2024-09-21T01:41:55.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 -- 172.21.15.66:0/3356325289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc4104800 msgr2=0x7f0bc4104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:55.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 --2- 172.21.15.66:0/3356325289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc4104800 0x7f0bc4104c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009900 tx=0x7f0bb802a480 comp rx=0 tx=0).stop 2024-09-21T01:41:55.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 -- 172.21.15.66:0/3356325289 shutdown_connections 2024-09-21T01:41:55.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 --2- 172.21.15.66:0/3356325289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc41051a0 0x7f0bc410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:55.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 --2- 172.21.15.66:0/3356325289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc4104800 0x7f0bc4104c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 --2- 172.21.15.66:0/3356325289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4103770 0x7f0bc4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 -- 172.21.15.66:0/3356325289 >> 172.21.15.66:0/3356325289 conn(0x7f0bc40ff0e0 msgr2=0x7f0bc4101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 -- 172.21.15.66:0/3356325289 shutdown_connections 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.571+0000 7f0bca222640 1 -- 172.21.15.66:0/3356325289 wait complete. 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 Processor -- start 2024-09-21T01:41:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 -- start start 2024-09-21T01:41:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 0x7f0bc419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4104800 0x7f0bc419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410f3a0 con 0x7f0bc4103770 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410f220 con 0x7f0bc41051a0 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bca222640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410f520 con 0x7f0bc4104800 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.572+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 0x7f0bc419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42464/0 (socket says 172.21.15.66:42464) 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 0x7f0bc419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.66:53450/0 (socket says 172.21.15.66:53450) 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 -- 172.21.15.66:0/2880506659 learned_addr learned my addr 172.21.15.66:0/2880506659 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc2ffd640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4104800 0x7f0bc419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 -- 172.21.15.66:0/2880506659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4104800 msgr2=0x7f0bc419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4104800 0x7f0bc419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 -- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 msgr2=0x7f0bc419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 0x7f0bc419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 -- 172.21.15.66:0/2880506659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc419b9f0 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc3fff640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00cad0 tx=0x7f0bbc00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00ad60 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bbc005020 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc419bce0 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00f6b0 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.573+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a22a0 con 0x7f0bc41051a0 2024-09-21T01:41:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.574+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc410a950 con 0x7f0bc41051a0 2024-09-21T01:41:55.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.576+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0bbc0051d0 con 0x7f0bc41051a0 2024-09-21T01:41:55.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.577+0000 7f0bc0ff9640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 0x7f0bac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:41:55.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.577+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7f0bbc0e64a0 con 0x7f0bc41051a0 2024-09-21T01:41:55.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.577+0000 7f0bc37fe640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 0x7f0bac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:41:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.578+0000 7f0bc37fe640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 0x7f0bac0b8a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400abe0 tx=0x7f0bb400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:41:55.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.580+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0bbc014030 con 0x7f0bc41051a0 2024-09-21T01:41:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:41:55.781+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0bc4063800 con 0x7f0bac0b6510 2024-09-21T01:41:56.171 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] boot 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi029 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi029 to 8546M 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:56.173 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:41:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:41:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] boot 2024-09-21T01:41:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi029 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi029 to 8546M 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:56.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:56.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:56.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:56.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: osd.3 [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] boot 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi029 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi029 to 8546M 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:41:56.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:41:57.208 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:56 smithi066 ceph-mon[35001]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:41:57.208 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:56 smithi066 ceph-mon[35001]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:57.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:56 smithi190 ceph-mon[34368]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:41:57.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:56 smithi190 ceph-mon[34368]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:56 smithi029 ceph-mon[31228]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:41:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:56 smithi029 ceph-mon[31228]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:57 smithi066 ceph-mon[35001]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:41:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:57 smithi190 ceph-mon[34368]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:41:58.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:57 smithi029 ceph-mon[31228]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/426889794' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"}]': finished 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:41:59.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:41:58 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/3493994726' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/426889794' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"}]': finished 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:41:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:41:59.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:41:58 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/3493994726' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:41:59.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/426889794' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"} : dispatch 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbf866-c852-4333-ac24-19c595975987"}]': finished 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:41:59.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:41:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/3493994726' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:01 smithi029 ceph-mon[31228]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T01:42:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:01 smithi190 ceph-mon[34368]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T01:42:01.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:01 smithi066 ceph-mon[35001]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T01:42:03.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:03 smithi029 ceph-mon[31228]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T01:42:03.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:03 smithi190 ceph-mon[34368]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T01:42:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:03 smithi066 ceph-mon[35001]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T01:42:05.296 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:05 smithi066 ceph-mon[35001]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:05.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:05 smithi029 ceph-mon[31228]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:05.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:05 smithi190 ceph-mon[34368]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:07.421 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:07 smithi066 ceph-mon[35001]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:42:07.421 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:42:07.421 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:07 smithi190 ceph-mon[34368]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:42:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:42:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:07 smithi029 ceph-mon[31228]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:42:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:42:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:08.142 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:08 smithi066 ceph-mon[35001]: Deploying daemon osd.4 on smithi066 2024-09-21T01:42:08.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:08 smithi190 ceph-mon[34368]: Deploying daemon osd.4 on smithi066 2024-09-21T01:42:08.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:08 smithi029 ceph-mon[31228]: Deploying daemon osd.4 on smithi066 2024-09-21T01:42:09.290 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:09 smithi066 ceph-mon[35001]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:42:09.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:09 smithi190 ceph-mon[34368]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:42:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:09 smithi029 ceph-mon[31228]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:42:11.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:11 smithi190 ceph-mon[34368]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:42:11.501 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:11 smithi066 ceph-mon[35001]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:42:11.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:11 smithi029 ceph-mon[31228]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:42:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:13 smithi066 ceph-mon[35001]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:13 smithi190 ceph-mon[34368]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:13.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:13 smithi029 ceph-mon[31228]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:13.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:13.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:13.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.388 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:15 smithi066 ceph-mon[35001]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:15.388 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.388 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:15 smithi190 ceph-mon[34368]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:15 smithi029 ceph-mon[31228]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:16.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:16.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:16.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:16.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:16.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:16.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:16.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.859+0000 7f0bc0ff9640 1 -- 172.21.15.66:0/2880506659 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bc4063800 con 0x7f0bac0b6510 2024-09-21T01:42:16.860 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 4 on host 'smithi066' 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 msgr2=0x7f0bac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 0x7f0bac0b8a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400abe0 tx=0x7f0bb400a2d0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 msgr2=0x7f0bc419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00cad0 tx=0x7f0bbc00ced0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 shutdown_connections 2024-09-21T01:42:16.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bac0b6510 0x7f0bac0b8a20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc41051a0 0x7f0bc419b260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc4104800 0x7f0bc419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 --2- 172.21.15.66:0/2880506659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bc4103770 0x7f0bc419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 >> 172.21.15.66:0/2880506659 conn(0x7f0bc40ff0e0 msgr2=0x7f0bc4107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 shutdown_connections 2024-09-21T01:42:16.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:42:16.862+0000 7f0bca222640 1 -- 172.21.15.66:0/2880506659 wait complete. 2024-09-21T01:42:17.397 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:17 smithi066 ceph-mon[35001]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:17.397 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:17 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:17.397 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:17 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:17.397 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:17 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:17.425 DEBUG:teuthology.orchestra.run.smithi066:osd.4> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T01:42:17.428 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-09-21T01:42:17.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:42:17.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:17 smithi029 ceph-mon[31228]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:17.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:17 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:17.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:17 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:17.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:17 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:17.680 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:42:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:17 smithi190 ceph-mon[34368]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:17 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:17 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:17 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:18.900 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 01:42:18 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T01:42:18.792+0000 7f8181c8a640 -1 osd.4 0 log_to_monitors true 2024-09-21T01:42:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:19 smithi066 ceph-mon[35001]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:19 smithi066 ceph-mon[35001]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:19 smithi066 ceph-mon[35001]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:19 smithi029 ceph-mon[31228]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:19.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:19 smithi029 ceph-mon[31228]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:19 smithi029 ceph-mon[31228]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:19 smithi190 ceph-mon[34368]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:19 smithi190 ceph-mon[34368]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:19 smithi190 ceph-mon[34368]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:42:19.932 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:42:19.957 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-09-21T01:42:20.199 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:42:20.350 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T01:42:20.299+0000 7f817dc0c640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T01:42:20.350 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T01:42:20.306+0000 7f8179a0a640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:42:20.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-mon[35001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:42:20.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-mon[35001]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-mon[35001]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:42:20.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:20.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:20 smithi066 ceph-mon[35001]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:20 smithi190 ceph-mon[34368]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:42:20.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:20 smithi190 ceph-mon[34368]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:20 smithi190 ceph-mon[34368]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:42:20.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:20 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:20.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:20 smithi190 ceph-mon[34368]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:20 smithi029 ceph-mon[31228]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:42:20.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:20 smithi029 ceph-mon[31228]: from='osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:20 smithi029 ceph-mon[31228]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:42:20.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:20 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:20.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:20 smithi029 ceph-mon[31228]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 -- 172.21.15.190:0/1109144981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c1036d0 msgr2=0x7fb47c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 --2- 172.21.15.190:0/1109144981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c1036d0 0x7fb47c103ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb470009900 tx=0x7fb47002a480 comp rx=0 tx=0).stop 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 -- 172.21.15.190:0/1109144981 shutdown_connections 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 --2- 172.21.15.190:0/1109144981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c104070 0x7fb47c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 --2- 172.21.15.190:0/1109144981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c1036d0 0x7fb47c103ae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 --2- 172.21.15.190:0/1109144981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 0x7fb47c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 -- 172.21.15.190:0/1109144981 >> 172.21.15.190:0/1109144981 conn(0x7fb47c0fdfb0 msgr2=0x7fb47c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 -- 172.21.15.190:0/1109144981 shutdown_connections 2024-09-21T01:42:20.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.659+0000 7fb481e2c640 1 -- 172.21.15.190:0/1109144981 wait complete. 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 Processor -- start 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 -- start start 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 0x7fb47c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c1036d0 0x7fb47c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:20.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e270 con 0x7fb47c1036d0 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e0f0 con 0x7fb47c102640 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb481e2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e3f0 con 0x7fb47c104070 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.660+0000 7fb47bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39654/0 (socket says 172.21.15.190:39654) 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 -- 172.21.15.190:0/2616275819 learned_addr learned my addr 172.21.15.190:0/2616275819 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c1036d0 0x7fb47c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47b7fe640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 0x7fb47c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 -- 172.21.15.190:0/2616275819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 msgr2=0x7fb47c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 0x7fb47c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 -- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c1036d0 msgr2=0x7fb47c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c1036d0 0x7fb47c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 -- 172.21.15.190:0/2616275819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c19b9c0 con 0x7fb47c104070 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb47bfff640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb47400eb10 tx=0x7fb47400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:42:20.663 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474004b20 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474004cc0 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c19bcb0 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474010670 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.661+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a23b0 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.662+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c1a2580 con 0x7fb47c104070 2024-09-21T01:42:20.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.664+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb474005000 con 0x7fb47c104070 2024-09-21T01:42:20.665 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.665+0000 7fb478ff9640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 0x7fb4640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:20.665 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.665+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7fb4740e69a0 con 0x7fb47c104070 2024-09-21T01:42:20.665 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.665+0000 7fb47b7fe640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 0x7fb4640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:20.666 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.666+0000 7fb47b7fe640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 0x7fb4640b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009b60 tx=0x7fb46c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:42:20.668 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.668+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb474056710 con 0x7fb47c104070 2024-09-21T01:42:20.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:20.854+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb47c0fffa0 con 0x7fb4640b6510 2024-09-21T01:42:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:42:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:42:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi066 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi066 to 8544M 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:21.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:21 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:21.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:21.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi066 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi066 to 8544M 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:21.596 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:21 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:42:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:42:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi066 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi066 to 8544M 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:21.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:21 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] boot 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:42:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:22 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:22.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:42:22.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:42:22.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:22.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] boot 2024-09-21T01:42:22.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:42:22.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:22 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: osd.4 [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] boot 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:42:22.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:22 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:42:23.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/648820842' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"}]': finished 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:42:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:23 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/648820842' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"}]': finished 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:42:23.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:23 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/648820842' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"} : dispatch 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "429eb433-7213-450d-a624-bb75cff27b4d"}]': finished 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:42:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:23 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:24 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/2536055025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:24.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:24 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/2536055025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:24.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:24 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/2536055025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:25 smithi190 ceph-mon[34368]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:25 smithi066 ceph-mon[35001]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:25 smithi029 ceph-mon[31228]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:27.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:27 smithi190 ceph-mon[34368]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:27 smithi066 ceph-mon[35001]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:27 smithi029 ceph-mon[31228]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:29.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:29 smithi190 ceph-mon[34368]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:29 smithi066 ceph-mon[35001]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:29 smithi029 ceph-mon[31228]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:31.709 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:31 smithi190 ceph-mon[34368]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:31.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:31 smithi066 ceph-mon[35001]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:31 smithi029 ceph-mon[31228]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:32.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:42:32.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:32.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:32 smithi190 ceph-mon[34368]: Deploying daemon osd.5 on smithi190 2024-09-21T01:42:32.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:42:32.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:32.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:32 smithi066 ceph-mon[35001]: Deploying daemon osd.5 on smithi190 2024-09-21T01:42:32.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:42:32.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:32.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:32 smithi029 ceph-mon[31228]: Deploying daemon osd.5 on smithi190 2024-09-21T01:42:33.469 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:33 smithi190 ceph-mon[34368]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:33.745 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:33 smithi029 ceph-mon[31228]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:33 smithi066 ceph-mon[35001]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:35 smithi190 ceph-mon[34368]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:35 smithi066 ceph-mon[35001]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:35.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:35 smithi029 ceph-mon[31228]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:37.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:37 smithi190 ceph-mon[34368]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:37 smithi066 ceph-mon[35001]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:37 smithi029 ceph-mon[31228]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:38.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:38.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:38.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:39.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:39 smithi190 ceph-mon[34368]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:39 smithi066 ceph-mon[35001]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:39 smithi029 ceph-mon[31228]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:40.907 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.907+0000 7fb478ff9640 1 -- 172.21.15.190:0/2616275819 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb47c0fffa0 con 0x7fb4640b6510 2024-09-21T01:42:40.907 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 msgr2=0x7fb4640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 0x7fb4640b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009b60 tx=0x7fb46c005f90 comp rx=0 tx=0).stop 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 msgr2=0x7fb47c19b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb47400eb10 tx=0x7fb47400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 shutdown_connections 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4640b6510 0x7fb4640b8a20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb47c104070 0x7fb47c19b1a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb47c1036d0 0x7fb47c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:40.910 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 --2- 172.21.15.190:0/2616275819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb47c102640 0x7fb47c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:40.911 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 >> 172.21.15.190:0/2616275819 conn(0x7fb47c0fdfb0 msgr2=0x7fb47c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:42:40.911 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 shutdown_connections 2024-09-21T01:42:40.911 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:42:40.909+0000 7fb481e2c640 1 -- 172.21.15.190:0/2616275819 wait complete. 2024-09-21T01:42:41.443 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:41.443 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.443 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.443 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:41.443 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:41.444 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.444 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:41.444 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.444 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.470 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T01:42:41.472 INFO:tasks.cephadm:Deploying osd.6 on smithi029 with /dev/vg_nvme/lv_2... 2024-09-21T01:42:41.472 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:42:41.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:41.742 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:42:43.505 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:43 smithi029 ceph-mon[31228]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:43.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:43 smithi066 ceph-mon[35001]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:43.697 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:43 smithi190 ceph-mon[34368]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:43.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 01:42:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T01:42:43.730+0000 7f8762338640 -1 osd.5 0 log_to_monitors true 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:44.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:44.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:42:44.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:42:44.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:44.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:42:44.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:42:44.694 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:42:44.720 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi029:vg_nvme/lv_2 2024-09-21T01:42:44.973 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi190 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:45.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:45 smithi029 ceph-mon[31228]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/3592338174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a81068e0 msgr2=0x7fb9a8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/3592338174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a81068e0 0x7fb9a8108d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009900 tx=0x7fb99c02a410 comp rx=0 tx=0).stop 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/3592338174 shutdown_connections 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/3592338174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 0x7fb9a810b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/3592338174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a81068e0 0x7fb9a8108d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/3592338174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a806c660 0x7fb9a81063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/3592338174 >> 172.21.15.29:0/3592338174 conn(0x7fb9a80fdfe0 msgr2=0x7fb9a8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:42:45.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/3592338174 shutdown_connections 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.478+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/3592338174 wait complete. 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 Processor -- start 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 -- start start 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a81068e0 0x7fb9a81022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 0x7fb9a81027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a810f390 con 0x7fb9a806c660 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a810f210 con 0x7fb9a81068e0 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9ae5f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a810f510 con 0x7fb9a8109480 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9a7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9a7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59666/0 (socket says 172.21.15.29:59666) 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9a7fff640 1 -- 172.21.15.29:0/1906056439 learned_addr learned my addr 172.21.15.29:0/1906056439 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9a77fe640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a81068e0 0x7fb9a81022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.479+0000 7fb9acb8f640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 0x7fb9a81027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 -- 172.21.15.29:0/1906056439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 msgr2=0x7fb9a81027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 0x7fb9a81027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 -- 172.21.15.29:0/1906056439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a81068e0 msgr2=0x7fb9a81022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a81068e0 0x7fb9a81022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 -- 172.21.15.29:0/1906056439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a8102ee0 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a7fff640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb99800d950 tx=0x7fb99800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998017030 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb998004020 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a81a4070 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980047a0 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.480+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a81a44d0 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.481+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80ffb00 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.484+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb998009a30 con 0x7fb9a806c660 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.484+0000 7fb9a57fa640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 0x7fb9900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:42:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.485+0000 7fb9a77fe640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 0x7fb9900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:42:45.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.485+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7fb9980e6390 con 0x7fb9a806c660 2024-09-21T01:42:45.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.485+0000 7fb9a77fe640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 0x7fb9900b8a20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c00 tx=0x7fb99c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:42:45.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.488+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9980adc00 con 0x7fb9a806c660 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi190 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:45 smithi066 ceph-mon[35001]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:42:45.686+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb9a8063800 con 0x7fb9900b6510 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi190 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: from='osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:45 smithi190 ceph-mon[34368]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:46.324 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:46.325 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:46.325 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:46.325 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:46 smithi029 ceph-mon[31228]: from='osd.5 ' entity='osd.5' 2024-09-21T01:42:46.478 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T01:42:46.171+0000 7f875e2ba640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T01:42:46.478 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T01:42:46.340+0000 7f87560b0640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:42:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:42:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:42:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:46.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:46 smithi190 ceph-mon[34368]: from='osd.5 ' entity='osd.5' 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:46.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:46 smithi066 ceph-mon[35001]: from='osd.5 ' entity='osd.5' 2024-09-21T01:42:47.470 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:42:47.470 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:42:47.470 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:47.470 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.471 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:42:47.471 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.471 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:47.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] boot 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"} : dispatch 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"}]': finished 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: osdmap e37: 7 total, 6 up, 7 in 2024-09-21T01:42:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:48 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] boot 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"} : dispatch 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"}]': finished 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: osdmap e37: 7 total, 6 up, 7 in 2024-09-21T01:42:48.580 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: osd.5 [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] boot 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"} : dispatch 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/174606713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d0fb70-1bc7-4a2d-9a80-eff59ed00770"}]': finished 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: osdmap e37: 7 total, 6 up, 7 in 2024-09-21T01:42:48.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:48 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:49 smithi066 ceph-mon[35001]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:49 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3774055123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:49 smithi066 ceph-mon[35001]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:42:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:49 smithi029 ceph-mon[31228]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:49 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3774055123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:49 smithi029 ceph-mon[31228]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:42:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:49 smithi190 ceph-mon[34368]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:42:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:49 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3774055123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:42:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:49 smithi190 ceph-mon[34368]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:42:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:42:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:51 smithi066 ceph-mon[35001]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:42:51.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:51 smithi029 ceph-mon[31228]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:42:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:51 smithi190 ceph-mon[34368]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:42:53.469 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:53 smithi029 ceph-mon[31228]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:42:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:53 smithi066 ceph-mon[35001]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:42:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:53 smithi190 ceph-mon[34368]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:42:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:55 smithi029 ceph-mon[31228]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:42:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:55 smithi190 ceph-mon[34368]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:42:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:55 smithi066 ceph-mon[35001]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:42:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:57 smithi190 ceph-mon[34368]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:57 smithi029 ceph-mon[31228]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:57.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:57 smithi066 ceph-mon[35001]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:42:58.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:42:58.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:58.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:58 smithi029 ceph-mon[31228]: Deploying daemon osd.6 on smithi029 2024-09-21T01:42:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:42:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:58.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:58 smithi190 ceph-mon[34368]: Deploying daemon osd.6 on smithi029 2024-09-21T01:42:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:42:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:42:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:58 smithi066 ceph-mon[35001]: Deploying daemon osd.6 on smithi029 2024-09-21T01:42:59.400 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:42:59 smithi029 ceph-mon[31228]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:42:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:42:59 smithi190 ceph-mon[34368]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:42:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:42:59 smithi066 ceph-mon[35001]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:43:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:01 smithi029 ceph-mon[31228]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T01:43:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:01 smithi190 ceph-mon[34368]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T01:43:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:01 smithi066 ceph-mon[35001]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T01:43:03.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:03 smithi029 ceph-mon[31228]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:03.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:03 smithi190 ceph-mon[34368]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:03.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:03 smithi066 ceph-mon[35001]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:04.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:04.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:04.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:05 smithi029 ceph-mon[31228]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:05 smithi190 ceph-mon[34368]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:05 smithi066 ceph-mon[35001]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:07.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:07.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.187+0000 7fb9a57fa640 1 -- 172.21.15.29:0/1906056439 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9a8063800 con 0x7fb9900b6510 2024-09-21T01:43:08.188 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 6 on host 'smithi029' 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 msgr2=0x7fb9900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 0x7fb9900b8a20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c00 tx=0x7fb99c031400 comp rx=0 tx=0).stop 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 msgr2=0x7fb9a81039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb99800d950 tx=0x7fb99800de10 comp rx=0 tx=0).stop 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 shutdown_connections 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb9900b6510 0x7fb9900b8a20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a8109480 0x7fb9a81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a81068e0 0x7fb9a81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 --2- 172.21.15.29:0/1906056439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9a806c660 0x7fb9a81039d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 >> 172.21.15.29:0/1906056439 conn(0x7fb9a80fdfe0 msgr2=0x7fb9a8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:43:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 shutdown_connections 2024-09-21T01:43:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:43:08.190+0000 7fb9ae5f2640 1 -- 172.21.15.29:0/1906056439 wait complete. 2024-09-21T01:43:08.577 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:08.577 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.577 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:08.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:08.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:08.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:09.058 DEBUG:teuthology.orchestra.run.smithi029:osd.6> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T01:43:09.061 INFO:tasks.cephadm:Deploying osd.7 on smithi066 with /dev/vg_nvme/lv_2... 2024-09-21T01:43:09.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:43:09.332 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:43:09.440 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:09 smithi066 ceph-mon[35001]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:09 smithi029 ceph-mon[31228]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:09.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:09 smithi190 ceph-mon[34368]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:10.332 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 01:43:10 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T01:43:10.041+0000 7f464e060640 -1 osd.6 0 log_to_monitors true 2024-09-21T01:43:10.588 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:10 smithi029 ceph-mon[31228]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:43:10.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:10 smithi190 ceph-mon[34368]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:43:10.739 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:10 smithi066 ceph-mon[35001]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:43:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:11 smithi066 ceph-mon[35001]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:11 smithi066 ceph-mon[35001]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:43:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:11 smithi066 ceph-mon[35001]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:43:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:11 smithi066 ceph-mon[35001]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:43:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:11 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:11.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:11 smithi029 ceph-mon[31228]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:11.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:11 smithi029 ceph-mon[31228]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:43:11.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:11 smithi029 ceph-mon[31228]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:43:11.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:11 smithi029 ceph-mon[31228]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:43:11.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:11 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:11.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:11 smithi190 ceph-mon[34368]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:11.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:11 smithi190 ceph-mon[34368]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:43:11.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:11 smithi190 ceph-mon[34368]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:43:11.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:11 smithi190 ceph-mon[34368]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:43:11.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:11 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:11.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:43:11.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-09-21T01:43:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:43:12.323 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T01:43:12.112+0000 7f464a7f7640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T01:43:12.323 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T01:43:12.121+0000 7f4641dd8640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:43:12.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:12 smithi066 ceph-mon[35001]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:43:12.589 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:12 smithi066 ceph-mon[35001]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:43:12.589 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:12 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.589 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:12 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.589 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:12 smithi066 ceph-mon[35001]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' 2024-09-21T01:43:12.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-mon[31228]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:43:12.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-mon[31228]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:43:12.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:12 smithi029 ceph-mon[31228]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 -- 172.21.15.66:0/293790921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c0105180 msgr2=0x7f94c010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 --2- 172.21.15.66:0/293790921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c0105180 0x7f94c010bb20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ca0 tx=0x7f94b802a950 comp rx=0 tx=0).stop 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 -- 172.21.15.66:0/293790921 shutdown_connections 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 --2- 172.21.15.66:0/293790921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c0105180 0x7f94c010bb20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 --2- 172.21.15.66:0/293790921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c01047e0 0x7f94c0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 --2- 172.21.15.66:0/293790921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94c0103750 0x7f94c0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.683+0000 7f94c4c87640 1 -- 172.21.15.66:0/293790921 >> 172.21.15.66:0/293790921 conn(0x7f94c00ff0c0 msgr2=0x7f94c0101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.684+0000 7f94c4c87640 1 -- 172.21.15.66:0/293790921 shutdown_connections 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.684+0000 7f94c4c87640 1 -- 172.21.15.66:0/293790921 wait complete. 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.684+0000 7f94c4c87640 1 Processor -- start 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.684+0000 7f94c4c87640 1 -- start start 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 0x7f94c0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94c0105180 0x7f94c019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010f3a0 con 0x7f94c0103750 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010f220 con 0x7f94c01047e0 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94c4c87640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010f520 con 0x7f94c0105180 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43352/0 (socket says 172.21.15.66:43352) 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 -- 172.21.15.66:0/3789396468 learned_addr learned my addr 172.21.15.66:0/3789396468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94be59c640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 0x7f94c0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 -- 172.21.15.66:0/3789396468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94c0105180 msgr2=0x7f94c019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94c0105180 0x7f94c019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 -- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 msgr2=0x7f94c0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 0x7f94c0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 -- 172.21.15.66:0/3789396468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c019b9a0 con 0x7f94c01047e0 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94be59c640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 0x7f94c0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b7fff640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f94a800e9f0 tx=0x7f94a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:43:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.685+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a800cd80 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.686+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a80108e0 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.686+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c019bc90 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.686+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8018980 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.686+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01a2390 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.687+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c010a930 con 0x7f94c01047e0 2024-09-21T01:43:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.688+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f94a8010430 con 0x7f94c01047e0 2024-09-21T01:43:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.689+0000 7f94b77fe640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 0x7f94ac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.690+0000 7f94be59c640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 0x7f94ac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.690+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f94a80e6d90 con 0x7f94c01047e0 2024-09-21T01:43:12.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.690+0000 7f94be59c640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 0x7f94ac0b8a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005fd0 tx=0x7f94b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:43:12.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.693+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94a80ae430 con 0x7f94c01047e0 2024-09-21T01:43:12.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:12 smithi190 ceph-mon[34368]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:43:12.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:12 smithi190 ceph-mon[34368]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:43:12.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:12 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:12 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:12.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:12 smithi190 ceph-mon[34368]: from='osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744]' entity='osd.6' 2024-09-21T01:43:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:12.894+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f94c0063800 con 0x7f94ac0b6510 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi029 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi029 to 5697M 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] boot 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:43:13.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:13 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:43:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:43:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi029 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi029 to 5697M 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] boot 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:43:13.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:13 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi029 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi029 to 5697M 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: osd.6 [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] boot 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:43:13.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:13 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:43:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:14 smithi066 ceph-mon[35001]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:14.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:14 smithi190 ceph-mon[34368]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:14.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:14 smithi029 ceph-mon[31228]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:15.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/2054046905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"}]': finished 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:43:15.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:15 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T01:43:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/2054046905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"}]': finished 2024-09-21T01:43:15.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:43:15.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:15 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:15.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:15.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T01:43:15.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/2054046905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"} : dispatch 2024-09-21T01:43:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bf68e6-9205-4bee-9d74-38c475daeada"}]': finished 2024-09-21T01:43:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:43:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:15 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:16.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/2002182511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:16.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/2002182511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:16.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/2002182511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:17 smithi190 ceph-mon[34368]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:17.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:17 smithi066 ceph-mon[35001]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:17.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:17 smithi029 ceph-mon[31228]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:19 smithi190 ceph-mon[34368]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:19 smithi066 ceph-mon[35001]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:19.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:19 smithi029 ceph-mon[31228]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:21.429 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:21 smithi066 ceph-mon[35001]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:21 smithi190 ceph-mon[34368]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:21 smithi029 ceph-mon[31228]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:23.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:23 smithi066 ceph-mon[35001]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:23.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:23 smithi190 ceph-mon[34368]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:23 smithi029 ceph-mon[31228]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:24.703 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:43:24.703 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:43:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:24.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:43:24.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:25.646 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:25 smithi066 ceph-mon[35001]: Deploying daemon osd.7 on smithi066 2024-09-21T01:43:25.646 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:25 smithi066 ceph-mon[35001]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:25 smithi190 ceph-mon[34368]: Deploying daemon osd.7 on smithi066 2024-09-21T01:43:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:25 smithi190 ceph-mon[34368]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:25 smithi029 ceph-mon[31228]: Deploying daemon osd.7 on smithi066 2024-09-21T01:43:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:25 smithi029 ceph-mon[31228]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:27.539 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:27 smithi066 ceph-mon[35001]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:27.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:27 smithi190 ceph-mon[34368]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:27 smithi029 ceph-mon[31228]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:29.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:29 smithi066 ceph-mon[35001]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:29.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:29 smithi190 ceph-mon[34368]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:29 smithi029 ceph-mon[31228]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:30.710 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:30.711 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:30.711 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:30.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:30.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:30.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:30.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:31.607 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:31 smithi066 ceph-mon[35001]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:31.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:31 smithi190 ceph-mon[34368]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:31 smithi029 ceph-mon[31228]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:33.765 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.807+0000 7f94b77fe640 1 -- 172.21.15.66:0/3789396468 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f94c0063800 con 0x7f94ac0b6510 2024-09-21T01:43:33.808 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 7 on host 'smithi066' 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 msgr2=0x7f94ac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 0x7f94ac0b8a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005fd0 tx=0x7f94b000b390 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 msgr2=0x7f94c019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f94a800e9f0 tx=0x7f94a800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 shutdown_connections 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94ac0b6510 0x7f94ac0b8a20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94c0105180 0x7f94c019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94c01047e0 0x7f94c019a5e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 --2- 172.21.15.66:0/3789396468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94c0103750 0x7f94c0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.809+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 >> 172.21.15.66:0/3789396468 conn(0x7f94c00ff0c0 msgr2=0x7f94c0107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.810+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 shutdown_connections 2024-09-21T01:43:33.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:43:33.810+0000 7f94c4c87640 1 -- 172.21.15.66:0/3789396468 wait complete. 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:34.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:34.354 DEBUG:teuthology.orchestra.run.smithi066:osd.7> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T01:43:34.357 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2024-09-21T01:43:34.357 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:43:34.610 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:43:34.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:34.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:34.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:35.922 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:35 smithi190 ceph-mon[34368]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:35 smithi066 ceph-mon[35001]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:36.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 01:43:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T01:43:35.909+0000 7f9252947640 -1 osd.7 0 log_to_monitors true 2024-09-21T01:43:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:35 smithi029 ceph-mon[31228]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:36.910 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:43:36.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-09-21T01:43:37.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:36 smithi066 ceph-mon[35001]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:36 smithi066 ceph-mon[35001]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:36 smithi029 ceph-mon[31228]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:36 smithi029 ceph-mon[31228]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.198 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:43:37.227 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:36 smithi190 ceph-mon[34368]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:36 smithi190 ceph-mon[34368]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:43:37.702 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.701+0000 7f6b58e30640 1 -- 172.21.15.190:0/573571578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b54105180 msgr2=0x7f6b5410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.701+0000 7f6b58e30640 1 --2- 172.21.15.190:0/573571578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b54105180 0x7f6b5410bb20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009ca0 tx=0x7f6b4c02a950 comp rx=0 tx=0).stop 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 -- 172.21.15.190:0/573571578 shutdown_connections 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 --2- 172.21.15.190:0/573571578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b54105180 0x7f6b5410bb20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 --2- 172.21.15.190:0/573571578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b541047e0 0x7f6b54104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 --2- 172.21.15.190:0/573571578 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54103750 0x7f6b54103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 -- 172.21.15.190:0/573571578 >> 172.21.15.190:0/573571578 conn(0x7f6b540ff0c0 msgr2=0x7f6b54101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 -- 172.21.15.190:0/573571578 shutdown_connections 2024-09-21T01:43:37.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 -- 172.21.15.190:0/573571578 wait complete. 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.702+0000 7f6b58e30640 1 Processor -- start 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 -- start start 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b54103750 0x7f6b54199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54105180 0x7f6b5419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410f3a0 con 0x7f6b54105180 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410f220 con 0x7f6b54103750 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b58e30640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410f520 con 0x7f6b541047e0 2024-09-21T01:43:37.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53952/0 (socket says 172.21.15.190:53952) 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 -- 172.21.15.190:0/3093558451 learned_addr learned my addr 172.21.15.190:0/3093558451 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b5259c640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b54103750 0x7f6b54199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b52d9d640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54105180 0x7f6b5419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 -- 172.21.15.190:0/3093558451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b54103750 msgr2=0x7f6b54199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b54103750 0x7f6b54199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 -- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54105180 msgr2=0x7f6b5419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54105180 0x7f6b5419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 -- 172.21.15.190:0/3093558451 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5419b8e0 con 0x7f6b541047e0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.703+0000 7f6b51d9b640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800e9f0 tx=0x7f6b4800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.704+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800ccc0 con 0x7f6b541047e0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.704+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b4800ce60 con 0x7f6b541047e0 2024-09-21T01:43:37.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.704+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48010430 con 0x7f6b541047e0 2024-09-21T01:43:37.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.704+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5419bbd0 con 0x7f6b541047e0 2024-09-21T01:43:37.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.704+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b541a2190 con 0x7f6b541047e0 2024-09-21T01:43:37.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.705+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5410a910 con 0x7f6b541047e0 2024-09-21T01:43:37.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.707+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6b480046c0 con 0x7f6b541047e0 2024-09-21T01:43:37.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.708+0000 7f6b437fe640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 0x7f6b3c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:43:37.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.708+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5101+0+0 (secure 0 0 0) 0x7f6b480e6c00 con 0x7f6b541047e0 2024-09-21T01:43:37.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.708+0000 7f6b5259c640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 0x7f6b3c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:43:37.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.709+0000 7f6b5259c640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 0x7f6b3c0b8a20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400abe0 tx=0x7f6b4400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:43:37.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.711+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b48056710 con 0x7f6b541047e0 2024-09-21T01:43:37.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:37.900+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6b54063800 con 0x7f6b3c0b6510 2024-09-21T01:43:37.900 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:37.901 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:37.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:38.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:38.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:38.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi066 2024-09-21T01:43:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi066 to 5696M 2024-09-21T01:43:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:43:38.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='osd.7 ' entity='osd.7' 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:43:38.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:38 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T01:43:38.765+0000 7f924e8c9640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T01:43:39.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T01:43:38.892+0000 7f92466bf640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi066 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi066 to 5696M 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='osd.7 ' entity='osd.7' 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:43:39.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:38 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi066 2024-09-21T01:43:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi066 to 5696M 2024-09-21T01:43:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:43:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:43:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='osd.7 ' entity='osd.7' 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:43:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:38 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:43:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] boot 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T01:43:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] boot 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T01:43:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: osd.7 [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] boot 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T01:43:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:43:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: pgmap v191: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/825192015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"}]': finished 2024-09-21T01:43:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T01:43:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:40 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/3961714570' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: pgmap v191: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/825192015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"}]': finished 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:40 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/3961714570' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: pgmap v191: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/825192015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"} : dispatch 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fafc4cd9-401d-402c-8248-1a219ef74397"}]': finished 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:41.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:40 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/3961714570' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:43:42.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:42 smithi190 ceph-mon[34368]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T01:43:42.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:42 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:42.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:42 smithi066 ceph-mon[35001]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T01:43:42.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:42 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:42.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:42 smithi029 ceph-mon[31228]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T01:43:42.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:42 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:43:43.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:43 smithi190 ceph-mon[34368]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:43.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:43 smithi066 ceph-mon[35001]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:43.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:43 smithi029 ceph-mon[31228]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:45.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:45 smithi190 ceph-mon[34368]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:45 smithi066 ceph-mon[35001]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:45 smithi029 ceph-mon[31228]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:47.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:47 smithi190 ceph-mon[34368]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s, 0 objects/s recovering 2024-09-21T01:43:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:47 smithi066 ceph-mon[35001]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s, 0 objects/s recovering 2024-09-21T01:43:47.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:47 smithi029 ceph-mon[31228]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s, 0 objects/s recovering 2024-09-21T01:43:49.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:49 smithi190 ceph-mon[34368]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:43:49.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:43:49.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:49.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:49 smithi190 ceph-mon[34368]: Deploying daemon osd.8 on smithi190 2024-09-21T01:43:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:49 smithi066 ceph-mon[35001]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:43:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:43:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:49 smithi066 ceph-mon[35001]: Deploying daemon osd.8 on smithi190 2024-09-21T01:43:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:49 smithi029 ceph-mon[31228]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T01:43:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:43:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:49 smithi029 ceph-mon[31228]: Deploying daemon osd.8 on smithi190 2024-09-21T01:43:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:51 smithi190 ceph-mon[34368]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T01:43:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:51 smithi066 ceph-mon[35001]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T01:43:51.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:51 smithi029 ceph-mon[31228]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T01:43:53.447 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:53 smithi190 ceph-mon[34368]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-21T01:43:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:53 smithi066 ceph-mon[35001]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-21T01:43:53.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:53 smithi029 ceph-mon[31228]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-21T01:43:55.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:55 smithi190 ceph-mon[34368]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:55.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:55.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:55.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:55.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:55 smithi066 ceph-mon[35001]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:55.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:55.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:55.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:55.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:55 smithi029 ceph-mon[31228]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:55.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:55.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:55.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:57 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:57 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:43:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:43:57.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:43:57.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:57 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:57.979 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.979+0000 7f6b437fe640 1 -- 172.21.15.190:0/3093558451 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b54063800 con 0x7f6b3c0b6510 2024-09-21T01:43:57.980 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2024-09-21T01:43:57.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.981+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 msgr2=0x7f6b3c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:57.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.981+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 0x7f6b3c0b8a20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400abe0 tx=0x7f6b4400a2d0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.981+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 msgr2=0x7f6b5419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800e9f0 tx=0x7f6b4800edf0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 shutdown_connections 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6b3c0b6510 0x7f6b3c0b8a20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b54105180 0x7f6b5419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b541047e0 0x7f6b5419a5b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 --2- 172.21.15.190:0/3093558451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b54103750 0x7f6b54199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 >> 172.21.15.190:0/3093558451 conn(0x7f6b540ff0c0 msgr2=0x7f6b54107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 shutdown_connections 2024-09-21T01:43:57.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:43:57.982+0000 7f6b58e30640 1 -- 172.21.15.190:0/3093558451 wait complete. 2024-09-21T01:43:58.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:58.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:58.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:43:58.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:43:58.612 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T01:43:58.614 INFO:tasks.cephadm:Deploying osd.9 on smithi029 with /dev/vg_nvme/lv_1... 2024-09-21T01:43:58.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:43:58.887 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:43:59.292 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:43:59 smithi029 ceph-mon[31228]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:59.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:43:59 smithi190 ceph-mon[34368]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:43:59.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:43:59 smithi066 ceph-mon[35001]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:00.978 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 01:44:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T01:44:00.748+0000 7f58e4f85640 -1 osd.8 0 log_to_monitors true 2024-09-21T01:44:01.310 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:01 smithi190 ceph-mon[34368]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:01.310 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:01 smithi190 ceph-mon[34368]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.310 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:01 smithi190 ceph-mon[34368]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.512 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:01 smithi029 ceph-mon[31228]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:01.512 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:01 smithi029 ceph-mon[31228]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.512 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:01 smithi029 ceph-mon[31228]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:01 smithi066 ceph-mon[35001]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:01.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:01 smithi066 ceph-mon[35001]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:01 smithi066 ceph-mon[35001]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:44:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:44:01.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi029:vg_nvme/lv_1 2024-09-21T01:44:02.107 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi190 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.534 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:02.535 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi190 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-21T01:44:02.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:02.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:02.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi190 2024-09-21T01:44:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:02.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 -- 172.21.15.29:0/751044692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 msgr2=0x7fef6c10a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 --2- 172.21.15.29:0/751044692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c10a950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef64009ca0 tx=0x7fef6402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 -- 172.21.15.29:0/751044692 shutdown_connections 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 --2- 172.21.15.29:0/751044692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c10a950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 --2- 172.21.15.29:0/751044692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 0x7fef6c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 --2- 172.21.15.29:0/751044692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 0x7fef6c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 -- 172.21.15.29:0/751044692 >> 172.21.15.29:0/751044692 conn(0x7fef6c0fd800 msgr2=0x7fef6c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 -- 172.21.15.29:0/751044692 shutdown_connections 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.632+0000 7fef73bad640 1 -- 172.21.15.29:0/751044692 wait complete. 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 Processor -- start 2024-09-21T01:44:02.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 -- start start 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 0x7fef6c104a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 0x7fef6c1032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10e290 con 0x7fef6c108440 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10e110 con 0x7fef6c105820 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef73bad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10e410 con 0x7fef6c102dc0 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef71949640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 0x7fef6c104a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60654/0 (socket says 172.21.15.29:60654) 2024-09-21T01:44:02.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 -- 172.21.15.29:0/4183078162 learned_addr learned my addr 172.21.15.29:0/4183078162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef71148640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 0x7fef6c1032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 -- 172.21.15.29:0/4183078162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 msgr2=0x7fef6c104a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 0x7fef6c104a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 -- 172.21.15.29:0/4183078162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 msgr2=0x7fef6c1032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 0x7fef6c1032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.633+0000 7fef7214a640 1 -- 172.21.15.29:0/4183078162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c103f30 con 0x7fef6c108440 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef7214a640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef640097b0 tx=0x7fef64031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64003bf0 con 0x7fef6c108440 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef64003d90 con 0x7fef6c108440 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1a3ff0 con 0x7fef6c108440 2024-09-21T01:44:02.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6402fc30 con 0x7fef6c108440 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.634+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a4390 con 0x7fef6c108440 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.635+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c105cf0 con 0x7fef6c108440 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.636+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fef64004040 con 0x7fef6c108440 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.637+0000 7fef62ffd640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 0x7fef580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.637+0000 7fef71949640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 0x7fef580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.637+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7fef64107e40 con 0x7fef6c108440 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.638+0000 7fef71949640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 0x7fef580b8a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef68009bd0 tx=0x7fef68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:02.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.640+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef640cf100 con 0x7fef6c108440 2024-09-21T01:44:02.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:02.842+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef6c1a4680 con 0x7fef580b6510 2024-09-21T01:44:03.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T01:44:03.144+0000 7f58e0f07640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T01:44:03.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T01:44:03.264+0000 7f58d8cfd640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:44:03.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:03.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:03 smithi190 ceph-mon[34368]: from='osd.8 ' entity='osd.8' 2024-09-21T01:44:03.508 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:03.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:03 smithi029 ceph-mon[31228]: from='osd.8 ' entity='osd.8' 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:03 smithi066 ceph-mon[35001]: from='osd.8 ' entity='osd.8' 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:04.541 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:04.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:05 smithi066 ceph-mon[35001]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:05 smithi066 ceph-mon[35001]: osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] boot 2024-09-21T01:44:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:05 smithi066 ceph-mon[35001]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T01:44:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:05 smithi029 ceph-mon[31228]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:05 smithi029 ceph-mon[31228]: osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] boot 2024-09-21T01:44:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:05 smithi029 ceph-mon[31228]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T01:44:05.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:05 smithi190 ceph-mon[34368]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:44:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:05 smithi190 ceph-mon[34368]: osd.8 [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] boot 2024-09-21T01:44:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:05 smithi190 ceph-mon[34368]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T01:44:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"} : dispatch 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"}]': finished 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:06.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2884356152' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:06.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:44:06.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"} : dispatch 2024-09-21T01:44:06.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"}]': finished 2024-09-21T01:44:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T01:44:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2884356152' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"} : dispatch 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/720342223' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d46d9b84-5641-4633-9077-c086f62e0617"}]': finished 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2884356152' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:07 smithi066 ceph-mon[35001]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:07.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:07 smithi029 ceph-mon[31228]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:07 smithi190 ceph-mon[34368]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:09.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:09 smithi066 ceph-mon[35001]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:09.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:09 smithi029 ceph-mon[31228]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:09.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:09 smithi190 ceph-mon[34368]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:11.756 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:11 smithi029 ceph-mon[31228]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:44:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:11 smithi066 ceph-mon[35001]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:44:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:11 smithi190 ceph-mon[34368]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:44:13.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:13 smithi029 ceph-mon[31228]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T01:44:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:13 smithi066 ceph-mon[35001]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T01:44:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:13 smithi190 ceph-mon[34368]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T01:44:14.609 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:14 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:44:14.609 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:14 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:14.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:14 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:44:14.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:14 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:14.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:14 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:44:14.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:14 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:15.761 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:15 smithi029 ceph-mon[31228]: Deploying daemon osd.9 on smithi029 2024-09-21T01:44:15.761 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:15 smithi029 ceph-mon[31228]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T01:44:15.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:15 smithi066 ceph-mon[35001]: Deploying daemon osd.9 on smithi029 2024-09-21T01:44:15.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:15 smithi066 ceph-mon[35001]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T01:44:15.879 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:15 smithi190 ceph-mon[34368]: Deploying daemon osd.9 on smithi029 2024-09-21T01:44:15.879 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:15 smithi190 ceph-mon[34368]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T01:44:17.778 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:17 smithi029 ceph-mon[31228]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:17 smithi066 ceph-mon[35001]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T01:44:17.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:17 smithi190 ceph-mon[34368]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T01:44:19.796 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:19 smithi029 ceph-mon[31228]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:19.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:19 smithi066 ceph-mon[35001]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:19 smithi190 ceph-mon[34368]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:20 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:20 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:20 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:20.801 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:20 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:20.801 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:20 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:20.801 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:20 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:20 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:20 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:20 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:21 smithi066 ceph-mon[35001]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:21 smithi029 ceph-mon[31228]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:21 smithi190 ceph-mon[34368]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:44:23.686 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:23 smithi029 ceph-mon[31228]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:23.725 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 9 on host 'smithi029' 2024-09-21T01:44:23.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.724+0000 7fef62ffd640 1 -- 172.21.15.29:0/4183078162 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef6c1a4680 con 0x7fef580b6510 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 msgr2=0x7fef580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 0x7fef580b8a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef68009bd0 tx=0x7fef68005f90 comp rx=0 tx=0).stop 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 msgr2=0x7fef6c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef640097b0 tx=0x7fef64031840 comp rx=0 tx=0).stop 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 shutdown_connections 2024-09-21T01:44:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef580b6510 0x7fef580b8a20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6c108440 0x7fef6c103830 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c105820 0x7fef6c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 --2- 172.21.15.29:0/4183078162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef6c102dc0 0x7fef6c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 >> 172.21.15.29:0/4183078162 conn(0x7fef6c0fd800 msgr2=0x7fef6c1079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 shutdown_connections 2024-09-21T01:44:23.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:44:23.727+0000 7fef73bad640 1 -- 172.21.15.29:0/4183078162 wait complete. 2024-09-21T01:44:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:23 smithi066 ceph-mon[35001]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:23 smithi190 ceph-mon[34368]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:24.405 DEBUG:teuthology.orchestra.run.smithi029:osd.9> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T01:44:24.408 INFO:tasks.cephadm:Deploying osd.10 on smithi066 with /dev/vg_nvme/lv_1... 2024-09-21T01:44:24.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:44:24.681 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:44:24.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:24.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:24.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:24.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:25 smithi029 ceph-mon[31228]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:25 smithi029 ceph-mon[31228]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:44:25.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 01:44:25 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T01:44:25.557+0000 7f2dc182a640 -1 osd.9 0 log_to_monitors true 2024-09-21T01:44:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:25 smithi066 ceph-mon[35001]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:25 smithi066 ceph-mon[35001]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:44:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:25 smithi190 ceph-mon[34368]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:25 smithi190 ceph-mon[34368]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:44:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:44:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.083 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:44:27.083 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T01:44:27.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]} : dispatch 2024-09-21T01:44:27.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: from='osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-09-21T01:44:27.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:44:27.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.084 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T01:44:26.745+0000 7f2dbd7ac640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T01:44:27.084 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 01:44:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T01:44:26.762+0000 7f2db95aa640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:44:27.156 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-21T01:44:27.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-09-21T01:44:27.451 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:44:27.915 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:27 smithi066 ceph-mon[35001]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:27.915 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:27 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.915 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:27 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:27.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 -- 172.21.15.66:0/1616152961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd14102640 msgr2=0x7fdd14102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 --2- 172.21.15.66:0/1616152961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd14102640 0x7fdd14102ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009900 tx=0x7fdd0402a480 comp rx=0 tx=0).stop 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 -- 172.21.15.66:0/1616152961 shutdown_connections 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 --2- 172.21.15.66:0/1616152961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 0x7fdd1410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 --2- 172.21.15.66:0/1616152961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd141036d0 0x7fdd14103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 --2- 172.21.15.66:0/1616152961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd14102640 0x7fdd14102ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 -- 172.21.15.66:0/1616152961 >> 172.21.15.66:0/1616152961 conn(0x7fdd140fdfb0 msgr2=0x7fdd14100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 -- 172.21.15.66:0/1616152961 shutdown_connections 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.937+0000 7fdd19b82640 1 -- 172.21.15.66:0/1616152961 wait complete. 2024-09-21T01:44:27.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 Processor -- start 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 -- start start 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd14102640 0x7fdd1406b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 0x7fdd141a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e260 con 0x7fdd14104070 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e0e0 con 0x7fdd141036d0 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd19b82640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e3e0 con 0x7fdd14102640 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.938+0000 7fdd12ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44788/0 (socket says 172.21.15.66:44788) 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 -- 172.21.15.66:0/647791293 learned_addr learned my addr 172.21.15.66:0/647791293 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd13fff640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 0x7fdd141a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd137fe640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd14102640 0x7fdd1406b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 -- 172.21.15.66:0/647791293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd14102640 msgr2=0x7fdd1406b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd14102640 0x7fdd1406b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 -- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 msgr2=0x7fdd141a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 0x7fdd141a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 -- 172.21.15.66:0/647791293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141a1a60 con 0x7fdd141036d0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd12ffd640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00d950 tx=0x7fdcfc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc017070 con 0x7fdd141036d0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141a1d50 con 0x7fdd141036d0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcfc009c30 con 0x7fdd141036d0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00f620 con 0x7fdd141036d0 2024-09-21T01:44:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.939+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a2230 con 0x7fdd141036d0 2024-09-21T01:44:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.941+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd14109820 con 0x7fdd141036d0 2024-09-21T01:44:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.942+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdcfc005030 con 0x7fdd141036d0 2024-09-21T01:44:27.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.943+0000 7fdd10ff9640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 0x7fdd000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:27.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.943+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6261+0+0 (secure 0 0 0) 0x7fdcfc0e6630 con 0x7fdd141036d0 2024-09-21T01:44:27.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.943+0000 7fdd137fe640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 0x7fdd000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:27.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.944+0000 7fdd137fe640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 0x7fdd000b8a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000c00 tx=0x7fdd04031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:27.947+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcfc009de0 con 0x7fdd141036d0 2024-09-21T01:44:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:27 smithi029 ceph-mon[31228]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:27 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:27 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:28.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:28.147+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdd14063800 con 0x7fdd000b6510 2024-09-21T01:44:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:27 smithi190 ceph-mon[34368]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:27 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:27 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:44:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:44:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] boot 2024-09-21T01:44:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T01:44:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:29.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:29.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] boot 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:29.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: osd.9 [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] boot 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:29.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:29.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:29 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi029 2024-09-21T01:44:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:29 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi029 to 4273M 2024-09-21T01:44:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:29 smithi066 ceph-mon[35001]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:29 smithi066 ceph-mon[35001]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:29 smithi066 ceph-mon[35001]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T01:44:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:29 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi029 2024-09-21T01:44:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:29 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi029 to 4273M 2024-09-21T01:44:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:29 smithi029 ceph-mon[31228]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:29 smithi029 ceph-mon[31228]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:29 smithi029 ceph-mon[31228]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T01:44:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:29 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi029 2024-09-21T01:44:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:29 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi029 to 4273M 2024-09-21T01:44:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:29 smithi190 ceph-mon[34368]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:44:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:29 smithi190 ceph-mon[34368]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:29 smithi190 ceph-mon[34368]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T01:44:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:30 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/3598471901' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:30 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:30 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"}]': finished 2024-09-21T01:44:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:30 smithi066 ceph-mon[35001]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T01:44:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:30 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/3598471901' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:30 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:30 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"}]': finished 2024-09-21T01:44:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:30 smithi029 ceph-mon[31228]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T01:44:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:30 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/3598471901' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:30 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"} : dispatch 2024-09-21T01:44:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:30 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd901192-e9bc-46ef-835f-a4dd14265604"}]': finished 2024-09-21T01:44:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:30 smithi190 ceph-mon[34368]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T01:44:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:31 smithi029 ceph-mon[31228]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:31 smithi029 ceph-mon[31228]: from='client.? 172.21.15.66:0/2466511503' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:31 smithi190 ceph-mon[34368]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:31 smithi190 ceph-mon[34368]: from='client.? 172.21.15.66:0/2466511503' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:31 smithi066 ceph-mon[35001]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:31 smithi066 ceph-mon[35001]: from='client.? 172.21.15.66:0/2466511503' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:33 smithi029 ceph-mon[31228]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:33 smithi190 ceph-mon[34368]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:34.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:33 smithi066 ceph-mon[35001]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:36.004 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:35 smithi066 ceph-mon[35001]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:35 smithi029 ceph-mon[31228]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:35 smithi190 ceph-mon[34368]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:37 smithi066 ceph-mon[35001]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:37 smithi029 ceph-mon[31228]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:37 smithi190 ceph-mon[34368]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:40.005 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:39 smithi066 ceph-mon[35001]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:40.005 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:44:40.005 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:39 smithi029 ceph-mon[31228]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:44:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:39 smithi190 ceph-mon[34368]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:44:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:41.044 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:40 smithi066 ceph-mon[35001]: Deploying daemon osd.10 on smithi066 2024-09-21T01:44:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:40 smithi029 ceph-mon[31228]: Deploying daemon osd.10 on smithi066 2024-09-21T01:44:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:40 smithi190 ceph-mon[34368]: Deploying daemon osd.10 on smithi066 2024-09-21T01:44:42.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:41 smithi066 ceph-mon[35001]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:41 smithi029 ceph-mon[31228]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:41 smithi190 ceph-mon[34368]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:44.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:43 smithi066 ceph-mon[35001]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:43 smithi029 ceph-mon[31228]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:43 smithi190 ceph-mon[34368]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:45 smithi066 ceph-mon[35001]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:45 smithi029 ceph-mon[31228]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:45 smithi190 ceph-mon[34368]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:47.845 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:47 smithi066 ceph-mon[35001]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:47.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.884+0000 7fdd10ff9640 1 -- 172.21.15.66:0/647791293 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdd14063800 con 0x7fdd000b6510 2024-09-21T01:44:47.884 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 10 on host 'smithi066' 2024-09-21T01:44:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 msgr2=0x7fdd000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 0x7fdd000b8a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000c00 tx=0x7fdd04031400 comp rx=0 tx=0).stop 2024-09-21T01:44:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 msgr2=0x7fdd1419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00d950 tx=0x7fdcfc00de10 comp rx=0 tx=0).stop 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 shutdown_connections 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdd000b6510 0x7fdd000b8a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd14104070 0x7fdd141a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd141036d0 0x7fdd1419ecb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 --2- 172.21.15.66:0/647791293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd14102640 0x7fdd1406b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 >> 172.21.15.66:0/647791293 conn(0x7fdd140fdfb0 msgr2=0x7fdd141069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 shutdown_connections 2024-09-21T01:44:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-21T01:44:47.887+0000 7fdd19b82640 1 -- 172.21.15.66:0/647791293 wait complete. 2024-09-21T01:44:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:47 smithi029 ceph-mon[31228]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:47 smithi190 ceph-mon[34368]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:48.771 DEBUG:teuthology.orchestra.run.smithi066:osd.10> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T01:44:48.774 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2024-09-21T01:44:48.774 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:44:49.038 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:44:49.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:48 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:48 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:48 smithi066 ceph-mon[35001]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:49.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:48 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:48 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:48 smithi190 ceph-mon[34368]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:49.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:48 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:49.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:48 smithi029 ceph-mon[31228]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:50.957 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 01:44:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T01:44:50.938+0000 7f35700e4640 -1 osd.10 0 log_to_monitors true 2024-09-21T01:44:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:50 smithi190 ceph-mon[34368]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:50 smithi190 ceph-mon[34368]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:50 smithi190 ceph-mon[34368]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:50 smithi066 ceph-mon[35001]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:50 smithi066 ceph-mon[35001]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:50 smithi066 ceph-mon[35001]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:50 smithi029 ceph-mon[31228]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:51.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:50 smithi029 ceph-mon[31228]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:50 smithi029 ceph-mon[31228]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:44:51.624 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-21T01:44:51.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-09-21T01:44:51.894 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:44:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:51 smithi190 ceph-mon[34368]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:44:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:51 smithi190 ceph-mon[34368]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T01:44:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:51 smithi190 ceph-mon[34368]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:51 smithi190 ceph-mon[34368]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.289 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-mon[35001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:44:52.289 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-mon[35001]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T01:44:52.289 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:52.289 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-mon[35001]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.290 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-mon[35001]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.290 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T01:44:51.966+0000 7f356c066640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T01:44:52.290 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 01:44:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T01:44:51.980+0000 7f3567e64640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:44:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:51 smithi029 ceph-mon[31228]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:44:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:51 smithi029 ceph-mon[31228]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T01:44:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:51 smithi029 ceph-mon[31228]: from='osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:51 smithi029 ceph-mon[31228]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]} : dispatch 2024-09-21T01:44:52.353 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 -- 172.21.15.190:0/12652359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 msgr2=0x7fdac4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:52.353 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 --2- 172.21.15.190:0/12652359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac4101f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab402a410 comp rx=0 tx=0).stop 2024-09-21T01:44:52.353 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 -- 172.21.15.190:0/12652359 shutdown_connections 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 --2- 172.21.15.190:0/12652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 0x7fdac410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 --2- 172.21.15.190:0/12652359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 0x7fdac4108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 --2- 172.21.15.190:0/12652359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac4101f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 -- 172.21.15.190:0/12652359 >> 172.21.15.190:0/12652359 conn(0x7fdac40fd770 msgr2=0x7fdac40ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 -- 172.21.15.190:0/12652359 shutdown_connections 2024-09-21T01:44:52.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.353+0000 7fdacb970640 1 -- 172.21.15.190:0/12652359 wait complete. 2024-09-21T01:44:52.355 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 Processor -- start 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 -- start start 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 0x7fdac4102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 0x7fdac4102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410f3b0 con 0x7fdac4109500 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410f230 con 0x7fdac4106ba0 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdacb970640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410f530 con 0x7fdac4101b00 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49424/0 (socket says 172.21.15.190:49424) 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 -- 172.21.15.190:0/2764910074 learned_addr learned my addr 172.21.15.190:0/2764910074 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac8f0b640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 0x7fdac4102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac9f0d640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 0x7fdac4102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 -- 172.21.15.190:0/2764910074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 msgr2=0x7fdac4102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 0x7fdac4102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 -- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 msgr2=0x7fdac4102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 0x7fdac4102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.354+0000 7fdac970c640 1 -- 172.21.15.190:0/2764910074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac41036e0 con 0x7fdac4101b00 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdac970c640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdab4031440 tx=0x7fdab4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab403f070 con 0x7fdac4101b00 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab4031c50 con 0x7fdac4101b00 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac41a4010 con 0x7fdac4101b00 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4035030 con 0x7fdac4101b00 2024-09-21T01:44:52.357 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.355+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41a43b0 con 0x7fdac4101b00 2024-09-21T01:44:52.358 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.356+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac4101f10 con 0x7fdac4101b00 2024-09-21T01:44:52.358 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.357+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdab4042070 con 0x7fdac4101b00 2024-09-21T01:44:52.358 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.358+0000 7fdac27fc640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:44:52.359 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.358+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7fdab4107ae0 con 0x7fdac4101b00 2024-09-21T01:44:52.359 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.358+0000 7fdac8f0b640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:44:52.359 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.359+0000 7fdac8f0b640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 0x7fdaac0b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdab800bbe0 tx=0x7fdab800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:44:52.362 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.362+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdab40d2ec0 con 0x7fdac4101b00 2024-09-21T01:44:52.556 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:44:52.555+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdac4063800 con 0x7fdaac0b6510 2024-09-21T01:44:53.202 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:44:53.202 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:44:53.202 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.202 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi066 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi066 to 4272M 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:53.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi066 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi066 to 4272M 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:53.301 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-21T01:44:53.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:44:53.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi066 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi066 to 4272M 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:44:53.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:44:54.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:53 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:44:54.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:53 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:44:54.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:53 smithi190 ceph-mon[34368]: osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] boot 2024-09-21T01:44:54.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:53 smithi190 ceph-mon[34368]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T01:44:54.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:53 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:53 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:53 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:53 smithi066 ceph-mon[35001]: osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] boot 2024-09-21T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:53 smithi066 ceph-mon[35001]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:53 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:53 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:44:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:53 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:44:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:53 smithi029 ceph-mon[31228]: osd.10 [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] boot 2024-09-21T01:44:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:53 smithi029 ceph-mon[31228]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T01:44:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:53 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/686639708' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"}]': finished 2024-09-21T01:44:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T01:44:55.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:54 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/686639708' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"}]': finished 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T01:44:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:54 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:44:55.400 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T01:44:55.400 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:44:55.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/686639708' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"} : dispatch 2024-09-21T01:44:55.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962f6e91-94b7-4a35-9aec-7011884d4b92"}]': finished 2024-09-21T01:44:55.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T01:44:55.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:54 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:44:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:56 smithi066 ceph-mon[35001]: from='client.? 172.21.15.190:0/1226753987' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:56.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:56 smithi029 ceph-mon[31228]: from='client.? 172.21.15.190:0/1226753987' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:56.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:56 smithi190 ceph-mon[34368]: from='client.? 172.21.15.190:0/1226753987' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:44:57.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:57 smithi066 ceph-mon[35001]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T01:44:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:57 smithi029 ceph-mon[31228]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T01:44:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:57 smithi190 ceph-mon[34368]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T01:44:59.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:44:59 smithi190 ceph-mon[34368]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:44:59.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:44:59 smithi066 ceph-mon[35001]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:44:59.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:44:59 smithi029 ceph-mon[31228]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:45:01.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:01 smithi190 ceph-mon[34368]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:45:01.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:01 smithi066 ceph-mon[35001]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:45:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:01 smithi029 ceph-mon[31228]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:45:03.202 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:03 smithi190 ceph-mon[34368]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:45:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:03 smithi066 ceph-mon[35001]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:45:03.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:03 smithi029 ceph-mon[31228]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:45:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:45:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:04 smithi190 ceph-mon[34368]: Deploying daemon osd.11 on smithi190 2024-09-21T01:45:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:45:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:04.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:04 smithi066 ceph-mon[35001]: Deploying daemon osd.11 on smithi190 2024-09-21T01:45:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:45:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:04.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:04 smithi029 ceph-mon[31228]: Deploying daemon osd.11 on smithi190 2024-09-21T01:45:05.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:05 smithi190 ceph-mon[34368]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:45:05.496 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:05 smithi029 ceph-mon[31228]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:45:05.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:05 smithi066 ceph-mon[35001]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:45:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:07 smithi190 ceph-mon[34368]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:45:07.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:07 smithi066 ceph-mon[35001]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:45:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:07 smithi029 ceph-mon[31228]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:45:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:09 smithi190 ceph-mon[34368]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:09 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:09 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:09 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:09 smithi066 ceph-mon[35001]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:09 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:09 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:09 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:09 smithi029 ceph-mon[31228]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:09 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:09 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:09.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:09 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:11.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:11 smithi190 ceph-mon[34368]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:11 smithi066 ceph-mon[35001]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:11.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:11 smithi029 ceph-mon[31228]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:11.898 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.898+0000 7fdac27fc640 1 -- 172.21.15.190:0/2764910074 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdac4063800 con 0x7fdaac0b6510 2024-09-21T01:45:11.899 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 msgr2=0x7fdaac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 0x7fdaac0b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdab800bbe0 tx=0x7fdab800b390 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 msgr2=0x7fdac41041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdab4031440 tx=0x7fdab4003c10 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 shutdown_connections 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac4109500 0x7fdac4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdac4106ba0 0x7fdac4102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 --2- 172.21.15.190:0/2764910074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac4101b00 0x7fdac41041d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 >> 172.21.15.190:0/2764910074 conn(0x7fdac40fd770 msgr2=0x7fdac4107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 shutdown_connections 2024-09-21T01:45:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:11.900+0000 7fdacb970640 1 -- 172.21.15.190:0/2764910074 wait complete. 2024-09-21T01:45:12.499 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T01:45:12.501 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T01:45:12.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:45:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:13.183 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:12 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.183 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:12 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.183 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:12 smithi029 ceph-mon[31228]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:13.227 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:12 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:12 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:12 smithi190 ceph-mon[34368]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/877186224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 msgr2=0x7fe6c0102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/877186224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0102ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009900 tx=0x7fe6b002a480 comp rx=0 tx=0).stop 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/877186224 shutdown_connections 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/877186224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c010aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/877186224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 0x7fe6c0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/877186224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0102ac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/877186224 >> 172.21.15.29:0/877186224 conn(0x7fe6c00fdfe0 msgr2=0x7fe6c0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/877186224 shutdown_connections 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/877186224 wait complete. 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 Processor -- start 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.276+0000 7fe6c6e0c640 1 -- start start 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 0x7fe6c019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010e290 con 0x7fe6c0102650 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010e110 con 0x7fe6c0104080 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c6e0c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010e410 con 0x7fe6c01036e0 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59650/0 (socket says 172.21.15.29:59650) 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 -- 172.21.15.29:0/3982532840 learned_addr learned my addr 172.21.15.29:0/3982532840 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6bffff640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 0x7fe6c019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c53a9640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 -- 172.21.15.29:0/3982532840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 msgr2=0x7fe6c019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 0x7fe6c019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 -- 172.21.15.29:0/3982532840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 msgr2=0x7fe6c019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 -- 172.21.15.29:0/3982532840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019ba10 con 0x7fe6c0102650 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c53a9640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:45:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.277+0000 7fe6c4ba8640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:13.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.278+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b003f030 con 0x7fe6c0102650 2024-09-21T01:45:13.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.278+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b0003cb0 con 0x7fe6c0102650 2024-09-21T01:45:13.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.278+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019bca0 con 0x7fe6c0102650 2024-09-21T01:45:13.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.278+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0035070 con 0x7fe6c0102650 2024-09-21T01:45:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.278+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a2400 con 0x7fe6c0102650 2024-09-21T01:45:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.279+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c0102ac0 con 0x7fe6c0102650 2024-09-21T01:45:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.281+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe6b0042030 con 0x7fe6c0102650 2024-09-21T01:45:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.281+0000 7fe6bdffb640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 0x7fe6a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.281+0000 7fe6bffff640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 0x7fe6a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.281+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6809+0+0 (secure 0 0 0) 0x7fe6b0107ab0 con 0x7fe6c0102650 2024-09-21T01:45:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.282+0000 7fe6bffff640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 0x7fe6a80b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005fd0 tx=0x7fe6b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:13.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.285+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6b00ce880 con 0x7fe6c0102650 2024-09-21T01:45:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:12 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:12 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:12 smithi066 ceph-mon[35001]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:13.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.472+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe6c01a2790 con 0x7fe6c0102650 2024-09-21T01:45:13.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.472+0000 7fe6bdffb640 1 -- 172.21.15.29:0/3982532840 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7fe6c01a2790 con 0x7fe6c0102650 2024-09-21T01:45:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:13.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 msgr2=0x7fe6a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 0x7fe6a80b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005fd0 tx=0x7fe6b400b390 comp rx=0 tx=0).stop 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 msgr2=0x7fe6c0199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b0004340 comp rx=0 tx=0).stop 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 shutdown_connections 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6a80b6590 0x7fe6a80b8aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c0104080 0x7fe6c019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c01036e0 0x7fe6c019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 --2- 172.21.15.29:0/3982532840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0102650 0x7fe6c0199fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:13.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 >> 172.21.15.29:0/3982532840 conn(0x7fe6c00fdfe0 msgr2=0x7fe6c006bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:13.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 shutdown_connections 2024-09-21T01:45:13.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:13.475+0000 7fe6c6e0c640 1 -- 172.21.15.29:0/3982532840 wait complete. 2024-09-21T01:45:14.294 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:14 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.294 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:14 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.294 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:14 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3982532840' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:14.327 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":65,"num_osds":12,"num_up_osds":11,"osd_up_since":1726883092,"num_in_osds":12,"osd_in_since":1726883094,"num_remapped_pgs":0} 2024-09-21T01:45:14.457 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:14 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.458 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:14 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.458 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:14 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3982532840' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:14 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:14 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:14 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3982532840' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:15.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:45:15.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:15 smithi029 ceph-mon[31228]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:15.354 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:15 smithi190 ceph-mon[34368]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:15.450 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:15 smithi066 ceph-mon[35001]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:15.598 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:15.652 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 01:45:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T01:45:15.353+0000 7f21de954640 -1 osd.11 0 log_to_monitors true 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 -- 172.21.15.29:0/2889338259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc105180 msgr2=0x7fdebc10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 --2- 172.21.15.29:0/2889338259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc105180 0x7fdebc10bb20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009ca0 tx=0x7fdeb402a950 comp rx=0 tx=0).stop 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 -- 172.21.15.29:0/2889338259 shutdown_connections 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 --2- 172.21.15.29:0/2889338259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc105180 0x7fdebc10bb20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 --2- 172.21.15.29:0/2889338259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc1047e0 0x7fdebc104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 --2- 172.21.15.29:0/2889338259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc103750 0x7fdebc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 -- 172.21.15.29:0/2889338259 >> 172.21.15.29:0/2889338259 conn(0x7fdebc0ff0c0 msgr2=0x7fdebc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 -- 172.21.15.29:0/2889338259 shutdown_connections 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 -- 172.21.15.29:0/2889338259 wait complete. 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.120+0000 7fdec192c640 1 Processor -- start 2024-09-21T01:45:16.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 -- start start 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 0x7fdebc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc105180 0x7fdebc19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10f3a0 con 0x7fdebc103750 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10f220 con 0x7fdebc105180 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdec192c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10f520 con 0x7fdebc1047e0 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59670/0 (socket says 172.21.15.29:59670) 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 -- 172.21.15.29:0/1740477140 learned_addr learned my addr 172.21.15.29:0/1740477140 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebb7fe640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc105180 0x7fdebc19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdeba7fc640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 0x7fdebc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 -- 172.21.15.29:0/1740477140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 msgr2=0x7fdebc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:16.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 0x7fdebc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 -- 172.21.15.29:0/1740477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc105180 msgr2=0x7fdebc19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc105180 0x7fdebc19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 -- 172.21.15.29:0/1740477140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc19b9c0 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdeba7fc640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 0x7fdebc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.121+0000 7fdebaffd640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00e9f0 tx=0x7fdeac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.122+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00cd80 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.122+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc19bcb0 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.122+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac0054d0 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.122+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac010690 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.122+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc1a23b0 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.123+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc10a930 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.124+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdeac004e90 con 0x7fdebc103750 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.125+0000 7fdec092a640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 0x7fdea40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:16.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.125+0000 7fdeba7fc640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 0x7fdea40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:16.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.125+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6831+0+0 (secure 0 0 0) 0x7fdeac0e7270 con 0x7fdebc103750 2024-09-21T01:45:16.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.125+0000 7fdeba7fc640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 0x7fdea40b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009c20 tx=0x7fdeb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:16.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.129+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdeac0ae030 con 0x7fdebc103750 2024-09-21T01:45:16.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.317+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdebc063fc0 con 0x7fdebc103750 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.318 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.319 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:16 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.317+0000 7fdec092a640 1 -- 172.21.15.29:0/1740477140 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fdeac0b2410 con 0x7fdebc103750 2024-09-21T01:45:16.320 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:16.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 msgr2=0x7fdea40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:16.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 0x7fdea40b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009c20 tx=0x7fdeb0005f90 comp rx=0 tx=0).stop 2024-09-21T01:45:16.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 msgr2=0x7fdebc199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00e9f0 tx=0x7fdeac00edf0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 shutdown_connections 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdea40b6510 0x7fdea40b8a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdebc105180 0x7fdebc19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdebc1047e0 0x7fdebc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 --2- 172.21.15.29:0/1740477140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdebc103750 0x7fdebc199f90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 >> 172.21.15.29:0/1740477140 conn(0x7fdebc0ff0c0 msgr2=0x7fdebc107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 shutdown_connections 2024-09-21T01:45:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:16.320+0000 7fdec192c640 1 -- 172.21.15.29:0/1740477140 wait complete. 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:16 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:16.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:16.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:16.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:16.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:16 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:17.043 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726883092,"num_in_osds":12,"osd_in_since":1726883094,"num_remapped_pgs":0} 2024-09-21T01:45:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: Detected new or changed devices on smithi190 2024-09-21T01:45:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-21T01:45:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:45:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T01:45:17.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:17.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:17.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1740477140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: Detected new or changed devices on smithi190 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:17.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1740477140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: Detected new or changed devices on smithi190 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: from='osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-09-21T01:45:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:17.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1740477140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:18.044 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:45:18.083 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 01:45:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T01:45:17.977+0000 7f21db0eb640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T01:45:18.315 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:18 smithi029 ceph-mon[31228]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:45:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:18 smithi029 ceph-mon[31228]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T01:45:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:18 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:18 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:18 smithi029 ceph-mon[31228]: from='osd.11 ' entity='osd.11' 2024-09-21T01:45:18.478 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T01:45:18.118+0000 7f21d26cc640 -1 osd.11 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:45:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-mon[34368]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:45:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-mon[34368]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T01:45:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:18 smithi190 ceph-mon[34368]: from='osd.11 ' entity='osd.11' 2024-09-21T01:45:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:18 smithi066 ceph-mon[35001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-21T01:45:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:18 smithi066 ceph-mon[35001]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T01:45:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:18 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:18 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:18 smithi066 ceph-mon[35001]: from='osd.11 ' entity='osd.11' 2024-09-21T01:45:18.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 -- 172.21.15.29:0/3926532038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1581069a0 msgr2=0x7fe158108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:18.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 --2- 172.21.15.29:0/3926532038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1581069a0 0x7fe158108dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe140009900 tx=0x7fe14002a410 comp rx=0 tx=0).stop 2024-09-21T01:45:18.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 -- 172.21.15.29:0/3926532038 shutdown_connections 2024-09-21T01:45:18.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 --2- 172.21.15.29:0/3926532038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581094b0 0x7fe15810b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 --2- 172.21.15.29:0/3926532038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1581069a0 0x7fe158108dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 --2- 172.21.15.29:0/3926532038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe15806c5c0 0x7fe158106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 -- 172.21.15.29:0/3926532038 >> 172.21.15.29:0/3926532038 conn(0x7fe1580fdfe0 msgr2=0x7fe158100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 -- 172.21.15.29:0/3926532038 shutdown_connections 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.785+0000 7fe15d297640 1 -- 172.21.15.29:0/3926532038 wait complete. 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 Processor -- start 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 -- start start 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581069a0 0x7fe15819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1581094b0 0x7fe15819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810f3c0 con 0x7fe15806c5c0 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810f240 con 0x7fe1581094b0 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe15d297640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810f540 con 0x7fe1581069a0 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:18.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59680/0 (socket says 172.21.15.29:59680) 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 -- 172.21.15.29:0/2991172420 learned_addr learned my addr 172.21.15.29:0/2991172420 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe14ffff640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581069a0 0x7fe15819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe1577fe640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1581094b0 0x7fe15819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 -- 172.21.15.29:0/2991172420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581069a0 msgr2=0x7fe15819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581069a0 0x7fe15819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 -- 172.21.15.29:0/2991172420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1581094b0 msgr2=0x7fe15819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.786+0000 7fe156ffd640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1581094b0 0x7fe15819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe156ffd640 1 -- 172.21.15.29:0/2991172420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819ba10 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe156ffd640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e9f0 tx=0x7fe14800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cd80 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819bd00 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1480108e0 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148018980 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.787+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a2400 con 0x7fe15806c5c0 2024-09-21T01:45:18.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.788+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1580ffb00 con 0x7fe15806c5c0 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.790+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe148010430 con 0x7fe15806c5c0 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.790+0000 7fe154ff9640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 0x7fe1440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.791+0000 7fe14ffff640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 0x7fe1440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.791+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6847+0+0 (secure 0 0 0) 0x7fe1480e7630 con 0x7fe15806c5c0 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.791+0000 7fe14ffff640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 0x7fe1440b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe140031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:18.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.794+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1480ae3e0 con 0x7fe15806c5c0 2024-09-21T01:45:18.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.982+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe158063fc0 con 0x7fe15806c5c0 2024-09-21T01:45:18.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.983+0000 7fe154ff9640 1 -- 172.21.15.29:0/2991172420 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7fe1480b27c0 con 0x7fe15806c5c0 2024-09-21T01:45:18.984 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 msgr2=0x7fe1440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 0x7fe1440b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe140031400 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 msgr2=0x7fe158199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e9f0 tx=0x7fe14800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 shutdown_connections 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1440b6510 0x7fe1440b8a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1581094b0 0x7fe15819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1581069a0 0x7fe15819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 --2- 172.21.15.29:0/2991172420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe15806c5c0 0x7fe158199fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 >> 172.21.15.29:0/2991172420 conn(0x7fe1580fdfe0 msgr2=0x7fe158104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 shutdown_connections 2024-09-21T01:45:18.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:18.986+0000 7fe15d297640 1 -- 172.21.15.29:0/2991172420 wait complete. 2024-09-21T01:45:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:19 smithi029 ceph-mon[31228]: purged_snaps scrub starts 2024-09-21T01:45:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:19 smithi029 ceph-mon[31228]: purged_snaps scrub ok 2024-09-21T01:45:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:19 smithi029 ceph-mon[31228]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:19 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:19 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2991172420' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:19 smithi190 ceph-mon[34368]: purged_snaps scrub starts 2024-09-21T01:45:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:19 smithi190 ceph-mon[34368]: purged_snaps scrub ok 2024-09-21T01:45:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:19 smithi190 ceph-mon[34368]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:19 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:19 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2991172420' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:19 smithi066 ceph-mon[35001]: purged_snaps scrub starts 2024-09-21T01:45:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:19 smithi066 ceph-mon[35001]: purged_snaps scrub ok 2024-09-21T01:45:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:19 smithi066 ceph-mon[35001]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:19 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2991172420' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:19.820 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726883092,"num_in_osds":12,"osd_in_since":1726883094,"num_remapped_pgs":0} 2024-09-21T01:45:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:20 smithi029 ceph-mon[31228]: osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] boot 2024-09-21T01:45:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:20 smithi029 ceph-mon[31228]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T01:45:20.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:20 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:20 smithi190 ceph-mon[34368]: osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] boot 2024-09-21T01:45:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:20 smithi190 ceph-mon[34368]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T01:45:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:20 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:20 smithi066 ceph-mon[35001]: osd.11 [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] boot 2024-09-21T01:45:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:20 smithi066 ceph-mon[35001]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T01:45:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:20 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:45:20.822 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:45:21.089 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:21.211 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:21 smithi029 ceph-mon[31228]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:21.212 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:21 smithi029 ceph-mon[31228]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T01:45:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:21 smithi190 ceph-mon[34368]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:21 smithi190 ceph-mon[34368]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T01:45:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:21 smithi066 ceph-mon[35001]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:45:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:21 smithi066 ceph-mon[35001]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 -- 172.21.15.29:0/2785976372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c106ba0 msgr2=0x7f068c108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 --2- 172.21.15.29:0/2785976372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c106ba0 0x7f068c108fc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f067c009900 tx=0x7f067c02a410 comp rx=0 tx=0).stop 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 -- 172.21.15.29:0/2785976372 shutdown_connections 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 --2- 172.21.15.29:0/2785976372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c109500 0x7f068c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 --2- 172.21.15.29:0/2785976372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c106ba0 0x7f068c108fc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 --2- 172.21.15.29:0/2785976372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 -- 172.21.15.29:0/2785976372 >> 172.21.15.29:0/2785976372 conn(0x7f068c0fd770 msgr2=0x7f068c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 -- 172.21.15.29:0/2785976372 shutdown_connections 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.601+0000 7f06937a1640 1 -- 172.21.15.29:0/2785976372 wait complete. 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 Processor -- start 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 -- start start 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 0x7f068c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10f370 con 0x7f068c109500 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10f1f0 con 0x7f068c106ba0 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f06937a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10f4f0 con 0x7f068c101b00 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59696/0 (socket says 172.21.15.29:59696) 2024-09-21T01:45:21.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 -- 172.21.15.29:0/2608138890 learned_addr learned my addr 172.21.15.29:0/2608138890 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0690d3c640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 0x7f068c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f069153d640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 -- 172.21.15.29:0/2608138890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 msgr2=0x7f068c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 -- 172.21.15.29:0/2608138890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 msgr2=0x7f068c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 0x7f068c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0691d3e640 1 -- 172.21.15.29:0/2608138890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c19b9f0 con 0x7f068c109500 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f0690d3c640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 0x7f068c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.602+0000 7f069153d640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f0691d3e640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f068400cad0 tx=0x7f068400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068400ae20 con 0x7f068c109500 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0684005020 con 0x7f068c109500 2024-09-21T01:45:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c19bce0 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068400f6b0 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.603+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a23e0 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.604+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068c0ff290 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.605+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f06840051d0 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.606+0000 7f06827fc640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 0x7f06780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.606+0000 7f069153d640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 0x7f06780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.606+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f06840e72b0 con 0x7f068c109500 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.607+0000 7f069153d640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 0x7f06780b8a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0688005d00 tx=0x7f0688005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:21.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.609+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0684014030 con 0x7f068c109500 2024-09-21T01:45:21.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.798+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f068c063fc0 con 0x7f068c109500 2024-09-21T01:45:21.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.799+0000 7f06827fc640 1 -- 172.21.15.29:0/2608138890 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f06840adfe0 con 0x7f068c109500 2024-09-21T01:45:21.800 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:21.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 msgr2=0x7f06780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:21.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 0x7f06780b8a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0688005d00 tx=0x7f0688005c70 comp rx=0 tx=0).stop 2024-09-21T01:45:21.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 msgr2=0x7f068c19b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f068400cad0 tx=0x7f068400cf90 comp rx=0 tx=0).stop 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 shutdown_connections 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06780b6510 0x7f06780b8a20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f068c109500 0x7f068c19b1d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f068c106ba0 0x7f068c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 --2- 172.21.15.29:0/2608138890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068c101b00 0x7f068c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 >> 172.21.15.29:0/2608138890 conn(0x7f068c0fd770 msgr2=0x7f068c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 shutdown_connections 2024-09-21T01:45:21.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:21.801+0000 7f06937a1640 1 -- 172.21.15.29:0/2608138890 wait complete. 2024-09-21T01:45:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:22 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2608138890' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:22 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2608138890' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:22.555 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:22 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2608138890' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:45:22.586 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726883119,"num_in_osds":12,"osd_in_since":1726883094,"num_remapped_pgs":0} 2024-09-21T01:45:22.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:45:22.842 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:23.237 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:23 smithi029 ceph-mon[31228]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 -- 172.21.15.29:0/3307884431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15681083d0 msgr2=0x7f15681087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 --2- 172.21.15.29:0/3307884431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15681083d0 0x7f15681087e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f155c009900 tx=0x7f155c02a410 comp rx=0 tx=0).stop 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 -- 172.21.15.29:0/3307884431 shutdown_connections 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 --2- 172.21.15.29:0/3307884431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568108d20 0x7f156810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 --2- 172.21.15.29:0/3307884431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15681083d0 0x7f15681087e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 --2- 172.21.15.29:0/3307884431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568102ae0 0x7f1568102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 -- 172.21.15.29:0/3307884431 >> 172.21.15.29:0/3307884431 conn(0x7f1568079760 msgr2=0x7f1568079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 -- 172.21.15.29:0/3307884431 shutdown_connections 2024-09-21T01:45:23.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.328+0000 7f156e32d640 1 -- 172.21.15.29:0/3307884431 wait complete. 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.329+0000 7f156e32d640 1 Processor -- start 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.329+0000 7f156e32d640 1 -- start start 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15681083d0 0x7f156819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 0x7f15681a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1568112990 con 0x7f1568102ae0 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1568112810 con 0x7f15681083d0 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156e32d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1568112b10 con 0x7f1568108d20 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52712/0 (socket says 172.21.15.29:52712) 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 -- 172.21.15.29:0/3631855809 learned_addr learned my addr 172.21.15.29:0/3631855809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f15677fe640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15681083d0 0x7f156819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156c8ca640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 0x7f15681a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:23.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 -- 172.21.15.29:0/3631855809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 msgr2=0x7f15681a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 0x7f15681a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 -- 172.21.15.29:0/3631855809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15681083d0 msgr2=0x7f156819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15681083d0 0x7f156819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 -- 172.21.15.29:0/3631855809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681a2600 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f156c8ca640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 0x7f15681a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f1567fff640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f155800d770 tx=0x7f155800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.330+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558017070 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.331+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15580054d0 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.331+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15681a28f0 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.331+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558004c30 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.331+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a9050 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.332+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568103d10 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.333+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1558004e10 con 0x7f1568102ae0 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.334+0000 7f15657fa640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 0x7f15500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:23.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.334+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f15580e6950 con 0x7f1568102ae0 2024-09-21T01:45:23.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.334+0000 7f15677fe640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 0x7f15500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:23.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.335+0000 7f15677fe640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 0x7f15500b8a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:23.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.338+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15580ad680 con 0x7f1568102ae0 2024-09-21T01:45:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:23 smithi190 ceph-mon[34368]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:23.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.527+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f156819d7c0 con 0x7f1568102ae0 2024-09-21T01:45:23.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.528+0000 7f15657fa640 1 -- 172.21.15.29:0/3631855809 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+18432 (secure 0 0 0) 0x7f15580b1a60 con 0x7f1568102ae0 2024-09-21T01:45:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:23.530 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":69,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:45:20.090936+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:45:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 msgr2=0x7f15500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 0x7f15500b8a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c031400 comp rx=0 tx=0).stop 2024-09-21T01:45:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 msgr2=0x7f156819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f155800d770 tx=0x7f155800dc30 comp rx=0 tx=0).stop 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 shutdown_connections 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15500b6510 0x7f15500b8a20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568108d20 0x7f15681a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15681083d0 0x7f156819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 --2- 172.21.15.29:0/3631855809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1568102ae0 0x7f156819c690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 >> 172.21.15.29:0/3631855809 conn(0x7f1568079760 msgr2=0x7f15681044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.531+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 shutdown_connections 2024-09-21T01:45:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:23.532+0000 7f156e32d640 1 -- 172.21.15.29:0/3631855809 wait complete. 2024-09-21T01:45:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:23 smithi066 ceph-mon[35001]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:24.341 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:24 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3631855809' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:45:24.375 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:41:30.465145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T01:45:24.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T01:45:24.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:24 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3631855809' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:45:24.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:24 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3631855809' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:45:24.638 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/4107949950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0103950 msgr2=0x7f7fd010f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/4107949950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0103950 0x7f7fd010f500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009ca0 tx=0x7f7fc802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/4107949950 shutdown_connections 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/4107949950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0103950 0x7f7fd010f500 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/4107949950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 0x7f7fd0103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/4107949950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0107690 0x7f7fd0107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/4107949950 >> 172.21.15.29:0/4107949950 conn(0x7f7fd0079ea0 msgr2=0x7f7fd007a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/4107949950 shutdown_connections 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.118+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/4107949950 wait complete. 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 Processor -- start 2024-09-21T01:45:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 -- start start 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 0x7f7fd01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0103950 0x7f7fd0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112a40 con 0x7f7fd0107690 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01128c0 con 0x7f7fd0103950 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd7fbc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112bc0 con 0x7f7fd0103000 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52728/0 (socket says 172.21.15.29:52728) 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd5d58640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 0x7f7fd01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 -- 172.21.15.29:0/3239262947 learned_addr learned my addr 172.21.15.29:0/3239262947 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd5557640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0103950 0x7f7fd0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 -- 172.21.15.29:0/3239262947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 msgr2=0x7f7fd01994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 0x7f7fd01994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 -- 172.21.15.29:0/3239262947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0103950 msgr2=0x7f7fd0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0103950 0x7f7fd0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.119+0000 7f7fd6559640 1 -- 172.21.15.29:0/3239262947 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd019f440 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fd6559640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005cc0 tx=0x7f7fc802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003bf0 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd019f6d0 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8003d90 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc802fbc0 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.120+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd0101ef0 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.121+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd0103410 con 0x7f7fd0107690 2024-09-21T01:45:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.123+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f7fc802fd60 con 0x7f7fd0107690 2024-09-21T01:45:25.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.123+0000 7f7fc6ffd640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 0x7f7fbc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:25.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.123+0000 7f7fd5d58640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 0x7f7fbc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:25.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.124+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f7fc8108500 con 0x7f7fd0107690 2024-09-21T01:45:25.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.124+0000 7f7fd5d58640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 0x7f7fbc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d00 tx=0x7f7fcc00d620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.126+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fc8083710 con 0x7f7fd0107690 2024-09-21T01:45:25.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.313+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7fd01029d0 con 0x7f7fd0107690 2024-09-21T01:45:25.315 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:25 smithi029 ceph-mon[31228]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:25.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.314+0000 7f7fc6ffd640 1 -- 172.21.15.29:0/3239262947 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f7fc80cf230 con 0x7f7fd0107690 2024-09-21T01:45:25.316 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.316+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 msgr2=0x7f7fbc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.316+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 0x7f7fbc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d00 tx=0x7f7fcc00d620 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.316+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 msgr2=0x7f7fd019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.316+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005cc0 tx=0x7f7fc802f380 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 shutdown_connections 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7fbc0b6510 0x7f7fbc0b8a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd0107690 0x7f7fd019ed10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fd0103950 0x7f7fd0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 --2- 172.21.15.29:0/3239262947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fd0103000 0x7f7fd01994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 >> 172.21.15.29:0/3239262947 conn(0x7f7fd0079ea0 msgr2=0x7f7fd0105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:25.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 shutdown_connections 2024-09-21T01:45:25.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:25.317+0000 7f7fd7fbc640 1 -- 172.21.15.29:0/3239262947 wait complete. 2024-09-21T01:45:25.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:25 smithi190 ceph-mon[34368]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:25.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:25 smithi066 ceph-mon[35001]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:26.064 INFO:tasks.cephadm:Adding mds.a on smithi029 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.d on smithi029 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.g on smithi029 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.j on smithi029 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.b on smithi066 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.e on smithi066 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.h on smithi066 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.k on smithi066 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.c on smithi190 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.f on smithi190 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.i on smithi190 2024-09-21T01:45:26.065 INFO:tasks.cephadm:Adding mds.l on smithi190 2024-09-21T01:45:26.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2024-09-21T01:45:26.328 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:45:26.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:26 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3239262947' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:45:26.449 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:26 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3239262947' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:45:26.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:26 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3239262947' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 -- 172.21.15.190:0/683963054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 msgr2=0x7fa7a810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 --2- 172.21.15.190:0/683963054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a810aa40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009ca0 tx=0x7fa7a002a950 comp rx=0 tx=0).stop 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 -- 172.21.15.190:0/683963054 shutdown_connections 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 --2- 172.21.15.190:0/683963054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a810aa40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 --2- 172.21.15.190:0/683963054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8103700 0x7fa7a8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 --2- 172.21.15.190:0/683963054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8102670 0x7fa7a8102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 -- 172.21.15.190:0/683963054 >> 172.21.15.190:0/683963054 conn(0x7fa7a80fdfe0 msgr2=0x7fa7a8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.857+0000 7fa7ad212640 1 -- 172.21.15.190:0/683963054 shutdown_connections 2024-09-21T01:45:26.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.858+0000 7fa7ad212640 1 -- 172.21.15.190:0/683963054 wait complete. 2024-09-21T01:45:26.860 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.858+0000 7fa7ad212640 1 Processor -- start 2024-09-21T01:45:26.860 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.858+0000 7fa7ad212640 1 -- start start 2024-09-21T01:45:26.861 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:26.861 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 0x7fa7a8104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:26.861 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a8105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810e2c0 con 0x7fa7a81040a0 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810e140 con 0x7fa7a8103700 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7ad212640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810e440 con 0x7fa7a8102670 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46130/0 (socket says 172.21.15.190:46130) 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 -- 172.21.15.190:0/3216783735 learned_addr learned my addr 172.21.15.190:0/3216783735 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa79ffff640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 0x7fa7a8104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a759e640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a8105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 -- 172.21.15.190:0/3216783735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 msgr2=0x7fa7a8104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 0x7fa7a8104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 -- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 msgr2=0x7fa7a8105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a8105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:26.862 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 -- 172.21.15.190:0/3216783735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a8105750 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa79ffff640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 0x7fa7a8104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a759e640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a8105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a6d9d640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa79800e9f0 tx=0x7fa79800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.859+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79800cd80 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.860+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a81a4060 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.860+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7980108e0 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.860+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798018980 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.860+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a4510 con 0x7fa7a8102670 2024-09-21T01:45:26.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.861+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a8102ae0 con 0x7fa7a8102670 2024-09-21T01:45:26.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.862+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fa798010430 con 0x7fa7a8102670 2024-09-21T01:45:26.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.863+0000 7fa7a4d99640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 0x7fa7940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:26.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.863+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fa7980e77b0 con 0x7fa7a8102670 2024-09-21T01:45:26.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.863+0000 7fa79ffff640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 0x7fa7940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:26.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.864+0000 7fa79ffff640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 0x7fa7940b8a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa790009b50 tx=0x7fa790005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:26.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:26.866+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa7980ae4e0 con 0x7fa7a8102670 2024-09-21T01:45:27.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.055+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) -- 0x7fa7a81000c0 con 0x7fa7940b6510 2024-09-21T01:45:27.065 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.064+0000 7fa7a4d99640 1 -- 172.21.15.190:0/3216783735 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fa7a81000c0 con 0x7fa7940b6510 2024-09-21T01:45:27.065 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2024-09-21T01:45:27.067 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 msgr2=0x7fa7940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 0x7fa7940b8a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa790009b50 tx=0x7fa790005f90 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 msgr2=0x7fa7a8106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa79800e9f0 tx=0x7fa79800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 shutdown_connections 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7940b6510 0x7fa7940b8a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81040a0 0x7fa7a8105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a8103700 0x7fa7a8104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 --2- 172.21.15.190:0/3216783735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102670 0x7fa7a8106240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 >> 172.21.15.190:0/3216783735 conn(0x7fa7a80fdfe0 msgr2=0x7fa7a80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 shutdown_connections 2024-09-21T01:45:27.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-21T01:45:27.067+0000 7fa7ad212640 1 -- 172.21.15.190:0/3216783735 wait complete. 2024-09-21T01:45:27.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:27 smithi190 ceph-mon[34368]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:27.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:27 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:27.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:27 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:27 smithi066 ceph-mon[35001]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:27 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:27 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:27 smithi029 ceph-mon[31228]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:27 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:27 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:45:27.796 DEBUG:teuthology.orchestra.run.smithi029:mds.a> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T01:45:27.799 DEBUG:teuthology.orchestra.run.smithi029:mds.d> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T01:45:27.843 DEBUG:teuthology.orchestra.run.smithi029:mds.g> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T01:45:27.886 DEBUG:teuthology.orchestra.run.smithi029:mds.j> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T01:45:27.930 DEBUG:teuthology.orchestra.run.smithi066:mds.b> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T01:45:27.933 DEBUG:teuthology.orchestra.run.smithi066:mds.e> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T01:45:27.977 DEBUG:teuthology.orchestra.run.smithi066:mds.h> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T01:45:28.020 DEBUG:teuthology.orchestra.run.smithi066:mds.k> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T01:45:28.063 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T01:45:28.065 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T01:45:28.108 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T01:45:28.151 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T01:45:28.194 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T01:45:28.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: Saving service mds.all spec with placement smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:28 smithi066 ceph-mon[35001]: Deploying daemon mds.l on smithi190 2024-09-21T01:45:28.452 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:45:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: Saving service mds.all spec with placement smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-21T01:45:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:28.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:28.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:28.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:28 smithi190 ceph-mon[34368]: Deploying daemon mds.l on smithi190 2024-09-21T01:45:28.573 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:45:28.573 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: Saving service mds.all spec with placement smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:28.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:28 smithi029 ceph-mon[31228]: Deploying daemon mds.l on smithi190 2024-09-21T01:45:28.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.980+0000 7f2c820c1640 1 -- 172.21.15.29:0/1222615889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 msgr2=0x7f2c7c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.980+0000 7f2c820c1640 1 --2- 172.21.15.29:0/1222615889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c1026b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009900 tx=0x7f2c6c02a410 comp rx=0 tx=0).stop 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 -- 172.21.15.29:0/1222615889 shutdown_connections 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 --2- 172.21.15.29:0/1222615889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c108530 0x7f2c7c10ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 --2- 172.21.15.29:0/1222615889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c107be0 0x7f2c7c107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 --2- 172.21.15.29:0/1222615889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c1026b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 -- 172.21.15.29:0/1222615889 >> 172.21.15.29:0/1222615889 conn(0x7f2c7c079ec0 msgr2=0x7f2c7c07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 -- 172.21.15.29:0/1222615889 shutdown_connections 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 -- 172.21.15.29:0/1222615889 wait complete. 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.981+0000 7f2c820c1640 1 Processor -- start 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 -- start start 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:29.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c107be0 0x7f2c7c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c108530 0x7f2c7c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c1119d0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c111850 con 0x7f2c7c108530 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c820c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c111b50 con 0x7f2c7c107be0 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52736/0 (socket says 172.21.15.29:52736) 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 -- 172.21.15.29:0/3873734503 learned_addr learned my addr 172.21.15.29:0/3873734503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7affd640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c107be0 0x7f2c7c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7bfff640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c108530 0x7f2c7c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 -- 172.21.15.29:0/3873734503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c107be0 msgr2=0x7f2c7c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c107be0 0x7f2c7c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 -- 172.21.15.29:0/3873734503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c108530 msgr2=0x7f2c7c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c108530 0x7f2c7c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.982+0000 7f2c7b7fe640 1 -- 172.21.15.29:0/3873734503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c1a0510 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c7b7fe640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005940 tx=0x7f2c6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c03f030 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c6c003cb0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1a07a0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c035070 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.983+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a6f60 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.984+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c103520 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.986+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2c6c042030 con 0x7f2c7c1022a0 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.986+0000 7f2c78ff9640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 0x7f2c640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:29.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.986+0000 7f2c7affd640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 0x7f2c640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:29.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.987+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f2c6c1078a0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.987+0000 7f2c7affd640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 0x7f2c640b8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c06b210 tx=0x7f2c7000b490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:29.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:28.990+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c6c0d29b0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.217+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7c1026b0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.217+0000 7f2c78ff9640 1 -- 172.21.15.29:0/3873734503 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f2c7c1026b0 con 0x7f2c7c1022a0 2024-09-21T01:45:29.219 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:29.219 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":1,"btime":"2024-09-21T01:37:42:108364+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T01:45:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 1 2024-09-21T01:45:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 msgr2=0x7f2c640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 0x7f2c640b8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c06b210 tx=0x7f2c7000b490 comp rx=0 tx=0).stop 2024-09-21T01:45:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 msgr2=0x7f2c7c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005940 tx=0x7f2c6c0042f0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 shutdown_connections 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c640b6510 0x7f2c640b8a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c108530 0x7f2c7c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c107be0 0x7f2c7c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 --2- 172.21.15.29:0/3873734503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7c1022a0 0x7f2c7c19a5a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 >> 172.21.15.29:0/3873734503 conn(0x7f2c7c079ec0 msgr2=0x7f2c7c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 shutdown_connections 2024-09-21T01:45:29.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:29.220+0000 7f2c820c1640 1 -- 172.21.15.29:0/3873734503 wait complete. 2024-09-21T01:45:29.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:29 smithi029 ceph-mon[31228]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:29.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:29 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3873734503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:29.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:29 smithi190 ceph-mon[34368]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:29.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:29 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3873734503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:29 smithi066 ceph-mon[35001]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:29 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3873734503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:30.228 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 01:45:29 smithi190 systemd[1]: Started Ceph mds.l for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6832/3991511259,v1:172.21.15.190:6833/3991511259] up:boot 2024-09-21T01:45:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: fsmap 1 up:standby 2024-09-21T01:45:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:45:31.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:31.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:31.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6832/3991511259,v1:172.21.15.190:6833/3991511259] up:boot 2024-09-21T01:45:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: fsmap 1 up:standby 2024-09-21T01:45:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:45:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6832/3991511259,v1:172.21.15.190:6833/3991511259] up:boot 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: fsmap 1 up:standby 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:31.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:31.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:31.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:31.782 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:31 smithi066 systemd[1]: Starting Ceph mds.k for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:31 smithi066 ceph-mon[35001]: Deploying daemon mds.k on smithi066 2024-09-21T01:45:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:31 smithi066 ceph-mon[35001]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:32.049 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:31 smithi066 podman[62758]: 2024-09-21 01:45:31.820659786 +0000 UTC m=+0.090221485 container create e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T01:45:32.050 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:31 smithi066 podman[62758]: 2024-09-21 01:45:31.74661131 +0000 UTC m=+0.016172951 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:32.050 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 podman[62758]: 2024-09-21 01:45:32.022866718 +0000 UTC m=+0.292428348 container init e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T01:45:32.050 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 podman[62758]: 2024-09-21 01:45:32.025862428 +0000 UTC m=+0.295424059 container start e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T01:45:32.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:32.086 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:31 smithi029 ceph-mon[31228]: Deploying daemon mds.k on smithi066 2024-09-21T01:45:32.086 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:31 smithi029 ceph-mon[31228]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:31 smithi190 ceph-mon[34368]: Deploying daemon mds.k on smithi066 2024-09-21T01:45:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:31 smithi190 ceph-mon[34368]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:32.549 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 ceph-mds[62777]: -- 172.21.15.66:0/99426207 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56196bca3860 con 0x56196fe4b180 2024-09-21T01:45:32.549 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: starting mds.k at 2024-09-21T01:45:32.549 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 bash[62758]: e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 2024-09-21T01:45:32.549 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 01:45:32 smithi066 systemd[1]: Started Ceph mds.k for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:32.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 -- 172.21.15.29:0/1328598369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0109df0 msgr2=0x7fb4d0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:32.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/1328598369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0109df0 0x7fb4d0110620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009ca0 tx=0x7fb4c802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 -- 172.21.15.29:0/1328598369 shutdown_connections 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/1328598369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0109df0 0x7fb4d0110620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/1328598369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 0x7fb4d01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/1328598369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0103b60 0x7fb4d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 -- 172.21.15.29:0/1328598369 >> 172.21.15.29:0/1328598369 conn(0x7fb4d007b850 msgr2=0x7fb4d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 -- 172.21.15.29:0/1328598369 shutdown_connections 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.839+0000 7fb4d81d4640 1 -- 172.21.15.29:0/1328598369 wait complete. 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 Processor -- start 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 -- start start 2024-09-21T01:45:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 0x7fb4d019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 0x7fb4d019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0113b10 con 0x7fb4d0103b60 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0113990 con 0x7fb4d01094a0 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d81d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0113c90 con 0x7fb4d0109df0 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.840+0000 7fb4d5f70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49192/0 (socket says 172.21.15.29:49192) 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 -- 172.21.15.29:0/257671039 learned_addr learned my addr 172.21.15.29:0/257671039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d576f640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 0x7fb4d019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d6771640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 0x7fb4d019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 -- 172.21.15.29:0/257671039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 msgr2=0x7fb4d019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 0x7fb4d019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 -- 172.21.15.29:0/257671039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 msgr2=0x7fb4d019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 0x7fb4d019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 -- 172.21.15.29:0/257671039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d01a03f0 con 0x7fb4d0103b60 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d6771640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 0x7fb4d019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d5f70640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00b4a0 tx=0x7fb4cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc014070 con 0x7fb4d0103b60 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc010ac0 con 0x7fb4d0103b60 2024-09-21T01:45:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d01a06e0 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc017c30 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.841+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d01a6e40 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.842+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d0103f70 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.844+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.845+0000 7fb4c6ffd640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 0x7fb4bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.845+0000 7fb4d576f640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 0x7fb4bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.845+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fb4cc0e64f0 con 0x7fb4d0103b60 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.845+0000 7fb4d576f640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 0x7fb4bc0b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0109190 tx=0x7fb4c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:32.848+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4cc0ad220 con 0x7fb4d0103b60 2024-09-21T01:45:33.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.091+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4d019b5b0 con 0x7fb4d0103b60 2024-09-21T01:45:33.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.092+0000 7fb4c6ffd640 1 -- 172.21.15.29:0/257671039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7fb4cc0b1600 con 0x7fb4d0103b60 2024-09-21T01:45:33.093 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:33.093 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":3,"btime":"2024-09-21T01:45:32:782328+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T01:45:33.093 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 3 2024-09-21T01:45:33.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 msgr2=0x7fb4bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 0x7fb4bc0b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0109190 tx=0x7fb4c000b390 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 msgr2=0x7fb4d019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00b4a0 tx=0x7fb4cc00b960 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 shutdown_connections 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4bc0b6510 0x7fb4bc0b8a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0109df0 0x7fb4d019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4d01094a0 0x7fb4d019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 --2- 172.21.15.29:0/257671039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0103b60 0x7fb4d019a480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 >> 172.21.15.29:0/257671039 conn(0x7fb4d007b850 msgr2=0x7fb4d01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 shutdown_connections 2024-09-21T01:45:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:33.094+0000 7fb4d81d4640 1 -- 172.21.15.29:0/257671039 wait complete. 2024-09-21T01:45:33.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:33.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: Deploying daemon mds.j on smithi029 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6832/3428222627,v1:172.21.15.66:6833/3428222627] up:boot 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: fsmap 2 up:standby 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:45:33.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:33 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/257671039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:33.430 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:33.430 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: Deploying daemon mds.j on smithi029 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6832/3428222627,v1:172.21.15.66:6833/3428222627] up:boot 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: fsmap 2 up:standby 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:45:33.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:33 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/257671039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:33.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:33.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: Deploying daemon mds.j on smithi029 2024-09-21T01:45:33.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6832/3428222627,v1:172.21.15.66:6833/3428222627] up:boot 2024-09-21T01:45:33.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: fsmap 2 up:standby 2024-09-21T01:45:33.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:45:33.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:33 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/257671039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:35.094 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 systemd[1]: Starting Ceph mds.j for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:35.425 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:35 smithi029 ceph-mon[31228]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:35.425 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 podman[68639]: 2024-09-21 01:45:35.158456768 +0000 UTC m=+0.017064977 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:35.425 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 podman[68639]: 2024-09-21 01:45:35.291553861 +0000 UTC m=+0.150162104 container create 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T01:45:35.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:35 smithi190 ceph-mon[34368]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:35.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:35 smithi066 ceph-mon[35001]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:35.844 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 podman[68639]: 2024-09-21 01:45:35.537783458 +0000 UTC m=+0.396391660 container init 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T01:45:35.844 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 podman[68639]: 2024-09-21 01:45:35.540709658 +0000 UTC m=+0.399317858 container start 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T01:45:35.844 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 ceph-mds[68658]: -- 172.21.15.29:0/4075023810 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3a8977860 con 0x55e3acb1fa80 2024-09-21T01:45:35.845 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j[68654]: starting mds.j at 2024-09-21T01:45:35.845 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 bash[68639]: 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 2024-09-21T01:45:35.845 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 01:45:35 smithi029 systemd[1]: Started Ceph mds.j for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: Deploying daemon mds.d on smithi029 2024-09-21T01:45:36.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6834/3427004495,v1:172.21.15.29:6835/3427004495] up:boot 2024-09-21T01:45:36.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: fsmap 3 up:standby 2024-09-21T01:45:36.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:45:37.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:37.036 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.036 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.036 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.036 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: Deploying daemon mds.d on smithi029 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6834/3427004495,v1:172.21.15.29:6835/3427004495] up:boot 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: fsmap 3 up:standby 2024-09-21T01:45:37.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: Deploying daemon mds.d on smithi029 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6834/3427004495,v1:172.21.15.29:6835/3427004495] up:boot 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: fsmap 3 up:standby 2024-09-21T01:45:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:45:37.376 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:37 smithi029 ceph-mon[31228]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:37 smithi190 ceph-mon[34368]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:37 smithi066 ceph-mon[35001]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 -- 172.21.15.29:0/3753259047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 msgr2=0x7efcf810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 --2- 172.21.15.29:0/3753259047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf810f550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009ca0 tx=0x7efcf002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 -- 172.21.15.29:0/3753259047 shutdown_connections 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 --2- 172.21.15.29:0/3753259047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf810f550 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 --2- 172.21.15.29:0/3753259047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf81083d0 0x7efcf81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 --2- 172.21.15.29:0/3753259047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf8102ae0 0x7efcf8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 -- 172.21.15.29:0/3753259047 >> 172.21.15.29:0/3753259047 conn(0x7efcf80797b0 msgr2=0x7efcf8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 -- 172.21.15.29:0/3753259047 shutdown_connections 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.061+0000 7efcfee25640 1 -- 172.21.15.29:0/3753259047 wait complete. 2024-09-21T01:45:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 Processor -- start 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 -- start start 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf8102ae0 0x7efcf819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 0x7efcf819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81129e0 con 0x7efcf8108d20 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8112860 con 0x7efcf8102ae0 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfee25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8112b60 con 0x7efcf81083d0 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfd3c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.062+0000 7efcfd3c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49230/0 (socket says 172.21.15.29:49230) 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 -- 172.21.15.29:0/334531846 learned_addr learned my addr 172.21.15.29:0/334531846 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfcbc1640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf8102ae0 0x7efcf819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcf7fff640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 0x7efcf819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 -- 172.21.15.29:0/334531846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 msgr2=0x7efcf819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 0x7efcf819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 -- 172.21.15.29:0/334531846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf8102ae0 msgr2=0x7efcf819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf8102ae0 0x7efcf819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 -- 172.21.15.29:0/334531846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf81a2630 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcf7fff640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 0x7efcf819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfd3c2640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcf00098d0 tx=0x7efcf00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0003b70 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81a28c0 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0003d10 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf002fa20 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.063+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a9080 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.065+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8102ef0 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.067+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7efcf002fc20 con 0x7efcf8108d20 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.068+0000 7efcf5ffb640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 0x7efce80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.069+0000 7efcfcbc1640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 0x7efce80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.069+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7efcf01082f0 con 0x7efcf8108d20 2024-09-21T01:45:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.069+0000 7efcfcbc1640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 0x7efce80b8a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcec005d00 tx=0x7efcec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.071+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcf00cf020 con 0x7efcf8108d20 2024-09-21T01:45:38.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.327+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf819d7f0 con 0x7efcf8108d20 2024-09-21T01:45:38.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.328+0000 7efcf5ffb640 1 -- 172.21.15.29:0/334531846 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7efcf819d7f0 con 0x7efcf8108d20 2024-09-21T01:45:38.329 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:38.329 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":4,"btime":"2024-09-21T01:45:36:152543+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T01:45:38.330 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 4 2024-09-21T01:45:38.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 msgr2=0x7efce80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 0x7efce80b8a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcec005d00 tx=0x7efcec00b410 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 msgr2=0x7efcf81a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcf00098d0 tx=0x7efcf00317b0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 shutdown_connections 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efce80b6510 0x7efce80b8a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf8108d20 0x7efcf81a1f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf81083d0 0x7efcf819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 --2- 172.21.15.29:0/334531846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf8102ae0 0x7efcf819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 >> 172.21.15.29:0/334531846 conn(0x7efcf80797b0 msgr2=0x7efcf806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:38.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 shutdown_connections 2024-09-21T01:45:38.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:38.330+0000 7efcfee25640 1 -- 172.21.15.29:0/334531846 wait complete. 2024-09-21T01:45:39.032 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:38 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/334531846' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:39.032 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 01:45:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d[69040]: starting mds.d at 2024-09-21T01:45:39.033 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 01:45:39 smithi029 bash[69012]: b60110428a8d51bde122e75bb1e62aff5f4f5a6c39fd7a04a8435c173b304165 2024-09-21T01:45:39.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:38 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/334531846' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:39.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:38 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/334531846' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:39.344 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 01:45:39 smithi029 systemd[1]: Started Ceph mds.d for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:39.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:39.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:39 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:39 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:39 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:40 smithi066 ceph-mon[35001]: Deploying daemon mds.e on smithi066 2024-09-21T01:45:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:40 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6836/333752891,v1:172.21.15.29:6837/333752891] up:boot 2024-09-21T01:45:41.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:40 smithi066 ceph-mon[35001]: fsmap 4 up:standby 2024-09-21T01:45:41.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:40 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:45:41.050 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:40 smithi066 systemd[1]: Starting Ceph mds.e for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:40 smithi029 ceph-mon[31228]: Deploying daemon mds.e on smithi066 2024-09-21T01:45:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:40 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6836/333752891,v1:172.21.15.29:6837/333752891] up:boot 2024-09-21T01:45:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:40 smithi029 ceph-mon[31228]: fsmap 4 up:standby 2024-09-21T01:45:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:40 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:45:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:40 smithi190 ceph-mon[34368]: Deploying daemon mds.e on smithi066 2024-09-21T01:45:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:40 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6836/333752891,v1:172.21.15.29:6837/333752891] up:boot 2024-09-21T01:45:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:40 smithi190 ceph-mon[34368]: fsmap 4 up:standby 2024-09-21T01:45:41.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:40 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:45:41.354 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 podman[63037]: 2024-09-21 01:45:41.073227576 +0000 UTC m=+0.091147399 container create 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:45:41.354 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 podman[63037]: 2024-09-21 01:45:40.998362835 +0000 UTC m=+0.016282670 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:41.354 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 podman[63037]: 2024-09-21 01:45:41.31744092 +0000 UTC m=+0.335360741 container init 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T01:45:41.354 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 podman[63037]: 2024-09-21 01:45:41.32035025 +0000 UTC m=+0.338270078 container start 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T01:45:41.354 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mds[63056]: -- 172.21.15.66:0/1158587138 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55752e705860 con 0x5575328ada80 2024-09-21T01:45:41.374 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:41.653 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:41.748 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: starting mds.e at 2024-09-21T01:45:41.749 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 bash[63037]: 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 2024-09-21T01:45:41.749 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 01:45:41 smithi066 systemd[1]: Started Ceph mds.e for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:41 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:42.061 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:41 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:42.064 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:41 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:42.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 -- 172.21.15.29:0/1646816466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698108cf0 msgr2=0x7fa69810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:42.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/1646816466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698108cf0 0x7fa69810f520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa690009ca0 tx=0x7fa69002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 -- 172.21.15.29:0/1646816466 shutdown_connections 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/1646816466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698108cf0 0x7fa69810f520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/1646816466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6981083a0 0x7fa6981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/1646816466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa698102a60 0x7fa698102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 -- 172.21.15.29:0/1646816466 >> 172.21.15.29:0/1646816466 conn(0x7fa698079730 msgr2=0x7fa698079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 -- 172.21.15.29:0/1646816466 shutdown_connections 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.152+0000 7fa69dcc4640 1 -- 172.21.15.29:0/1646816466 wait complete. 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 Processor -- start 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 -- start start 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 0x7fa69819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa698108cf0 0x7fa69819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:42.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa698113ac0 con 0x7fa698102a60 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa698113940 con 0x7fa6981083a0 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.153+0000 7fa69dcc4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa698113c40 con 0x7fa698108cf0 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58570/0 (socket says 172.21.15.29:58570) 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 -- 172.21.15.29:0/2311460010 learned_addr learned my addr 172.21.15.29:0/2311460010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa696ffd640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 0x7fa69819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa697fff640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa698108cf0 0x7fa69819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 -- 172.21.15.29:0/2311460010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa698108cf0 msgr2=0x7fa69819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa698108cf0 0x7fa69819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 -- 172.21.15.29:0/2311460010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 msgr2=0x7fa69819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 0x7fa69819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 -- 172.21.15.29:0/2311460010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6981a0440 con 0x7fa698102a60 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa696ffd640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 0x7fa69819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:45:42.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa6977fe640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d950 tx=0x7fa68800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688017070 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6981a0730 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa68800de80 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68800f620 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.154+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a6e90 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.155+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa698102e70 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.157+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fa688009c30 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.158+0000 7fa694ff9640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 0x7fa6800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.158+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fa6880e6970 con 0x7fa698102a60 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.158+0000 7fa696ffd640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 0x7fa6800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.158+0000 7fa696ffd640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 0x7fa6800b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa698079d30 tx=0x7fa68c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:42.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.161+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6880ad6a0 con 0x7fa698102a60 2024-09-21T01:45:42.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.391+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa69819b600 con 0x7fa698102a60 2024-09-21T01:45:42.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.392+0000 7fa694ff9640 1 -- 172.21.15.29:0/2311460010 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fa6880b1a80 con 0x7fa698102a60 2024-09-21T01:45:42.393 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:42.393 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"btime":"2024-09-21T01:45:41:743189+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T01:45:42.394 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 msgr2=0x7fa6800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 0x7fa6800b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa698079d30 tx=0x7fa68c009110 comp rx=0 tx=0).stop 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 msgr2=0x7fa69819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d950 tx=0x7fa68800dd50 comp rx=0 tx=0).stop 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 shutdown_connections 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6800b6510 0x7fa6800b8a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa698108cf0 0x7fa69819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6981083a0 0x7fa69819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 --2- 172.21.15.29:0/2311460010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa698102a60 0x7fa69819a4d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:42.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 >> 172.21.15.29:0/2311460010 conn(0x7fa698079730 msgr2=0x7fa6981052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:42.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 shutdown_connections 2024-09-21T01:45:42.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:42.394+0000 7fa69dcc4640 1 -- 172.21.15.29:0/2311460010 wait complete. 2024-09-21T01:45:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:42 smithi190 ceph-mon[34368]: Deploying daemon mds.f on smithi190 2024-09-21T01:45:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:42 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:boot 2024-09-21T01:45:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:42 smithi190 ceph-mon[34368]: fsmap 5 up:standby 2024-09-21T01:45:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:42 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:45:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:42 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2311460010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:42 smithi066 ceph-mon[35001]: Deploying daemon mds.f on smithi190 2024-09-21T01:45:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:42 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:boot 2024-09-21T01:45:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:42 smithi066 ceph-mon[35001]: fsmap 5 up:standby 2024-09-21T01:45:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:42 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:45:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:42 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2311460010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:43.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:42 smithi029 ceph-mon[31228]: Deploying daemon mds.f on smithi190 2024-09-21T01:45:43.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:42 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:boot 2024-09-21T01:45:43.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:42 smithi029 ceph-mon[31228]: fsmap 5 up:standby 2024-09-21T01:45:43.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:42 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:45:43.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:42 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2311460010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:43.416 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 systemd[1]: Starting Ceph mds.f for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:43.672 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 podman[61921]: 2024-09-21 01:45:43.318427923 +0000 UTC m=+0.015584858 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:43.672 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 podman[61921]: 2024-09-21 01:45:43.445422178 +0000 UTC m=+0.142579111 container create 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:45:43.673 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 podman[61921]: 2024-09-21 01:45:43.672833788 +0000 UTC m=+0.369990737 container init 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:43 smithi190 ceph-mon[34368]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 podman[61921]: 2024-09-21 01:45:43.675886032 +0000 UTC m=+0.373042965 container start 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 ceph-mds[61940]: -- 172.21.15.190:0/786207973 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cf1de5860 con 0x561cf5f8d180 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: starting mds.f at 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 bash[61921]: 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb 2024-09-21T01:45:43.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 01:45:43 smithi190 systemd[1]: Started Ceph mds.f for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:43 smithi066 ceph-mon[35001]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:43 smithi029 ceph-mon[31228]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:44.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6834/4140562793,v1:172.21.15.190:6835/4140562793] up:boot 2024-09-21T01:45:44.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: fsmap 6 up:standby 2024-09-21T01:45:44.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:45:44.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:44.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:44.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:44.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:44.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:44.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:44.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:44 smithi190 ceph-mon[34368]: Deploying daemon mds.c on smithi190 2024-09-21T01:45:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6834/4140562793,v1:172.21.15.190:6835/4140562793] up:boot 2024-09-21T01:45:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: fsmap 6 up:standby 2024-09-21T01:45:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:45.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:44 smithi066 ceph-mon[35001]: Deploying daemon mds.c on smithi190 2024-09-21T01:45:45.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6834/4140562793,v1:172.21.15.190:6835/4140562793] up:boot 2024-09-21T01:45:45.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: fsmap 6 up:standby 2024-09-21T01:45:45.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:44 smithi029 ceph-mon[31228]: Deploying daemon mds.c on smithi190 2024-09-21T01:45:45.144 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:45.417 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:45.826 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:45 smithi190 systemd[1]: Starting Ceph mds.c for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 -- 172.21.15.29:0/1347230670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c1083a0 msgr2=0x7fc32c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 --2- 172.21.15.29:0/1347230670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c1083a0 0x7fc32c1087b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009900 tx=0x7fc31c02a410 comp rx=0 tx=0).stop 2024-09-21T01:45:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 -- 172.21.15.29:0/1347230670 shutdown_connections 2024-09-21T01:45:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 --2- 172.21.15.29:0/1347230670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c108cf0 0x7fc32c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 --2- 172.21.15.29:0/1347230670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c1083a0 0x7fc32c1087b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 --2- 172.21.15.29:0/1347230670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c102a60 0x7fc32c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 -- 172.21.15.29:0/1347230670 >> 172.21.15.29:0/1347230670 conn(0x7fc32c079730 msgr2=0x7fc32c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 -- 172.21.15.29:0/1347230670 shutdown_connections 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.904+0000 7fc33389f640 1 -- 172.21.15.29:0/1347230670 wait complete. 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 Processor -- start 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 -- start start 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c102a60 0x7fc32c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c1083a0 0x7fc32c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:45.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c112960 con 0x7fc32c108cf0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1127e0 con 0x7fc32c102a60 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33389f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c112ae0 con 0x7fc32c1083a0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc331e3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc331e3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58590/0 (socket says 172.21.15.29:58590) 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc33163b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c102a60 0x7fc32c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc331e3c640 1 -- 172.21.15.29:0/1976387758 learned_addr learned my addr 172.21.15.29:0/1976387758 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.905+0000 7fc330e3a640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c1083a0 0x7fc32c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 -- 172.21.15.29:0/1976387758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c1083a0 msgr2=0x7fc32c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c1083a0 0x7fc32c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 -- 172.21.15.29:0/1976387758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c102a60 msgr2=0x7fc32c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c102a60 0x7fc32c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 -- 172.21.15.29:0/1976387758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c1a8790 con 0x7fc32c108cf0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc331e3c640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc32400eb10 tx=0x7fc32400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:45.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324004b20 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc324004cc0 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c1a8a80 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324010670 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.906+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1a8f40 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.907+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c102e70 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.909+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc324005000 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.910+0000 7fc3227fc640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 0x7fc3180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.910+0000 7fc33163b640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 0x7fc3180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.910+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fc3240e7530 con 0x7fc32c108cf0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.910+0000 7fc33163b640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 0x7fc3180b8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc32c079b80 tx=0x7fc328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:45.913+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc324056710 con 0x7fc32c108cf0 2024-09-21T01:45:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:45 smithi029 ceph-mon[31228]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:46.125 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:45 smithi190 ceph-mon[34368]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:46.126 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:45 smithi190 podman[62192]: 2024-09-21 01:45:45.795734881 +0000 UTC m=+0.015800285 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:46.126 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:45 smithi190 podman[62192]: 2024-09-21 01:45:45.898244083 +0000 UTC m=+0.118309480 container create b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T01:45:46.126 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 podman[62192]: 2024-09-21 01:45:46.125868442 +0000 UTC m=+0.345933847 container init b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T01:45:46.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.141+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc32c06b270 con 0x7fc32c108cf0 2024-09-21T01:45:46.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.142+0000 7fc3227fc640 1 -- 172.21.15.29:0/1976387758 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7fc3240ae260 con 0x7fc32c108cf0 2024-09-21T01:45:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:46.143 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":7,"btime":"2024-09-21T01:45:43:754555+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24613,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T01:45:46.143 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 7 2024-09-21T01:45:46.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.144+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 msgr2=0x7fc3180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:46.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.144+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 0x7fc3180b8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc32c079b80 tx=0x7fc328005f90 comp rx=0 tx=0).stop 2024-09-21T01:45:46.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 msgr2=0x7fc32c199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:46.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc32400eb10 tx=0x7fc32400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:45:46.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 shutdown_connections 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3180b6510 0x7fc3180b8a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c108cf0 0x7fc32c199b60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c1083a0 0x7fc32c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 --2- 172.21.15.29:0/1976387758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c102a60 0x7fc32c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 >> 172.21.15.29:0/1976387758 conn(0x7fc32c079730 msgr2=0x7fc32c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 shutdown_connections 2024-09-21T01:45:46.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:46.145+0000 7fc33389f640 1 -- 172.21.15.29:0/1976387758 wait complete. 2024-09-21T01:45:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:45 smithi066 ceph-mon[35001]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:46.478 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 podman[62192]: 2024-09-21 01:45:46.128776351 +0000 UTC m=+0.348841750 container start b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T01:45:46.478 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mds[62211]: -- 172.21.15.190:0/2984434740 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556eec6f3860 con 0x556ef089b180 2024-09-21T01:45:46.479 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: starting mds.c at 2024-09-21T01:45:46.479 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 bash[62192]: b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 2024-09-21T01:45:46.479 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 01:45:46 smithi190 systemd[1]: Started Ceph mds.c for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:46.962 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1976387758' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:46.962 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:46.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:46.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:46.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:46.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:46.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1976387758' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:47.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:47.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1976387758' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:47.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:47.983 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:47 smithi066 ceph-mon[35001]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:47.983 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:47 smithi066 ceph-mon[35001]: Deploying daemon mds.h on smithi066 2024-09-21T01:45:47.983 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:47 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:boot 2024-09-21T01:45:47.983 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:47 smithi066 ceph-mon[35001]: fsmap 7 up:standby 2024-09-21T01:45:47.983 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:45:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:47 smithi029 ceph-mon[31228]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:47 smithi029 ceph-mon[31228]: Deploying daemon mds.h on smithi066 2024-09-21T01:45:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:47 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:boot 2024-09-21T01:45:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:47 smithi029 ceph-mon[31228]: fsmap 7 up:standby 2024-09-21T01:45:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:45:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:47 smithi190 ceph-mon[34368]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:47 smithi190 ceph-mon[34368]: Deploying daemon mds.h on smithi066 2024-09-21T01:45:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:47 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:boot 2024-09-21T01:45:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:47 smithi190 ceph-mon[34368]: fsmap 7 up:standby 2024-09-21T01:45:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:45:48.894 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:49.049 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 01:45:48 smithi066 systemd[1]: Started Ceph mds.h for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:49.176 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 -- 172.21.15.29:0/3307106513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 msgr2=0x7f23901087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 --2- 172.21.15.29:0/3307106513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f23901087b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2384009900 tx=0x7f238402a410 comp rx=0 tx=0).stop 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 -- 172.21.15.29:0/3307106513 shutdown_connections 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 --2- 172.21.15.29:0/3307106513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390108cf0 0x7f239010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 --2- 172.21.15.29:0/3307106513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f23901087b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 --2- 172.21.15.29:0/3307106513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390102a60 0x7f2390102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 -- 172.21.15.29:0/3307106513 >> 172.21.15.29:0/3307106513 conn(0x7f2390079730 msgr2=0x7f2390079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:49.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 -- 172.21.15.29:0/3307106513 shutdown_connections 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.681+0000 7f2395df3640 1 -- 172.21.15.29:0/3307106513 wait complete. 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 Processor -- start 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 -- start start 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390102a60 0x7f239019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390108cf0 0x7f239019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2390112960 con 0x7f23901083a0 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23901127e0 con 0x7f2390108cf0 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f2395df3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2390112ae0 con 0x7f2390102a60 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.682+0000 7f238effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58608/0 (socket says 172.21.15.29:58608) 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 -- 172.21.15.29:0/2049940666 learned_addr learned my addr 172.21.15.29:0/2049940666 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238ffff640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390108cf0 0x7f239019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238f7fe640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390102a60 0x7f239019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 -- 172.21.15.29:0/2049940666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390102a60 msgr2=0x7f239019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390102a60 0x7f239019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 -- 172.21.15.29:0/2049940666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390108cf0 msgr2=0x7f239019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390108cf0 0x7f239019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 -- 172.21.15.29:0/2049940666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901a6590 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238effd640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2384032fd0 tx=0x7f2384004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238403f030 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23901a6820 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2384004490 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384035070 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.683+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a6d00 con 0x7f23901083a0 2024-09-21T01:45:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.684+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390102e70 con 0x7f23901083a0 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.687+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2384042030 con 0x7f23901083a0 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.687+0000 7f238cff9640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 0x7f23780b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.688+0000 7f238f7fe640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 0x7f23780b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.688+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f2384107700 con 0x7f23901083a0 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.688+0000 7f238f7fe640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 0x7f23780b8a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f239006b9c0 tx=0x7f23800075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.691+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23840d2770 con 0x7f23901083a0 2024-09-21T01:45:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:49.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:49.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:49.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.925+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f239006b270 con 0x7f23901083a0 2024-09-21T01:45:49.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.925+0000 7f238cff9640 1 -- 172.21.15.29:0/2049940666 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7f239006b270 con 0x7f23901083a0 2024-09-21T01:45:49.926 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:49.926 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"btime":"2024-09-21T01:45:49:834668+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24613,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24644,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T01:45:49.926 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 msgr2=0x7f23780b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 0x7f23780b8a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f239006b9c0 tx=0x7f23800075b0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 msgr2=0x7f239019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2384032fd0 tx=0x7f2384004270 comp rx=0 tx=0).stop 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 shutdown_connections 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23780b6540 0x7f23780b8a50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390108cf0 0x7f239019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23901083a0 0x7f239019b8d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 --2- 172.21.15.29:0/2049940666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2390102a60 0x7f239019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:49.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.927+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 >> 172.21.15.29:0/2049940666 conn(0x7f2390079730 msgr2=0x7f2390104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:49.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.928+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 shutdown_connections 2024-09-21T01:45:49.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:49.928+0000 7f2395df3640 1 -- 172.21.15.29:0/2049940666 wait complete. 2024-09-21T01:45:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:49.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:49.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:50.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:50.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:50 smithi190 ceph-mon[34368]: Deploying daemon mds.i on smithi190 2024-09-21T01:45:50.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:50 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:boot 2024-09-21T01:45:50.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:50 smithi190 ceph-mon[34368]: fsmap 8 up:standby 2024-09-21T01:45:50.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:50 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:45:50.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:50 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2049940666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:51.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:50 smithi029 ceph-mon[31228]: Deploying daemon mds.i on smithi190 2024-09-21T01:45:51.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:50 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:boot 2024-09-21T01:45:51.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:50 smithi029 ceph-mon[31228]: fsmap 8 up:standby 2024-09-21T01:45:51.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:50 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:45:51.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:50 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2049940666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:50 smithi066 ceph-mon[35001]: Deploying daemon mds.i on smithi190 2024-09-21T01:45:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:50 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:boot 2024-09-21T01:45:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:50 smithi066 ceph-mon[35001]: fsmap 8 up:standby 2024-09-21T01:45:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:50 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:45:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:50 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2049940666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:51.478 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 01:45:51 smithi190 systemd[1]: Started Ceph mds.i for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:52.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:52.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:52.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:52.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:52.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:51 smithi029 ceph-mon[31228]: Deploying daemon mds.g on smithi029 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:51 smithi190 ceph-mon[34368]: Deploying daemon mds.g on smithi029 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:51 smithi066 ceph-mon[35001]: Deploying daemon mds.g on smithi029 2024-09-21T01:45:52.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:53.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:52 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:boot 2024-09-21T01:45:53.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:52 smithi029 ceph-mon[31228]: fsmap 9 up:standby 2024-09-21T01:45:53.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:52 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:45:53.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:52 smithi029 ceph-mon[31228]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:52 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:boot 2024-09-21T01:45:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:52 smithi190 ceph-mon[34368]: fsmap 9 up:standby 2024-09-21T01:45:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:52 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:45:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:52 smithi190 ceph-mon[34368]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:52 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:boot 2024-09-21T01:45:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:52 smithi066 ceph-mon[35001]: fsmap 9 up:standby 2024-09-21T01:45:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:52 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:45:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:52 smithi066 ceph-mon[35001]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:53.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 -- 172.21.15.29:0/4281222792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95201060e0 msgr2=0x7f95201064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:53.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 --2- 172.21.15.29:0/4281222792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95201060e0 0x7f95201064f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9514009900 tx=0x7f951402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 -- 172.21.15.29:0/4281222792 shutdown_connections 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 --2- 172.21.15.29:0/4281222792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9520106a30 0x7f9520110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 --2- 172.21.15.29:0/4281222792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95201060e0 0x7f95201064f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 --2- 172.21.15.29:0/4281222792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520101ef0 0x7f9520102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 -- 172.21.15.29:0/4281222792 >> 172.21.15.29:0/4281222792 conn(0x7f9520077a10 msgr2=0x7f9520077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 -- 172.21.15.29:0/4281222792 shutdown_connections 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 -- 172.21.15.29:0/4281222792 wait complete. 2024-09-21T01:45:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.556+0000 7f951ffff640 1 Processor -- start 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 -- start start 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95201060e0 0x7f952019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520106a30 0x7f952019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9520113900 con 0x7f9520101ef0 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9520113780 con 0x7f9520106a30 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9520113a80 con 0x7f95201060e0 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33092/0 (socket says 172.21.15.29:33092) 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 -- 172.21.15.29:0/2347279398 learned_addr learned my addr 172.21.15.29:0/2347279398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951e7fc640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95201060e0 0x7f952019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:53.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951f7fe640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520106a30 0x7f952019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 -- 172.21.15.29:0/2347279398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95201060e0 msgr2=0x7f952019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95201060e0 0x7f952019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 -- 172.21.15.29:0/2347279398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520106a30 msgr2=0x7f952019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520106a30 0x7f952019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.557+0000 7f951effd640 1 -- 172.21.15.29:0/2347279398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a0260 con 0x7f9520101ef0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f951effd640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f951000d950 tx=0x7f951000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510017070 con 0x7f9520101ef0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9510009c30 con 0x7f9520101ef0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a0550 con 0x7f9520101ef0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000f620 con 0x7f9520101ef0 2024-09-21T01:45:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.558+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a6cb0 con 0x7f9520101ef0 2024-09-21T01:45:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.559+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520102300 con 0x7f9520101ef0 2024-09-21T01:45:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.562+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f9510005030 con 0x7f9520101ef0 2024-09-21T01:45:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.562+0000 7f9507fff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 0x7f95080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.563+0000 7f951e7fc640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 0x7f95080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.563+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f95100e6a30 con 0x7f9520101ef0 2024-09-21T01:45:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.563+0000 7f951e7fc640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 0x7f95080b8a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9514009230 tx=0x7f9514031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.568+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95100ad760 con 0x7f9520101ef0 2024-09-21T01:45:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.813+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f952019b420 con 0x7f9520101ef0 2024-09-21T01:45:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.814+0000 7f9507fff640 1 -- 172.21.15.29:0/2347279398 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7f9510009de0 con 0x7f9520101ef0 2024-09-21T01:45:53.814 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:53.814 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":10,"btime":"2024-09-21T01:45:51:848667+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24613,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24640,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24644,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T01:45:53.815 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 10 2024-09-21T01:45:53.815 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:53 smithi029 podman[69742]: 2024-09-21 01:45:53.672437528 +0000 UTC m=+0.016189225 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 msgr2=0x7f95080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 0x7f95080b8a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9514009230 tx=0x7f9514031400 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 msgr2=0x7f952019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f951000d950 tx=0x7f951000de10 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 shutdown_connections 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f95080b6510 0x7f95080b8a20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9520106a30 0x7f952019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95201060e0 0x7f952019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 --2- 172.21.15.29:0/2347279398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9520101ef0 0x7f952019a2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.816+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 >> 172.21.15.29:0/2347279398 conn(0x7f9520077a10 msgr2=0x7f9520107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:53.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.817+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 shutdown_connections 2024-09-21T01:45:53.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:53.817+0000 7f951ffff640 1 -- 172.21.15.29:0/2347279398 wait complete. 2024-09-21T01:45:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:53 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2347279398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:54.094 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:53 smithi029 podman[69742]: 2024-09-21 01:45:53.900757018 +0000 UTC m=+0.244508744 container create e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T01:45:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:53 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2347279398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:53 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2347279398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 podman[69742]: 2024-09-21 01:45:54.188641079 +0000 UTC m=+0.532392770 container init e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 podman[69742]: 2024-09-21 01:45:54.191788127 +0000 UTC m=+0.535539826 container start e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 ceph-mds[69775]: -- 172.21.15.29:0/848472543 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c8da4d860 con 0x564c91bf5180 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: starting mds.g at 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 bash[69742]: e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf 2024-09-21T01:45:54.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 01:45:54 smithi029 systemd[1]: Started Ceph mds.g for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: Deploying daemon mds.a on smithi029 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6838/3540512645,v1:172.21.15.29:6839/3540512645] up:boot 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: fsmap 10 up:standby 2024-09-21T01:45:55.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:55 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: Deploying daemon mds.a on smithi029 2024-09-21T01:45:55.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6838/3540512645,v1:172.21.15.29:6839/3540512645] up:boot 2024-09-21T01:45:55.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: fsmap 10 up:standby 2024-09-21T01:45:55.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:55 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: Deploying daemon mds.a on smithi029 2024-09-21T01:45:55.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6838/3540512645,v1:172.21.15.29:6839/3540512645] up:boot 2024-09-21T01:45:55.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: fsmap 10 up:standby 2024-09-21T01:45:55.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:55 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:45:57.018 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:45:57.509 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:57 smithi029 ceph-mon[31228]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:57.673 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:45:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:57 smithi190 ceph-mon[34368]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:57.797 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 01:45:57 smithi029 systemd[1]: Started Ceph mds.a for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:45:57.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:57 smithi066 ceph-mon[35001]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:58.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 -- 172.21.15.29:0/1735862615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec1083a0 msgr2=0x7f1cec1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:58.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 --2- 172.21.15.29:0/1735862615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec1083a0 0x7f1cec1087b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009900 tx=0x7f1cdc02a410 comp rx=0 tx=0).stop 2024-09-21T01:45:58.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 -- 172.21.15.29:0/1735862615 shutdown_connections 2024-09-21T01:45:58.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 --2- 172.21.15.29:0/1735862615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec108cf0 0x7f1cec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 --2- 172.21.15.29:0/1735862615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec1083a0 0x7f1cec1087b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 --2- 172.21.15.29:0/1735862615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec102a60 0x7f1cec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 -- 172.21.15.29:0/1735862615 >> 172.21.15.29:0/1735862615 conn(0x7f1cec079730 msgr2=0x7f1cec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 -- 172.21.15.29:0/1735862615 shutdown_connections 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.221+0000 7f1cf420c640 1 -- 172.21.15.29:0/1735862615 wait complete. 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 Processor -- start 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 -- start start 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec1083a0 0x7f1cec19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec108cf0 0x7f1cec19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112960 con 0x7f1cec102a60 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1127e0 con 0x7f1cec108cf0 2024-09-21T01:45:58.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf420c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112ae0 con 0x7f1cec1083a0 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf1fa8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf1fa8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33150/0 (socket says 172.21.15.29:33150) 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.222+0000 7f1cf1fa8640 1 -- 172.21.15.29:0/410691720 learned_addr learned my addr 172.21.15.29:0/410691720 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf27a9640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec108cf0 0x7f1cec19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf17a7640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec1083a0 0x7f1cec19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 -- 172.21.15.29:0/410691720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec1083a0 msgr2=0x7f1cec19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec1083a0 0x7f1cec19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 -- 172.21.15.29:0/410691720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec108cf0 msgr2=0x7f1cec19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec108cf0 0x7f1cec19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 -- 172.21.15.29:0/410691720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1a65e0 con 0x7f1cec102a60 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf1fa8640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800ebc0 tx=0x7f1ce800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8010630 con 0x7f1cec102a60 2024-09-21T01:45:58.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1a68d0 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8010c80 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8018b60 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.223+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a6e30 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.224+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec102e70 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.226+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1ce8018d00 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.227+0000 7f1ce2ffd640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 0x7f1cd80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.227+0000 7f1cf17a7640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 0x7f1cd80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.227+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f1ce80e79d0 con 0x7f1cec102a60 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.227+0000 7f1cf17a7640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 0x7f1cd80b8a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000c00 tx=0x7f1cdc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:45:58.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.230+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ce80ae660 con 0x7f1cec102a60 2024-09-21T01:45:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.460+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cec02e640 con 0x7f1cec102a60 2024-09-21T01:45:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.460+0000 7f1ce2ffd640 1 -- 172.21.15.29:0/410691720 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f1ce80b2a40 con 0x7f1cec102a60 2024-09-21T01:45:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:45:58.463 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":12,"btime":"2024-09-21T01:45:58:343191+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24613,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24640,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24644,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T01:45:58.463 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 12 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 msgr2=0x7f1cd80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 0x7f1cd80b8a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000c00 tx=0x7f1cdc031400 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 msgr2=0x7f1cec19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800ebc0 tx=0x7f1ce800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 shutdown_connections 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cd80b6540 0x7f1cd80b8a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec108cf0 0x7f1cec19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec1083a0 0x7f1cec19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 --2- 172.21.15.29:0/410691720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cec102a60 0x7f1cec19b350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 >> 172.21.15.29:0/410691720 conn(0x7f1cec079730 msgr2=0x7f1cec104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 shutdown_connections 2024-09-21T01:45:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:45:58.463+0000 7f1cf420c640 1 -- 172.21.15.29:0/410691720 wait complete. 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:58.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: Deploying daemon mds.b on smithi066 2024-09-21T01:45:58.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6840/2160178023,v1:172.21.15.29:6841/2160178023] up:boot 2024-09-21T01:45:58.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: fsmap 11 up:standby 2024-09-21T01:45:58.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:45:58.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:58 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/410691720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:58.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: Deploying daemon mds.b on smithi066 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6840/2160178023,v1:172.21.15.29:6841/2160178023] up:boot 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: fsmap 11 up:standby 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:45:58.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/410691720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:45:58.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: Deploying daemon mds.b on smithi066 2024-09-21T01:45:58.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6840/2160178023,v1:172.21.15.29:6841/2160178023] up:boot 2024-09-21T01:45:58.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: fsmap 11 up:standby 2024-09-21T01:45:58.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:45:58.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:58 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/410691720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:45:59.958 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:45:59 smithi066 ceph-mon[35001]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:45:59.958 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:45:59 smithi066 systemd[1]: Starting Ceph mds.b for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T01:45:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:45:59 smithi190 ceph-mon[34368]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:45:59 smithi029 ceph-mon[31228]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:00.219 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:45:59 smithi066 podman[63587]: 2024-09-21 01:45:59.860568883 +0000 UTC m=+0.016073129 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:46:00.219 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:45:59 smithi066 podman[63587]: 2024-09-21 01:45:59.963149307 +0000 UTC m=+0.118653552 container create d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T01:46:00.220 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 podman[63587]: 2024-09-21 01:46:00.182163974 +0000 UTC m=+0.337668222 container init d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T01:46:00.220 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 podman[63587]: 2024-09-21 01:46:00.185040431 +0000 UTC m=+0.340544667 container start d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T01:46:00.549 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 ceph-mds[63606]: -- 172.21.15.66:0/789925092 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644be567860 con 0x5644c270fa80 2024-09-21T01:46:00.549 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: starting mds.b at 2024-09-21T01:46:00.549 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 bash[63587]: d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 2024-09-21T01:46:00.549 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:00 smithi066 systemd[1]: Started Ceph mds.b for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T01:46:01.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:46:01.568 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:01.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:boot 2024-09-21T01:46:01.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: fsmap 12 up:standby 2024-09-21T01:46:01.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:01 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:boot 2024-09-21T01:46:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: fsmap 12 up:standby 2024-09-21T01:46:01.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:01 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:boot 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: fsmap 12 up:standby 2024-09-21T01:46:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:01 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 -- 172.21.15.29:0/2505306898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0102fa0 msgr2=0x7fc5a01033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/2505306898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0102fa0 0x7fc5a01033b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc588009900 tx=0x7fc58802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 -- 172.21.15.29:0/2505306898 shutdown_connections 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/2505306898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 0x7fc5a010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/2505306898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0102fa0 0x7fc5a01033b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/2505306898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0107630 0x7fc5a0107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 -- 172.21.15.29:0/2505306898 >> 172.21.15.29:0/2505306898 conn(0x7fc5a0079e40 msgr2=0x7fc5a007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 -- 172.21.15.29:0/2505306898 shutdown_connections 2024-09-21T01:46:02.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.080+0000 7fc5a4d76640 1 -- 172.21.15.29:0/2505306898 wait complete. 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 Processor -- start 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 -- start start 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 0x7fc5a019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 0x7fc5a019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01129e0 con 0x7fc5a0107630 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0112860 con 0x7fc5a01038f0 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc5a4d76640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0112b60 con 0x7fc5a0102fa0 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc59ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc59dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 0x7fc5a019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc59ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44226/0 (socket says 172.21.15.29:44226) 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc59ed9d640 1 -- 172.21.15.29:0/1179120355 learned_addr learned my addr 172.21.15.29:0/1179120355 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.081+0000 7fc59e59c640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 0x7fc5a019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:02.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 -- 172.21.15.29:0/1179120355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 msgr2=0x7fc5a019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 0x7fc5a019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 -- 172.21.15.29:0/1179120355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 msgr2=0x7fc5a019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 0x7fc5a019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 -- 172.21.15.29:0/1179120355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a01a6560 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59e59c640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 0x7fc5a019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc59ed9d640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc59000cad0 tx=0x7fc59000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59000ae20 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc590005020 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a01a6850 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59000f6b0 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.082+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a01a6d60 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.083+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a01033b0 con 0x7fc5a0107630 2024-09-21T01:46:02.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.085+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc5900051d0 con 0x7fc5a0107630 2024-09-21T01:46:02.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.086+0000 7fc5977fe640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 0x7fc58c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:02.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.086+0000 7fc59e59c640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 0x7fc58c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:02.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.086+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fc5900e7280 con 0x7fc5a0107630 2024-09-21T01:46:02.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.087+0000 7fc59e59c640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 0x7fc58c0b8ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc598005fd0 tx=0x7fc598005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:02.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.089+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc590014030 con 0x7fc5a0107630 2024-09-21T01:46:02.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.323+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5a01022a0 con 0x7fc5a0107630 2024-09-21T01:46:02.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.324+0000 7fc5977fe640 1 -- 172.21.15.29:0/1179120355 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7fc5900adf90 con 0x7fc5a0107630 2024-09-21T01:46:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:02.325 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":13,"btime":"2024-09-21T01:46:00:712987+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24596,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24613,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24640,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24644,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T01:46:02.325 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 13 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 msgr2=0x7fc58c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 0x7fc58c0b8ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc598005fd0 tx=0x7fc598005850 comp rx=0 tx=0).stop 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 msgr2=0x7fc5a019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc59000cad0 tx=0x7fc59000cf90 comp rx=0 tx=0).stop 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 shutdown_connections 2024-09-21T01:46:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc58c0b65c0 0x7fc58c0b8ad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5a0107630 0x7fc5a019bd50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a01038f0 0x7fc5a019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 --2- 172.21.15.29:0/1179120355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a0102fa0 0x7fc5a019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.326+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 >> 172.21.15.29:0/1179120355 conn(0x7fc5a0079e40 msgr2=0x7fc5a0105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.327+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 shutdown_connections 2024-09-21T01:46:02.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:02.327+0000 7fc5a4d76640 1 -- 172.21.15.29:0/1179120355 wait complete. 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1179120355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:02 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1179120355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:03.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:02 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1179120355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:02 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T01:46:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:03.772 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:03 smithi029 ceph-mon[31228]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:03.773 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.773 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:03 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 -- 172.21.15.29:0/2025734910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 msgr2=0x7fd6f4102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 --2- 172.21.15.29:0/2025734910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f4102a60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009900 tx=0x7fd6f002a480 comp rx=0 tx=0).stop 2024-09-21T01:46:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 -- 172.21.15.29:0/2025734910 shutdown_connections 2024-09-21T01:46:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 --2- 172.21.15.29:0/2025734910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4110200 0x7fd6f4112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 --2- 172.21.15.29:0/2025734910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4102fa0 0x7fd6f410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 --2- 172.21.15.29:0/2025734910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f4102a60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 -- 172.21.15.29:0/2025734910 >> 172.21.15.29:0/2025734910 conn(0x7fd6f4079730 msgr2=0x7fd6f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 -- 172.21.15.29:0/2025734910 shutdown_connections 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.874+0000 7fd6fb973640 1 -- 172.21.15.29:0/2025734910 wait complete. 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 Processor -- start 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 -- start start 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4102fa0 0x7fd6f419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4110200 0x7fd6f41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4115c70 con 0x7fd6f4102650 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4115af0 con 0x7fd6f4110200 2024-09-21T01:46:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6fb973640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4115df0 con 0x7fd6f4102fa0 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6f970f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6f970f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44242/0 (socket says 172.21.15.29:44242) 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.875+0000 7fd6f970f640 1 -- 172.21.15.29:0/3953659115 learned_addr learned my addr 172.21.15.29:0/3953659115 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f9f10640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4110200 0x7fd6f41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f8f0e640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4102fa0 0x7fd6f419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 -- 172.21.15.29:0/3953659115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4102fa0 msgr2=0x7fd6f419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4102fa0 0x7fd6f419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 -- 172.21.15.29:0/3953659115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4110200 msgr2=0x7fd6f41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4110200 0x7fd6f41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 -- 172.21.15.29:0/3953659115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f41a25b0 con 0x7fd6f4102650 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6f970f640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005940 tx=0x7fd6f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f003f030 con 0x7fd6f4102650 2024-09-21T01:46:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6f0003cb0 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0035070 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f41a2840 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.876+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41a9000 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.877+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f4102a60 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.879+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd6f0042030 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.880+0000 7fd6ea7fc640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 0x7fd6e00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.880+0000 7fd6f8f0e640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 0x7fd6e00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.880+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fd6f0107a10 con 0x7fd6f4102650 2024-09-21T01:46:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.880+0000 7fd6f8f0e640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 0x7fd6e00b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4005fd0 tx=0x7fd6e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:03.883+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6f00ce740 con 0x7fd6f4102650 2024-09-21T01:46:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:03 smithi190 ceph-mon[34368]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:03 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:04.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:03 smithi066 ceph-mon[35001]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:03 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:04.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.076+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6f41a9200 con 0x7fd6f4102650 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.076+0000 7fd6ea7fc640 1 -- 172.21.15.29:0/3953659115 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7fd6f00d2b20 con 0x7fd6f4102650 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:46:04.077 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-09-21T01:46:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 msgr2=0x7fd6e00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 0x7fd6e00b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4005fd0 tx=0x7fd6e4005850 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 msgr2=0x7fd6f419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005940 tx=0x7fd6f00042f0 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 shutdown_connections 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd6e00b6510 0x7fd6e00b8a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f4110200 0x7fd6f41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f4102fa0 0x7fd6f419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 --2- 172.21.15.29:0/3953659115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6f4102650 0x7fd6f419c640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.079+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 >> 172.21.15.29:0/3953659115 conn(0x7fd6f4079730 msgr2=0x7fd6f4108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.080+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 shutdown_connections 2024-09-21T01:46:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:04.080+0000 7fd6fb973640 1 -- 172.21.15.29:0/3953659115 wait complete. 2024-09-21T01:46:04.741 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:04.741 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:04.741 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:04.741 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:04 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:04.741 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:04 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3953659115' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:46:04.781 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T01:46:04.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T01:46:04.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:04.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:04.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:04.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:04 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:04.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:04 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3953659115' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:46:05.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:05.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:05.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:05.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:04 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:46:05.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:04 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3953659115' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:46:05.581 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:06.014 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:05 smithi029 ceph-mon[31228]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:06.014 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.014 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.014 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.014 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:05 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 -- 172.21.15.29:0/4089724410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9601083c0 msgr2=0x7fc9601087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 --2- 172.21.15.29:0/4089724410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9601083c0 0x7fc9601087d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc95402a410 comp rx=0 tx=0).stop 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 -- 172.21.15.29:0/4089724410 shutdown_connections 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 --2- 172.21.15.29:0/4089724410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc960108d10 0x7fc96010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 --2- 172.21.15.29:0/4089724410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9601083c0 0x7fc9601087d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 --2- 172.21.15.29:0/4089724410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960102a80 0x7fc960102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 -- 172.21.15.29:0/4089724410 >> 172.21.15.29:0/4089724410 conn(0x7fc960079740 msgr2=0x7fc960079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 -- 172.21.15.29:0/4089724410 shutdown_connections 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.110+0000 7fc96524a640 1 -- 172.21.15.29:0/4089724410 wait complete. 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 Processor -- start 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 -- start start 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9601083c0 0x7fc96019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 0x7fc96019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc960113ae0 con 0x7fc960102a80 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc960113960 con 0x7fc960108d10 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc96524a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc960113c60 con 0x7fc9601083c0 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc95ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc95ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44258/0 (socket says 172.21.15.29:44258) 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.111+0000 7fc95ed9d640 1 -- 172.21.15.29:0/3187984330 learned_addr learned my addr 172.21.15.29:0/3187984330 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:06.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95e59c640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9601083c0 0x7fc96019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95f59e640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 0x7fc96019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 -- 172.21.15.29:0/3187984330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9601083c0 msgr2=0x7fc96019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9601083c0 0x7fc96019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 -- 172.21.15.29:0/3187984330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 msgr2=0x7fc96019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 0x7fc96019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 -- 172.21.15.29:0/3187984330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601a04a0 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95f59e640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 0x7fc96019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc95ed9d640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b4a0 tx=0x7fc95000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950014070 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc950010ac0 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9601a0790 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950017c30 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.112+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601a6ef0 con 0x7fc960102a80 2024-09-21T01:46:06.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.113+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc960103d00 con 0x7fc960102a80 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.116+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc950017070 con 0x7fc960102a80 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.117+0000 7fc947fff640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 0x7fc9480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.117+0000 7fc95e59c640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 0x7fc9480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.117+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fc9500e6460 con 0x7fc960102a80 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.117+0000 7fc95e59c640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 0x7fc9480b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc954000c00 tx=0x7fc954031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:06.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.120+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9500ad0f0 con 0x7fc960102a80 2024-09-21T01:46:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:05 smithi190 ceph-mon[34368]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:05 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:05 smithi066 ceph-mon[35001]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:05 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:06.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.316+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc96019b660 con 0x7fc960102a80 2024-09-21T01:46:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.810+0000 7fc947fff640 1 -- 172.21.15.29:0/3187984330 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7fc9500b14d0 con 0x7fc960102a80 2024-09-21T01:46:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.814+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 msgr2=0x7fc9480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.814+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 0x7fc9480b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc954000c00 tx=0x7fc954031400 comp rx=0 tx=0).stop 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.814+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 msgr2=0x7fc96019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.814+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b4a0 tx=0x7fc95000b960 comp rx=0 tx=0).stop 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.814+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 shutdown_connections 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9480b6510 0x7fc9480b8a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc960108d10 0x7fc96019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9601083c0 0x7fc96019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 --2- 172.21.15.29:0/3187984330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc960102a80 0x7fc96019a530 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 >> 172.21.15.29:0/3187984330 conn(0x7fc960079740 msgr2=0x7fc960105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 shutdown_connections 2024-09-21T01:46:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:06.815+0000 7fc96524a640 1 -- 172.21.15.29:0/3187984330 wait complete. 2024-09-21T01:46:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:46:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:06 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:07.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:06 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:46:07.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:46:07.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:46:07.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:06 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:46:07.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T01:46:07.779 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:08.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:07 smithi029 ceph-mon[31228]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:08.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:07 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:46:08.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:07 smithi029 ceph-mon[31228]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T01:46:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:07 smithi190 ceph-mon[34368]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:07 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:46:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:07 smithi190 ceph-mon[34368]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T01:46:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:07 smithi066 ceph-mon[35001]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:07 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3187984330' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:46:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:07 smithi066 ceph-mon[35001]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T01:46:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 -- 172.21.15.29:0/331759843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c81094f0 msgr2=0x7ff1c8109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 --2- 172.21.15.29:0/331759843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c81094f0 0x7ff1c8109900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ca0 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 -- 172.21.15.29:0/331759843 shutdown_connections 2024-09-21T01:46:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 --2- 172.21.15.29:0/331759843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c8109e40 0x7ff1c8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 --2- 172.21.15.29:0/331759843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c81094f0 0x7ff1c8109900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 --2- 172.21.15.29:0/331759843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8103bb0 0x7ff1c8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 -- 172.21.15.29:0/331759843 >> 172.21.15.29:0/331759843 conn(0x7ff1c80ff600 msgr2=0x7ff1c8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 -- 172.21.15.29:0/331759843 shutdown_connections 2024-09-21T01:46:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.359+0000 7ff1d0545640 1 -- 172.21.15.29:0/331759843 wait complete. 2024-09-21T01:46:08.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 Processor -- start 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 -- start start 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c81094f0 0x7ff1c819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8109e40 0x7ff1c819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8113b20 con 0x7ff1c8103bb0 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81139a0 con 0x7ff1c8109e40 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1d0545640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8113ca0 con 0x7ff1c81094f0 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1ce2e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1ce2e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44278/0 (socket says 172.21.15.29:44278) 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1ce2e1640 1 -- 172.21.15.29:0/3232683194 learned_addr learned my addr 172.21.15.29:0/3232683194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1cdae0640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c81094f0 0x7ff1c819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.360+0000 7ff1ceae2640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8109e40 0x7ff1c819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 -- 172.21.15.29:0/3232683194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c81094f0 msgr2=0x7ff1c819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c81094f0 0x7ff1c819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 -- 172.21.15.29:0/3232683194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8109e40 msgr2=0x7ff1c819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8109e40 0x7ff1c819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 -- 172.21.15.29:0/3232683194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c81a04c0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1ce2e1640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000d950 tx=0x7ff1c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0017070 con 0x7ff1c8103bb0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c81a07b0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c0009c30 con 0x7ff1c8103bb0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000f620 con 0x7ff1c8103bb0 2024-09-21T01:46:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.361+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c80755f0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.362+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c8103fc0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.364+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff1c0010460 con 0x7ff1c8103bb0 2024-09-21T01:46:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.364+0000 7ff1c77fe640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 0x7ff1b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.365+0000 7ff1cdae0640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 0x7ff1b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.365+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7497+0+0 (secure 0 0 0) 0x7ff1c00e6b40 con 0x7ff1c8103bb0 2024-09-21T01:46:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.365+0000 7ff1cdae0640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 0x7ff1b00b8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000f80 tx=0x7ff1bc009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:08.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.368+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1c0009de0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.557+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff1c819b680 con 0x7ff1c8103bb0 2024-09-21T01:46:08.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.823+0000 7ff1c77fe640 1 -- 172.21.15.29:0/3232683194 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7ff1c00b1ae0 con 0x7ff1c8103bb0 2024-09-21T01:46:08.824 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-09-21T01:46:08.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 msgr2=0x7ff1b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:08.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 0x7ff1b00b8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000f80 tx=0x7ff1bc009360 comp rx=0 tx=0).stop 2024-09-21T01:46:08.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 msgr2=0x7ff1c819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000d950 tx=0x7ff1c000de10 comp rx=0 tx=0).stop 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 shutdown_connections 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1b00b6510 0x7ff1b00b8a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1c8109e40 0x7ff1c819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c81094f0 0x7ff1c819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 --2- 172.21.15.29:0/3232683194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c8103bb0 0x7ff1c819a550 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 >> 172.21.15.29:0/3232683194 conn(0x7ff1c80ff600 msgr2=0x7ff1c8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 shutdown_connections 2024-09-21T01:46:08.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:08.827+0000 7ff1d0545640 1 -- 172.21.15.29:0/3232683194 wait complete. 2024-09-21T01:46:09.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:08 smithi029 ceph-mon[31228]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:46:09.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:08 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:46:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:08 smithi190 ceph-mon[34368]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:46:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:08 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:46:09.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:08 smithi066 ceph-mon[35001]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:46:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:08 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:46:09.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T01:46:09.863 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:09 smithi029 ceph-mon[31228]: pgmap v289: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:09 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:46:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:09 smithi029 ceph-mon[31228]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:46:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:09 smithi190 ceph-mon[34368]: pgmap v289: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:09 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:46:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:09 smithi190 ceph-mon[34368]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:46:10.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:09 smithi066 ceph-mon[35001]: pgmap v289: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:09 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3232683194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:46:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:09 smithi066 ceph-mon[35001]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:46:10.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/1724574914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e281094b0 msgr2=0x7f5e281098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:10.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/1724574914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e281094b0 0x7f5e281098c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009900 tx=0x7f5e1802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:10.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/1724574914 shutdown_connections 2024-09-21T01:46:10.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/1724574914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e28110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/1724574914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e281094b0 0x7f5e281098c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/1724574914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e28103bc0 0x7f5e28103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/1724574914 >> 172.21.15.29:0/1724574914 conn(0x7f5e2807b910 msgr2=0x7f5e2807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/1724574914 shutdown_connections 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.377+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/1724574914 wait complete. 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 Processor -- start 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 -- start start 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e281094b0 0x7f5e2819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e2819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28113ac0 con 0x7f5e28103bc0 2024-09-21T01:46:10.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28113940 con 0x7f5e281094b0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.378+0000 7f5e2f7f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28113c40 con 0x7f5e28109e00 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44292/0 (socket says 172.21.15.29:44292) 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 -- 172.21.15.29:0/2643086744 learned_addr learned my addr 172.21.15.29:0/2643086744 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2cd93640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e281094b0 0x7f5e2819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2dd95640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e2819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 -- 172.21.15.29:0/2643086744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 msgr2=0x7f5e2819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e2819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 -- 172.21.15.29:0/2643086744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e281094b0 msgr2=0x7f5e2819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e281094b0 0x7f5e2819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 -- 172.21.15.29:0/2643086744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e281a0440 con 0x7f5e28103bc0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2dd95640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e2819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2d594640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9f0 tx=0x7f5e2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:10.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400cd80 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e281a0730 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e240108e0 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24018980 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.379+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e281a6e90 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.380+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e28103fd0 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.382+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5e24010430 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.383+0000 7f5e1e7fc640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 0x7f5e140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.383+0000 7f5e2cd93640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 0x7f5e140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.383+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7f5e240e7a60 con 0x7f5e28103bc0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.383+0000 7f5e2cd93640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 0x7f5e140b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009230 tx=0x7f5e18031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:10.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.386+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5e240ae400 con 0x7f5e28103bc0 2024-09-21T01:46:10.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.614+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5e2819b600 con 0x7f5e28103bc0 2024-09-21T01:46:10.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.846+0000 7f5e1e7fc640 1 -- 172.21.15.29:0/2643086744 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5e240b27e0 con 0x7f5e28103bc0 2024-09-21T01:46:10.846 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 msgr2=0x7f5e140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 0x7f5e140b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009230 tx=0x7f5e18031400 comp rx=0 tx=0).stop 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 msgr2=0x7f5e2819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9f0 tx=0x7f5e2400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 shutdown_connections 2024-09-21T01:46:10.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.848+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e140b6510 0x7f5e140b8a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e28109e00 0x7f5e2819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e281094b0 0x7f5e2819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 --2- 172.21.15.29:0/2643086744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e28103bc0 0x7f5e2819a4d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 >> 172.21.15.29:0/2643086744 conn(0x7f5e2807b910 msgr2=0x7f5e281075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 shutdown_connections 2024-09-21T01:46:10.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:10.849+0000 7f5e2f7f8640 1 -- 172.21.15.29:0/2643086744 wait complete. 2024-09-21T01:46:11.217 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:10 smithi029 ceph-mon[31228]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:46:11.217 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:10 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:46:11.217 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:10 smithi029 ceph-mon[31228]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:46:11.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:10 smithi190 ceph-mon[34368]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:46:11.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:10 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:46:11.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:10 smithi190 ceph-mon[34368]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:46:11.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:10 smithi066 ceph-mon[35001]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:46:11.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:10 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:46:11.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:10 smithi066 ceph-mon[35001]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:46:11.637 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:46:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:12.022 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:11 smithi029 ceph-mon[31228]: pgmap v292: 97 pgs: 13 creating+activating, 15 creating+peering, 42 unknown, 27 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:12.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:11 smithi029 ceph-mon[31228]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:46:12.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:11 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:46:12.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:11 smithi029 ceph-mon[31228]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:46:12.023 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:11 smithi029 ceph-mon[31228]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:46:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:11 smithi190 ceph-mon[34368]: pgmap v292: 97 pgs: 13 creating+activating, 15 creating+peering, 42 unknown, 27 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:11 smithi190 ceph-mon[34368]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:46:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:11 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:46:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:11 smithi190 ceph-mon[34368]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:46:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:11 smithi190 ceph-mon[34368]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:46:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:11 smithi066 ceph-mon[35001]: pgmap v292: 97 pgs: 13 creating+activating, 15 creating+peering, 42 unknown, 27 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:11 smithi066 ceph-mon[35001]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:46:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:11 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2643086744' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:46:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:11 smithi066 ceph-mon[35001]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:46:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:11 smithi066 ceph-mon[35001]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.395+0000 7f3875e60640 1 -- 172.21.15.29:0/1812875066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38701094d0 msgr2=0x7f38701098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.395+0000 7f3875e60640 1 --2- 172.21.15.29:0/1812875066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38701094d0 0x7f38701098e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3858009900 tx=0x7f385802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 -- 172.21.15.29:0/1812875066 shutdown_connections 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 --2- 172.21.15.29:0/1812875066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 0x7f3870110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 --2- 172.21.15.29:0/1812875066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38701094d0 0x7f38701098e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 --2- 172.21.15.29:0/1812875066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870103b90 0x7f3870103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 -- 172.21.15.29:0/1812875066 >> 172.21.15.29:0/1812875066 conn(0x7f38700ff600 msgr2=0x7f3870101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 -- 172.21.15.29:0/1812875066 shutdown_connections 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 -- 172.21.15.29:0/1812875066 wait complete. 2024-09-21T01:46:12.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 Processor -- start 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.396+0000 7f3875e60640 1 -- start start 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 0x7f3870075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 0x7f3870076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3870113b00 con 0x7f3870103b90 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3870113980 con 0x7f38701094d0 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3870113c80 con 0x7f3870109e20 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51920/0 (socket says 172.21.15.29:51920) 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 -- 172.21.15.29:0/21887798 learned_addr learned my addr 172.21.15.29:0/21887798 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386ffff640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 0x7f3870076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3867fff640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 0x7f3870075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 -- 172.21.15.29:0/21887798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 msgr2=0x7f3870076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:12.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 0x7f3870076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 -- 172.21.15.29:0/21887798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 msgr2=0x7f3870075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 0x7f3870075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 -- 172.21.15.29:0/21887798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38701a87d0 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3867fff640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 0x7f3870075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386f7fe640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f386000c890 tx=0x7f386000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860004220 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38701a8ac0 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38600043c0 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.397+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860004a00 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.398+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38701a9050 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.398+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3870103fa0 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.400+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f38600079b0 con 0x7f3870103b90 2024-09-21T01:46:12.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.401+0000 7f386d7fa640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 0x7f385c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:12.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.402+0000 7f3867fff640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 0x7f385c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:12.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.402+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f38600e7320 con 0x7f3870103b90 2024-09-21T01:46:12.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.402+0000 7f3867fff640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 0x7f385c0b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3858000c00 tx=0x7f3858031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:12.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.405+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3860014030 con 0x7f3870103b90 2024-09-21T01:46:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.594+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3870076a10 con 0x7f3870103b90 2024-09-21T01:46:12.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.596+0000 7f386d7fa640 1 -- 172.21.15.29:0/21887798 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22243 (secure 0 0 0) 0x7f3870076a10 con 0x7f3870103b90 2024-09-21T01:46:12.596 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:12.597 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:10.616113+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 msgr2=0x7f385c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 0x7f385c0b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3858000c00 tx=0x7f3858031400 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 msgr2=0x7f3870075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f386000c890 tx=0x7f386000cd50 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 shutdown_connections 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f385c0b6510 0x7f385c0b8a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3870109e20 0x7f3870076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701094d0 0x7f3870075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 --2- 172.21.15.29:0/21887798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3870103b90 0x7f3870075710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 >> 172.21.15.29:0/21887798 conn(0x7f38700ff600 msgr2=0x7f3870101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 shutdown_connections 2024-09-21T01:46:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:12.598+0000 7f3875e60640 1 -- 172.21.15.29:0/21887798 wait complete. 2024-09-21T01:46:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:12 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:active 2024-09-21T01:46:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:12 smithi190 ceph-mon[34368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:12 smithi190 ceph-mon[34368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:12 smithi190 ceph-mon[34368]: pgmap v294: 97 pgs: 13 creating+activating, 15 creating+peering, 15 unknown, 54 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:13.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:12 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/21887798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:13.284 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:12 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:active 2024-09-21T01:46:13.284 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:12 smithi029 ceph-mon[31228]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.284 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:12 smithi029 ceph-mon[31228]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.284 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:12 smithi029 ceph-mon[31228]: pgmap v294: 97 pgs: 13 creating+activating, 15 creating+peering, 15 unknown, 54 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:13.284 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:12 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/21887798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:12 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] up:active 2024-09-21T01:46:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:12 smithi066 ceph-mon[35001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:12 smithi066 ceph-mon[35001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:12 smithi066 ceph-mon[35001]: pgmap v294: 97 pgs: 13 creating+activating, 15 creating+peering, 15 unknown, 54 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:46:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:12 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/21887798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:13.315 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:46:13.577 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:14.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 -- 172.21.15.29:0/3625249717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c104860 msgr2=0x7f161c104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:14.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 --2- 172.21.15.29:0/3625249717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c104860 0x7f161c104c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f160c009ca0 tx=0x7f160c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 -- 172.21.15.29:0/3625249717 shutdown_connections 2024-09-21T01:46:14.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 --2- 172.21.15.29:0/3625249717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c10de10 0x7f161c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 --2- 172.21.15.29:0/3625249717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 0x7f161c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 --2- 172.21.15.29:0/3625249717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c104860 0x7f161c104c70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 -- 172.21.15.29:0/3625249717 >> 172.21.15.29:0/3625249717 conn(0x7f161c079730 msgr2=0x7f161c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 -- 172.21.15.29:0/3625249717 shutdown_connections 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.067+0000 7f162109a640 1 -- 172.21.15.29:0/3625249717 wait complete. 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 Processor -- start 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 -- start start 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 0x7f161c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 0x7f161c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f161c113a80 con 0x7f161c10de10 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f161c113900 con 0x7f161c104860 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f162109a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f161c113c00 con 0x7f161c105290 2024-09-21T01:46:14.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f161b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f161b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51938/0 (socket says 172.21.15.29:51938) 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f161b59e640 1 -- 172.21.15.29:0/4021729031 learned_addr learned my addr 172.21.15.29:0/4021729031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f161a59c640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 0x7f161c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.068+0000 7f161ad9d640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 0x7f161c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 -- 172.21.15.29:0/4021729031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 msgr2=0x7f161c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 0x7f161c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 -- 172.21.15.29:0/4021729031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 msgr2=0x7f161c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 0x7f161c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 -- 172.21.15.29:0/4021729031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c1a8730 con 0x7f161c10de10 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161ad9d640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 0x7f161c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f161b59e640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614017030 con 0x7f161c10de10 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c1a8a20 con 0x7f161c10de10 2024-09-21T01:46:14.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1614004020 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614003b50 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.069+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c1a8f10 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.070+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c104c70 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.072+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f161401a070 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.073+0000 7f1603fff640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 0x7f16040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.073+0000 7f161ad9d640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 0x7f16040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.073+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f16140e6cc0 con 0x7f161c10de10 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.073+0000 7f161ad9d640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 0x7f16040b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f160c0094e0 tx=0x7f160c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:14.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.077+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1614021070 con 0x7f161c10de10 2024-09-21T01:46:14.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.265+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f161c103a70 con 0x7f161c10de10 2024-09-21T01:46:14.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.266+0000 7f1603fff640 1 -- 172.21.15.29:0/4021729031 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22243 (secure 0 0 0) 0x7f161c103a70 con 0x7f161c10de10 2024-09-21T01:46:14.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:14.267 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:10.616113+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:46:14.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 msgr2=0x7f16040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 0x7f16040b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f160c0094e0 tx=0x7f160c0093f0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 msgr2=0x7f161c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 shutdown_connections 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16040b6510 0x7f16040b8a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c10de10 0x7f161c1031f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f161c105290 0x7f161c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 --2- 172.21.15.29:0/4021729031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c104860 0x7f161c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 >> 172.21.15.29:0/4021729031 conn(0x7f161c079730 msgr2=0x7f161c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 shutdown_connections 2024-09-21T01:46:14.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:14.269+0000 7f162109a640 1 -- 172.21.15.29:0/4021729031 wait complete. 2024-09-21T01:46:15.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T01:46:15.398 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:15 smithi190 ceph-mon[34368]: pgmap v295: 97 pgs: 13 creating+activating, 8 creating+peering, 76 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-21T01:46:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:15 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4021729031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:15.519 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:15 smithi029 ceph-mon[31228]: pgmap v295: 97 pgs: 13 creating+activating, 8 creating+peering, 76 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-21T01:46:15.519 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:15 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4021729031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:15.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:15 smithi066 ceph-mon[35001]: pgmap v295: 97 pgs: 13 creating+activating, 8 creating+peering, 76 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-21T01:46:15.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:15 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4021729031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:15.839 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:15 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T01:46:15.726+0000 7f235bce4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.892+0000 7f62a58ca640 1 -- 172.21.15.29:0/2060407503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0104a30 msgr2=0x7f62a0104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.892+0000 7f62a58ca640 1 --2- 172.21.15.29:0/2060407503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0104a30 0x7f62a0104e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6290009900 tx=0x7f629002a410 comp rx=0 tx=0).stop 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 -- 172.21.15.29:0/2060407503 shutdown_connections 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 --2- 172.21.15.29:0/2060407503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a010de60 0x7f62a0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 --2- 172.21.15.29:0/2060407503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a0105380 0x7f62a010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 --2- 172.21.15.29:0/2060407503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0104a30 0x7f62a0104e40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 -- 172.21.15.29:0/2060407503 >> 172.21.15.29:0/2060407503 conn(0x7f62a0079760 msgr2=0x7f62a0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:15.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 -- 172.21.15.29:0/2060407503 shutdown_connections 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 -- 172.21.15.29:0/2060407503 wait complete. 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 Processor -- start 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.893+0000 7f62a58ca640 1 -- start start 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a0104a30 0x7f62a019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a010de60 0x7f62a01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0113aa0 con 0x7f62a0105380 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0113920 con 0x7f62a010de60 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a58ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0113c20 con 0x7f62a0104a30 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51956/0 (socket says 172.21.15.29:51956) 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 -- 172.21.15.29:0/1768021266 learned_addr learned my addr 172.21.15.29:0/1768021266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:15.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f629f7fe640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a010de60 0x7f62a01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f629effd640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a0104a30 0x7f62a019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 -- 172.21.15.29:0/1768021266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a0104a30 msgr2=0x7f62a019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a0104a30 0x7f62a019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 -- 172.21.15.29:0/1768021266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a010de60 msgr2=0x7f62a01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a010de60 0x7f62a01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 -- 172.21.15.29:0/1768021266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a01a2660 con 0x7f62a0105380 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62967fc640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f628800e9f0 tx=0x7f628800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.894+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800ccc0 con 0x7f62a0105380 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.895+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a01a2950 con 0x7f62a0105380 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.895+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62880108e0 con 0x7f62a0105380 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.895+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288018860 con 0x7f62a0105380 2024-09-21T01:46:15.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.895+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a01a90b0 con 0x7f62a0105380 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.896+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a006c1f0 con 0x7f62a0105380 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.898+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6288010430 con 0x7f62a0105380 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.899+0000 7f62a48c8640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 0x7f628c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.899+0000 7f629effd640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 0x7f628c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.899+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f62880e78b0 con 0x7f62a0105380 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.900+0000 7f629effd640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 0x7f628c0b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6290000c00 tx=0x7f6290031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:15.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:15.902+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62880b2670 con 0x7f62a0105380 2024-09-21T01:46:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.131+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f62a019d820 con 0x7f62a0105380 2024-09-21T01:46:16.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.149+0000 7f62a48c8640 1 -- 172.21.15.29:0/1768021266 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f62880b75b0 con 0x7f62a0105380 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 msgr2=0x7f628c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 0x7f628c0b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6290000c00 tx=0x7f6290031400 comp rx=0 tx=0).stop 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 msgr2=0x7f62a019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f628800e9f0 tx=0x7f628800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 shutdown_connections 2024-09-21T01:46:16.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f628c0b6510 0x7f628c0b8a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62a010de60 0x7f62a01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62a0105380 0x7f62a019cc90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 --2- 172.21.15.29:0/1768021266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a0104a30 0x7f62a019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 >> 172.21.15.29:0/1768021266 conn(0x7f62a0079760 msgr2=0x7f62a0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 shutdown_connections 2024-09-21T01:46:16.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:16.152+0000 7f62a58ca640 1 -- 172.21.15.29:0/1768021266 wait complete. 2024-09-21T01:46:16.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:46:16.530 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:46:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:46:16.892 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T01:46:16.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T01:46:17.156 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:17.437 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:17 smithi029 ceph-mon[31228]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T01:46:17.437 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:46:17.437 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:17 smithi029 ceph-mon[31228]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:17 smithi190 ceph-mon[34368]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T01:46:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:46:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:17 smithi190 ceph-mon[34368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:17 smithi066 ceph-mon[35001]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T01:46:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1768021266' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:46:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:17 smithi066 ceph-mon[35001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 -- 172.21.15.29:0/1842600185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74102a60 msgr2=0x7f6d74102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 --2- 172.21.15.29:0/1842600185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74102a60 0x7f6d74102e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009ca0 tx=0x7f6d6402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 -- 172.21.15.29:0/1842600185 shutdown_connections 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 --2- 172.21.15.29:0/1842600185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74108cf0 0x7f6d7410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 --2- 172.21.15.29:0/1842600185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 0x7f6d741087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 --2- 172.21.15.29:0/1842600185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74102a60 0x7f6d74102e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 -- 172.21.15.29:0/1842600185 >> 172.21.15.29:0/1842600185 conn(0x7f6d74079730 msgr2=0x7f6d74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 -- 172.21.15.29:0/1842600185 shutdown_connections 2024-09-21T01:46:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.659+0000 7f6d79216640 1 -- 172.21.15.29:0/1842600185 wait complete. 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 Processor -- start 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 -- start start 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 0x7f6d7419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 0x7f6d7419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74113a80 con 0x7f6d74108cf0 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74113900 con 0x7f6d74102a60 2024-09-21T01:46:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.660+0000 7f6d79216640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74113c00 con 0x7f6d741083a0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d72d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 0x7f6d7419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51968/0 (socket says 172.21.15.29:51968) 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 -- 172.21.15.29:0/2633015456 learned_addr learned my addr 172.21.15.29:0/2633015456 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7259c640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 0x7f6d7419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 -- 172.21.15.29:0/2633015456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 msgr2=0x7f6d7419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 0x7f6d7419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 -- 172.21.15.29:0/2633015456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 msgr2=0x7f6d7419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 0x7f6d7419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 -- 172.21.15.29:0/2633015456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d741a03c0 con 0x7f6d74108cf0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d72d9d640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 0x7f6d7419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d7359e640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d750 tx=0x7f6d6c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c017070 con 0x7f6d74108cf0 2024-09-21T01:46:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d741a06b0 con 0x7f6d74108cf0 2024-09-21T01:46:17.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6c0054d0 con 0x7f6d74108cf0 2024-09-21T01:46:17.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.661+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c004c60 con 0x7f6d74108cf0 2024-09-21T01:46:17.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.662+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d741a6e10 con 0x7f6d74108cf0 2024-09-21T01:46:17.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.663+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d74102e70 con 0x7f6d74108cf0 2024-09-21T01:46:17.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.664+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6d6c009bd0 con 0x7f6d74108cf0 2024-09-21T01:46:17.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.665+0000 7f6d6bfff640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 0x7f6d600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:17.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.665+0000 7f6d72d9d640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 0x7f6d600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:17.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.665+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f6d6c0e6f60 con 0x7f6d74108cf0 2024-09-21T01:46:17.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.666+0000 7f6d72d9d640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 0x7f6d600b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005e30 tx=0x7f6d64009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:17.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.668+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d6c0b1c80 con 0x7f6d74108cf0 2024-09-21T01:46:17.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:17.912+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6d7402e640 con 0x7f6d74108cf0 2024-09-21T01:46:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.156+0000 7f6d6bfff640 1 -- 172.21.15.29:0/2633015456 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6d6c0d0020 con 0x7f6d74108cf0 2024-09-21T01:46:18.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 msgr2=0x7f6d600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:18.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 0x7f6d600b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005e30 tx=0x7f6d64009360 comp rx=0 tx=0).stop 2024-09-21T01:46:18.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 msgr2=0x7f6d7419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d750 tx=0x7f6d6c00dc10 comp rx=0 tx=0).stop 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 shutdown_connections 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d600b6510 0x7f6d600b8a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74108cf0 0x7f6d7419fc90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d741083a0 0x7f6d7419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 --2- 172.21.15.29:0/2633015456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102a60 0x7f6d7419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 >> 172.21.15.29:0/2633015456 conn(0x7f6d74079730 msgr2=0x7f6d74101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 shutdown_connections 2024-09-21T01:46:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:18.159+0000 7f6d79216640 1 -- 172.21.15.29:0/2633015456 wait complete. 2024-09-21T01:46:18.173 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:18 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:46:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:18 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:46:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:18 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:46:18.899 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T01:46:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:19.405 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T01:46:19.405 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:46:19.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:46:19.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:46:19.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:19.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T01:46:19.406 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:19 smithi029 ceph-mon[31228]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T01:46:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:19 smithi190 ceph-mon[34368]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2633015456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:46:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T01:46:19.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:19 smithi066 ceph-mon[35001]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 -- 172.21.15.29:0/3729010557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d81083a0 msgr2=0x7ff5d81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 --2- 172.21.15.29:0/3729010557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d81083a0 0x7ff5d81087b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009900 tx=0x7ff5cc02a410 comp rx=0 tx=0).stop 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 -- 172.21.15.29:0/3729010557 shutdown_connections 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 --2- 172.21.15.29:0/3729010557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d8108cf0 0x7ff5d810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 --2- 172.21.15.29:0/3729010557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d81083a0 0x7ff5d81087b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 --2- 172.21.15.29:0/3729010557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 -- 172.21.15.29:0/3729010557 >> 172.21.15.29:0/3729010557 conn(0x7ff5d8079730 msgr2=0x7ff5d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 -- 172.21.15.29:0/3729010557 shutdown_connections 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.773+0000 7ff5de557640 1 -- 172.21.15.29:0/3729010557 wait complete. 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 Processor -- start 2024-09-21T01:46:19.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 -- start start 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d81083a0 0x7ff5d819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8112960 con 0x7ff5d8108cf0 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d81127e0 con 0x7ff5d81083a0 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.774+0000 7ff5de557640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8112ae0 con 0x7ff5d8102a60 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51986/0 (socket says 172.21.15.29:51986) 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 -- 172.21.15.29:0/3939799481 learned_addr learned my addr 172.21.15.29:0/3939799481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d7fff640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d77fe640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d81083a0 0x7ff5d819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 -- 172.21.15.29:0/3939799481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 msgr2=0x7ff5d819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:19.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 -- 172.21.15.29:0/3939799481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d81083a0 msgr2=0x7ff5d819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d81083a0 0x7ff5d819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 -- 172.21.15.29:0/3939799481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d81a03f0 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d7fff640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5dcaf4640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000cad0 tx=0x7ff5d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000ae20 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d81a06e0 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d0005020 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000f6b0 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.775+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d81a6e40 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.776+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d8102e70 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.778+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff5d00051d0 con 0x7ff5d8108cf0 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.779+0000 7ff5d57fa640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 0x7ff5c00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.779+0000 7ff5d7fff640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 0x7ff5c00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:19.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.779+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7ff5d00e75d0 con 0x7ff5d8108cf0 2024-09-21T01:46:19.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.779+0000 7ff5d7fff640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 0x7ff5c00b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005d00 tx=0x7ff5c8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:19.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:19.782+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5d0014030 con 0x7ff5d8108cf0 2024-09-21T01:46:20.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.007+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff5d819b5b0 con 0x7ff5d8108cf0 2024-09-21T01:46:20.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.177+0000 7ff5d57fa640 1 -- 172.21.15.29:0/3939799481 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7ff5d819b5b0 con 0x7ff5d8108cf0 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.179+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 msgr2=0x7ff5c00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.179+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 0x7ff5c00b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005d00 tx=0x7ff5c8005c70 comp rx=0 tx=0).stop 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.179+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 msgr2=0x7ff5d819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.179+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000cad0 tx=0x7ff5d000cf90 comp rx=0 tx=0).stop 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 shutdown_connections 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5c00b6510 0x7ff5c00b8a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5d8108cf0 0x7ff5d819fcc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5d81083a0 0x7ff5d819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 --2- 172.21.15.29:0/3939799481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d8102a60 0x7ff5d819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 >> 172.21.15.29:0/3939799481 conn(0x7ff5d8079730 msgr2=0x7ff5d806b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 shutdown_connections 2024-09-21T01:46:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:20.180+0000 7ff5de557640 1 -- 172.21.15.29:0/3939799481 wait complete. 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:active 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T01:46:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:20 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:active 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T01:46:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:20 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] up:active 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T01:46:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:46:21.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T01:46:20.727+0000 7f235bce4640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T01:46:21.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T01:46:20.727+0000 7f235bce4640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T01:46:21.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T01:46:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-21T01:46:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:46:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:active 2024-09-21T01:46:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:46:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T01:46:21.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T01:46:21.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:21 smithi029 ceph-mon[31228]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T01:46:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:active 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T01:46:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:21 smithi190 ceph-mon[34368]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3939799481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] up:active 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T01:46:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:21 smithi066 ceph-mon[35001]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T01:46:21.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 -- 172.21.15.29:0/1524414900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc984102ae0 msgr2=0x7fc984102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:21.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 --2- 172.21.15.29:0/1524414900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc984102ae0 0x7fc984102ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc97402a410 comp rx=0 tx=0).stop 2024-09-21T01:46:21.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 -- 172.21.15.29:0/1524414900 shutdown_connections 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 --2- 172.21.15.29:0/1524414900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984108d20 0x7fc98410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 --2- 172.21.15.29:0/1524414900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9841083d0 0x7fc9841087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 --2- 172.21.15.29:0/1524414900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc984102ae0 0x7fc984102ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 -- 172.21.15.29:0/1524414900 >> 172.21.15.29:0/1524414900 conn(0x7fc984079760 msgr2=0x7fc984079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 -- 172.21.15.29:0/1524414900 shutdown_connections 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.884+0000 7fc989222640 1 -- 172.21.15.29:0/1524414900 wait complete. 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 Processor -- start 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 -- start start 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984102ae0 0x7fc98419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc984108d20 0x7fc98419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113ab0 con 0x7fc9841083d0 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113930 con 0x7fc984102ae0 2024-09-21T01:46:21.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc989222640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113c30 con 0x7fc984108d20 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc98259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc98259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35716/0 (socket says 172.21.15.29:35716) 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc98259c640 1 -- 172.21.15.29:0/4022154114 learned_addr learned my addr 172.21.15.29:0/4022154114 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc98359e640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc984108d20 0x7fc98419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.885+0000 7fc982d9d640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984102ae0 0x7fc98419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 -- 172.21.15.29:0/4022154114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc984108d20 msgr2=0x7fc98419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc984108d20 0x7fc98419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 -- 172.21.15.29:0/4022154114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984102ae0 msgr2=0x7fc98419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984102ae0 0x7fc98419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 -- 172.21.15.29:0/4022154114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a03d0 con 0x7fc9841083d0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc98259c640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d740 tx=0x7fc97800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978017070 con 0x7fc9841083d0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a06c0 con 0x7fc9841083d0 2024-09-21T01:46:21.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97800de50 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978004ce0 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.886+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a6e20 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.887+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984103d10 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.889+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc978004e80 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.889+0000 7fc96bfff640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 0x7fc96c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.889+0000 7fc982d9d640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 0x7fc96c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.890+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7fc9780e6f20 con 0x7fc9841083d0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.890+0000 7fc982d9d640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 0x7fc96c0b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc974031440 tx=0x7fc974005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:21.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:21.893+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9780ad900 con 0x7fc9841083d0 2024-09-21T01:46:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.119+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc98419b590 con 0x7fc9841083d0 2024-09-21T01:46:22.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.196+0000 7fc96bfff640 1 -- 172.21.15.29:0/4022154114 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fc98419b590 con 0x7fc9841083d0 2024-09-21T01:46:22.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 msgr2=0x7fc96c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:22.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 0x7fc96c0b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc974031440 tx=0x7fc974005d50 comp rx=0 tx=0).stop 2024-09-21T01:46:22.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 msgr2=0x7fc98419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d740 tx=0x7fc97800dc00 comp rx=0 tx=0).stop 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 shutdown_connections 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc96c0b6510 0x7fc96c0b8a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc984108d20 0x7fc98419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9841083d0 0x7fc98419aa00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 --2- 172.21.15.29:0/4022154114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984102ae0 0x7fc98419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 >> 172.21.15.29:0/4022154114 conn(0x7fc984079760 msgr2=0x7fc984101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 shutdown_connections 2024-09-21T01:46:22.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:22.199+0000 7fc989222640 1 -- 172.21.15.29:0/4022154114 wait complete. 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:active 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: Cluster is now healthy 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T01:46:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:22 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:active 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: Cluster is now healthy 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T01:46:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:22 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] up:active 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: Cluster is now healthy 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T01:46:22.572 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:22 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:46:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:23 smithi190 ceph-mon[34368]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-09-21T01:46:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:23 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:46:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:23 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:active 2024-09-21T01:46:23.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:23 smithi190 ceph-mon[34368]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:46:23.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:23 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:46:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:23 smithi066 ceph-mon[35001]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-09-21T01:46:23.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:23 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:46:23.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:23 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:active 2024-09-21T01:46:23.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:23 smithi066 ceph-mon[35001]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:46:23.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:23 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:46:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:23 smithi029 ceph-mon[31228]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-09-21T01:46:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:23 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4022154114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:46:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:23 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] up:active 2024-09-21T01:46:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:23 smithi029 ceph-mon[31228]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:46:23.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:23 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:46:25.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:25 smithi190 ceph-mon[34368]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-21T01:46:25.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:25 smithi066 ceph-mon[35001]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-21T01:46:25.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:25 smithi029 ceph-mon[31228]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-21T01:46:26.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 01:46:25 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T01:46:25.727+0000 7f235bce4640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T01:46:27.313 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T01:46:27.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T01:46:27.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:27 smithi190 ceph-mon[34368]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T01:46:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:27 smithi066 ceph-mon[35001]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T01:46:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:27 smithi029 ceph-mon[31228]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 -- 172.21.15.29:0/1455004980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1741083d0 msgr2=0x7fc1741087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/1455004980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1741083d0 0x7fc1741087e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc16802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 -- 172.21.15.29:0/1455004980 shutdown_connections 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/1455004980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/1455004980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1741083d0 0x7fc1741087e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/1455004980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc174102ae0 0x7fc174102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 -- 172.21.15.29:0/1455004980 >> 172.21.15.29:0/1455004980 conn(0x7fc174079760 msgr2=0x7fc174079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 -- 172.21.15.29:0/1455004980 shutdown_connections 2024-09-21T01:46:28.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.055+0000 7fc17a4c9640 1 -- 172.21.15.29:0/1455004980 wait complete. 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.056+0000 7fc17a4c9640 1 Processor -- start 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.056+0000 7fc17a4c9640 1 -- start start 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1741083d0 0x7fc174199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc174112990 con 0x7fc174102ae0 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc174112810 con 0x7fc174108d20 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc17a4c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc174112b10 con 0x7fc1741083d0 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35738/0 (socket says 172.21.15.29:35738) 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 -- 172.21.15.29:0/2522532970 learned_addr learned my addr 172.21.15.29:0/2522532970 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc178a66640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:28.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc1737fe640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1741083d0 0x7fc174199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 -- 172.21.15.29:0/2522532970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1741083d0 msgr2=0x7fc174199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1741083d0 0x7fc174199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 -- 172.21.15.29:0/2522532970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 msgr2=0x7fc17419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 -- 172.21.15.29:0/2522532970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17419f250 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc178a66640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc173fff640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d770 tx=0x7fc16400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.057+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164017070 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.058+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17419f540 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.058+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1640054d0 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.058+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164004c30 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.058+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a5d60 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.058+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc174103d10 con 0x7fc174102ae0 2024-09-21T01:46:28.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.060+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc164004dd0 con 0x7fc174102ae0 2024-09-21T01:46:28.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.061+0000 7fc1717fa640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 0x7fc15c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:28.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.061+0000 7fc1737fe640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 0x7fc15c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:28.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.061+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7fc1640e70e0 con 0x7fc174102ae0 2024-09-21T01:46:28.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.062+0000 7fc1737fe640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 0x7fc15c0b8a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc16802a930 tx=0x7fc168000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:28.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.064+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1640adac0 con 0x7fc174102ae0 2024-09-21T01:46:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.277+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc17410ab50 con 0x7fc15c0b6540 2024-09-21T01:46:28.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.307+0000 7fc1717fa640 1 -- 172.21.15.29:0/2522532970 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc17410ab50 con 0x7fc15c0b6540 2024-09-21T01:46:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 msgr2=0x7fc15c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 0x7fc15c0b8a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc16802a930 tx=0x7fc168000bc0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 msgr2=0x7fc1741993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d770 tx=0x7fc16400dc30 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 shutdown_connections 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc15c0b6540 0x7fc15c0b8a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc174108d20 0x7fc17419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1741083d0 0x7fc174199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 --2- 172.21.15.29:0/2522532970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc174102ae0 0x7fc1741993a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 >> 172.21.15.29:0/2522532970 conn(0x7fc174079760 msgr2=0x7fc174104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 shutdown_connections 2024-09-21T01:46:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:28.309+0000 7fc17a4c9640 1 -- 172.21.15.29:0/2522532970 wait complete. 2024-09-21T01:46:28.594 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:46:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:46:28.282+0000 7f350d32c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:46:28.595 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:46:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:46:28.282+0000 7f350d32c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:46:28.595 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:46:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:46:28.282+0000 7f350d32c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:46:28.595 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:46:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:46:28.283+0000 7f350d32c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:46:28.595 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 01:46:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x[31455]: 2024-09-21T01:46:28.283+0000 7f350d32c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:46:29.126 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T01:46:29.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:29 smithi029 ceph-mon[31228]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T01:46:29.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:29 smithi029 ceph-mon[31228]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:29.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:29 smithi190 ceph-mon[34368]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T01:46:29.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:29 smithi190 ceph-mon[34368]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:29.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:29 smithi066 ceph-mon[35001]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T01:46:29.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:29 smithi066 ceph-mon[35001]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 -- 172.21.15.29:0/3307423598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 msgr2=0x7f94b01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 --2- 172.21.15.29:0/3307423598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b01098b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009900 tx=0x7f94a402a410 comp rx=0 tx=0).stop 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 -- 172.21.15.29:0/3307423598 shutdown_connections 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 --2- 172.21.15.29:0/3307423598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0109df0 0x7f94b0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 --2- 172.21.15.29:0/3307423598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b01098b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 --2- 172.21.15.29:0/3307423598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0103b60 0x7f94b0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 -- 172.21.15.29:0/3307423598 >> 172.21.15.29:0/3307423598 conn(0x7f94b007b850 msgr2=0x7f94b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 -- 172.21.15.29:0/3307423598 shutdown_connections 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.903+0000 7f94b70d7640 1 -- 172.21.15.29:0/3307423598 wait complete. 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 Processor -- start 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 -- start start 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0103b60 0x7f94b019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0109df0 0x7f94b019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113ab0 con 0x7f94b01094a0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113930 con 0x7f94b0103b60 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b70d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113c30 con 0x7f94b0109df0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35772/0 (socket says 172.21.15.29:35772) 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 -- 172.21.15.29:0/2438317559 learned_addr learned my addr 172.21.15.29:0/2438317559 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b5674640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0109df0 0x7f94b019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94b4e73640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0103b60 0x7f94b019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 -- 172.21.15.29:0/2438317559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0109df0 msgr2=0x7f94b019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0109df0 0x7f94b019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 -- 172.21.15.29:0/2438317559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0103b60 msgr2=0x7f94b019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0103b60 0x7f94b019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.904+0000 7f94affff640 1 -- 172.21.15.29:0/2438317559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b01a0410 con 0x7f94b01094a0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94affff640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94a402a930 tx=0x7f94a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a403f030 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a4003aa0 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b01a06a0 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4035070 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.905+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a0bd0 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.906+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0103f70 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.908+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f94a4042030 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.909+0000 7f94adffb640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 0x7f94980b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.909+0000 7f94b4e73640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 0x7f94980b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.909+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f94a4107bd0 con 0x7f94b01094a0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.910+0000 7f94b4e73640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 0x7f94980b8ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f94a000abe0 tx=0x7f94a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:29.912+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94b01a0bd0 con 0x7f94b01094a0 2024-09-21T01:46:30.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.132+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f94b0063800 con 0x7f94980b65c0 2024-09-21T01:46:30.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.163+0000 7f94adffb640 1 -- 172.21.15.29:0/2438317559 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f94b0063800 con 0x7f94980b65c0 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 msgr2=0x7f94980b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 0x7f94980b8ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f94a000abe0 tx=0x7f94a000a2d0 comp rx=0 tx=0).stop 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 msgr2=0x7f94b019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94a402a930 tx=0x7f94a4004270 comp rx=0 tx=0).stop 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 shutdown_connections 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.165+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94980b65c0 0x7f94980b8ad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94b0109df0 0x7f94b019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b01094a0 0x7f94b019ab00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 --2- 172.21.15.29:0/2438317559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b0103b60 0x7f94b019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 >> 172.21.15.29:0/2438317559 conn(0x7f94b007b850 msgr2=0x7f94b01075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 shutdown_connections 2024-09-21T01:46:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:30.166+0000 7f94b70d7640 1 -- 172.21.15.29:0/2438317559 wait complete. 2024-09-21T01:46:31.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T01:46:31.296 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:31.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:31 smithi029 ceph-mon[31228]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-09-21T01:46:31.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:31 smithi029 ceph-mon[31228]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:31.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:31 smithi029 ceph-mon[31228]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T01:46:31.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:31 smithi190 ceph-mon[34368]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-09-21T01:46:31.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:31 smithi190 ceph-mon[34368]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:31.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:31 smithi190 ceph-mon[34368]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T01:46:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:31 smithi066 ceph-mon[35001]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-09-21T01:46:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:31 smithi066 ceph-mon[35001]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:31 smithi066 ceph-mon[35001]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T01:46:31.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 -- 172.21.15.29:0/3653764786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe634109df0 msgr2=0x7fe634110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 --2- 172.21.15.29:0/3653764786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe634109df0 0x7fe634110620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009ca0 tx=0x7fe62c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 -- 172.21.15.29:0/3653764786 shutdown_connections 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 --2- 172.21.15.29:0/3653764786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe634109df0 0x7fe634110620 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 --2- 172.21.15.29:0/3653764786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6341094a0 0x7fe6341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 --2- 172.21.15.29:0/3653764786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 0x7fe634103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 -- 172.21.15.29:0/3653764786 >> 172.21.15.29:0/3653764786 conn(0x7fe63407b850 msgr2=0x7fe63407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 -- 172.21.15.29:0/3653764786 shutdown_connections 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.815+0000 7fe638b90640 1 -- 172.21.15.29:0/3653764786 wait complete. 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 Processor -- start 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 -- start start 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 0x7fe63419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:31.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe634109df0 0x7fe63419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe634113ab0 con 0x7fe6341094a0 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe634113930 con 0x7fe634109df0 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe638b90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe634113c30 con 0x7fe634103b60 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe62bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe62bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51580/0 (socket says 172.21.15.29:51580) 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe62bfff640 1 -- 172.21.15.29:0/704321691 learned_addr learned my addr 172.21.15.29:0/704321691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe63259c640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 0x7fe63419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.816+0000 7fe632d9d640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe634109df0 0x7fe63419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 -- 172.21.15.29:0/704321691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 msgr2=0x7fe63419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:31.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 0x7fe63419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 -- 172.21.15.29:0/704321691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe634109df0 msgr2=0x7fe63419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe634109df0 0x7fe63419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 -- 172.21.15.29:0/704321691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6341a03f0 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62bfff640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d950 tx=0x7fe61c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c017070 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61c009c30 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6341a06e0 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0047a0 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.817+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341a6e40 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.818+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe634103f70 con 0x7fe6341094a0 2024-09-21T01:46:31.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.820+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe61c004020 con 0x7fe6341094a0 2024-09-21T01:46:31.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.821+0000 7fe62b7fe640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 0x7fe6200b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:31.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.821+0000 7fe63259c640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 0x7fe6200b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:31.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.821+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7fe61c0e73e0 con 0x7fe6341094a0 2024-09-21T01:46:31.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.821+0000 7fe63259c640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 0x7fe6200b8ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe624009bd0 tx=0x7fe624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:31.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:31.824+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe61c020020 con 0x7fe6341094a0 2024-09-21T01:46:32.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.043+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe634063800 con 0x7fe6200b65a0 2024-09-21T01:46:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.050+0000 7fe62b7fe640 1 -- 172.21.15.29:0/704321691 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe634063800 con 0x7fe6200b65a0 2024-09-21T01:46:32.052 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/qa/sv_0/5ad50a62-6d08-470d-b9ca-04de47bdab6e 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.052+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 msgr2=0x7fe6200b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.052+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 0x7fe6200b8ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe624009bd0 tx=0x7fe624005f90 comp rx=0 tx=0).stop 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 msgr2=0x7fe63419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d950 tx=0x7fe61c00de10 comp rx=0 tx=0).stop 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 shutdown_connections 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6200b65a0 0x7fe6200b8ab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe634109df0 0x7fe63419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6341094a0 0x7fe63419aa20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 --2- 172.21.15.29:0/704321691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe634103b60 0x7fe63419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 >> 172.21.15.29:0/704321691 conn(0x7fe63407b850 msgr2=0x7fe634105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 shutdown_connections 2024-09-21T01:46:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:32.053+0000 7fe638b90640 1 -- 172.21.15.29:0/704321691 wait complete. 2024-09-21T01:46:32.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T01:46:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T01:46:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T01:46:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T01:46:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T01:46:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]} : dispatch 2024-09-21T01:46:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T01:46:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T01:46:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T01:46:32.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T01:46:33.173 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T01:46:33.480 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T01:46:33.481 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:33 smithi029 ceph-mon[31228]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-21T01:46:33.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T01:46:33.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-09-21T01:46:33.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T01:46:33.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T01:46:33.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:33 smithi066 ceph-mon[35001]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 -- 172.21.15.29:0/2872702200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700102a60 msgr2=0x7f0700102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 --2- 172.21.15.29:0/2872702200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700102a60 0x7f0700102e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009900 tx=0x7f06e802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 -- 172.21.15.29:0/2872702200 shutdown_connections 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 --2- 172.21.15.29:0/2872702200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700108cf0 0x7f070010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 --2- 172.21.15.29:0/2872702200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07001083a0 0x7f07001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 --2- 172.21.15.29:0/2872702200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700102a60 0x7f0700102e70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 -- 172.21.15.29:0/2872702200 >> 172.21.15.29:0/2872702200 conn(0x7f0700079730 msgr2=0x7f0700079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 -- 172.21.15.29:0/2872702200 shutdown_connections 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.678+0000 7f0706e45640 1 -- 172.21.15.29:0/2872702200 wait complete. 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.679+0000 7f0706e45640 1 Processor -- start 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.679+0000 7f0706e45640 1 -- start start 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.679+0000 7f0706e45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0700102a60 0x7f070019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.679+0000 7f0706e45640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07001083a0 0x7f070019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f0706e45640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f0706e45640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0700112960 con 0x7f0700108cf0 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f0706e45640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07001127e0 con 0x7f07001083a0 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f0706e45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0700112ae0 con 0x7f0700102a60 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:33.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51592/0 (socket says 172.21.15.29:51592) 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 -- 172.21.15.29:0/2732651826 learned_addr learned my addr 172.21.15.29:0/2732651826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f06fffff640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07001083a0 0x7f070019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f0704be1640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0700102a60 0x7f070019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 -- 172.21.15.29:0/2732651826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0700102a60 msgr2=0x7f070019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0700102a60 0x7f070019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 -- 172.21.15.29:0/2732651826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07001083a0 msgr2=0x7f070019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07001083a0 0x7f070019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 -- 172.21.15.29:0/2732651826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07001a03d0 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f07053e2640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c970 tx=0x7f06f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.680+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8007bb0 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.681+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8007d50 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.681+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07001a06c0 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.681+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8004e60 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.681+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07001a6e20 con 0x7f0700108cf0 2024-09-21T01:46:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.682+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0700102e70 con 0x7f0700108cf0 2024-09-21T01:46:33.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.683+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f06f8005020 con 0x7f0700108cf0 2024-09-21T01:46:33.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.684+0000 7f06fdffb640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 0x7f06ec0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:33.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.684+0000 7f0704be1640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 0x7f06ec0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:33.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.684+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f06f80e6fd0 con 0x7f0700108cf0 2024-09-21T01:46:33.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.685+0000 7f0704be1640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 0x7f06ec0b8ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06e8031440 tx=0x7f06e8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:33.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.689+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06f8014030 con 0x7f0700108cf0 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T01:46:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-21T01:46:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T01:46:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-09-21T01:46:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T01:46:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T01:46:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:33 smithi190 ceph-mon[34368]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:46:33.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.909+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f070006aeb0 con 0x7f06ec0b65a0 2024-09-21T01:46:33.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.938+0000 7f06fdffb640 1 -- 172.21.15.29:0/2732651826 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f070006aeb0 con 0x7f06ec0b65a0 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.940+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 msgr2=0x7f06ec0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.940+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 0x7f06ec0b8ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06e8031440 tx=0x7f06e8005e80 comp rx=0 tx=0).stop 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.940+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 msgr2=0x7f070019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.940+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c970 tx=0x7f06f800ce30 comp rx=0 tx=0).stop 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 shutdown_connections 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f06ec0b65a0 0x7f06ec0b8ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0700108cf0 0x7f070019fca0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07001083a0 0x7f070019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 --2- 172.21.15.29:0/2732651826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0700102a60 0x7f070019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 >> 172.21.15.29:0/2732651826 conn(0x7f0700079730 msgr2=0x7f0700101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 shutdown_connections 2024-09-21T01:46:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:33.941+0000 7f0706e45640 1 -- 172.21.15.29:0/2732651826 wait complete. 2024-09-21T01:46:34.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:34 smithi066 ceph-mon[35001]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:46:34.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:34 smithi029 ceph-mon[31228]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:46:34.686 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T01:46:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:34 smithi190 ceph-mon[34368]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:46:34.951 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:35.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:35 smithi029 ceph-mon[31228]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:35.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:35 smithi029 ceph-mon[31228]: pgmap v307: 97 pgs: 97 active+clean; 627 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T01:46:35.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 -- 172.21.15.29:0/11976555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c102ac0 msgr2=0x7fac8c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:35.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 --2- 172.21.15.29:0/11976555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c102ac0 0x7fac8c102ed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac88009900 tx=0x7fac8802a410 comp rx=0 tx=0).stop 2024-09-21T01:46:35.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 -- 172.21.15.29:0/11976555 shutdown_connections 2024-09-21T01:46:35.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 --2- 172.21.15.29:0/11976555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 --2- 172.21.15.29:0/11976555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c1083b0 0x7fac8c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 --2- 172.21.15.29:0/11976555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c102ac0 0x7fac8c102ed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 -- 172.21.15.29:0/11976555 >> 172.21.15.29:0/11976555 conn(0x7fac8c079790 msgr2=0x7fac8c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 -- 172.21.15.29:0/11976555 shutdown_connections 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.466+0000 7fac93569640 1 -- 172.21.15.29:0/11976555 wait complete. 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 Processor -- start 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 -- start start 2024-09-21T01:46:35.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c102ac0 0x7fac8c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:35.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:35.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:35.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c1129c0 con 0x7fac8c1083b0 2024-09-21T01:46:35.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c112840 con 0x7fac8c108d00 2024-09-21T01:46:35.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac93569640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c112b40 con 0x7fac8c102ac0 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac90b04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac90b04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51604/0 (socket says 172.21.15.29:51604) 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.467+0000 7fac90b04640 1 -- 172.21.15.29:0/4078315486 learned_addr learned my addr 172.21.15.29:0/4078315486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac91305640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c102ac0 0x7fac8c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac91b06640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 -- 172.21.15.29:0/4078315486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c102ac0 msgr2=0x7fac8c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c102ac0 0x7fac8c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 -- 172.21.15.29:0/4078315486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 msgr2=0x7fac8c19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:35.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 -- 172.21.15.29:0/4078315486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c1a65a0 con 0x7fac8c1083b0 2024-09-21T01:46:35.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac91b06640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:35.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac90b04640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d750 tx=0x7fac7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:35.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c017070 con 0x7fac8c1083b0 2024-09-21T01:46:35.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c1a6890 con 0x7fac8c1083b0 2024-09-21T01:46:35.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7c0054d0 con 0x7fac8c1083b0 2024-09-21T01:46:35.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c004c30 con 0x7fac8c1083b0 2024-09-21T01:46:35.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.468+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c1a6d80 con 0x7fac8c1083b0 2024-09-21T01:46:35.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.470+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c102ed0 con 0x7fac8c1083b0 2024-09-21T01:46:35.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.472+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fac7c004dd0 con 0x7fac8c1083b0 2024-09-21T01:46:35.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.473+0000 7fac827fc640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 0x7fac780b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:35.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.473+0000 7fac91305640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 0x7fac780b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:35.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.474+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac7c0e72f0 con 0x7fac8c1083b0 2024-09-21T01:46:35.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.474+0000 7fac91305640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 0x7fac780b8ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:35.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.476+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fac7c0ada30 con 0x7fac8c1083b0 2024-09-21T01:46:35.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:35 smithi066 ceph-mon[35001]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:35.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:35 smithi066 ceph-mon[35001]: pgmap v307: 97 pgs: 97 active+clean; 627 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T01:46:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.697+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fac8c063800 con 0x7fac780b65d0 2024-09-21T01:46:35.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.702+0000 7fac827fc640 1 -- 172.21.15.29:0/4078315486 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fac8c063800 con 0x7fac780b65d0 2024-09-21T01:46:35.703 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/qa/sv_1/c64ca240-4c76-4594-a4c1-13b15f731d9d 2024-09-21T01:46:35.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 msgr2=0x7fac780b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:35.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 0x7fac780b8ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88031400 comp rx=0 tx=0).stop 2024-09-21T01:46:35.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 msgr2=0x7fac8c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:35.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d750 tx=0x7fac7c00dc10 comp rx=0 tx=0).stop 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 shutdown_connections 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac780b65d0 0x7fac780b8ae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac8c108d00 0x7fac8c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac8c1083b0 0x7fac8c19b850 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 --2- 172.21.15.29:0/4078315486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac8c102ac0 0x7fac8c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 >> 172.21.15.29:0/4078315486 conn(0x7fac8c079790 msgr2=0x7fac8c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 shutdown_connections 2024-09-21T01:46:35.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:35.704+0000 7fac93569640 1 -- 172.21.15.29:0/4078315486 wait complete. 2024-09-21T01:46:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:35 smithi190 ceph-mon[34368]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:35 smithi190 ceph-mon[34368]: pgmap v307: 97 pgs: 97 active+clean; 627 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T01:46:36.361 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:36 smithi029 ceph-mon[31228]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:36.534 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:46:36.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:36 smithi066 ceph-mon[35001]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:36.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:36 smithi190 ceph-mon[34368]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:46:36.792 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 -- 172.21.15.29:0/3209697958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 msgr2=0x7ff510103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 --2- 172.21.15.29:0/3209697958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510103f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff50002a410 comp rx=0 tx=0).stop 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 -- 172.21.15.29:0/3209697958 shutdown_connections 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 --2- 172.21.15.29:0/3209697958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff510109df0 0x7ff510110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 --2- 172.21.15.29:0/3209697958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5101094a0 0x7ff5101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 --2- 172.21.15.29:0/3209697958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510103f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 -- 172.21.15.29:0/3209697958 >> 172.21.15.29:0/3209697958 conn(0x7ff51007b850 msgr2=0x7ff51007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 -- 172.21.15.29:0/3209697958 shutdown_connections 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.292+0000 7ff514ffa640 1 -- 172.21.15.29:0/3209697958 wait complete. 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.293+0000 7ff514ffa640 1 Processor -- start 2024-09-21T01:46:37.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.293+0000 7ff514ffa640 1 -- start start 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 0x7ff510075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510109df0 0x7ff510076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff510113b10 con 0x7ff510103b60 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff510113990 con 0x7ff510109df0 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff514ffa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff510113c90 con 0x7ff5101094a0 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51630/0 (socket says 172.21.15.29:51630) 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 -- 172.21.15.29:0/889257894 learned_addr learned my addr 172.21.15.29:0/889257894 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50ed9d640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510109df0 0x7ff510076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50dd9b640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 0x7ff510075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 -- 172.21.15.29:0/889257894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 msgr2=0x7ff510075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 0x7ff510075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 -- 172.21.15.29:0/889257894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510109df0 msgr2=0x7ff510076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510109df0 0x7ff510076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 -- 172.21.15.29:0/889257894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5101a88a0 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50dd9b640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 0x7ff510075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff50e59c640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff500005940 tx=0x7ff5000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.294+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50003f030 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.295+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500005d10 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.295+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5101a8b30 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.295+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500035070 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.295+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101a9010 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.296+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff510103f70 con 0x7ff510103b60 2024-09-21T01:46:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.297+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff500042030 con 0x7ff510103b60 2024-09-21T01:46:37.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.298+0000 7ff4ff7fe640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 0x7ff4f80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:37.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.298+0000 7ff50dd9b640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 0x7ff4f80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:37.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.298+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff500107c30 con 0x7ff510103b60 2024-09-21T01:46:37.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.299+0000 7ff50dd9b640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 0x7ff4f80b8ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff510109200 tx=0x7ff504008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:37.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.302+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5000ce370 con 0x7ff510103b60 2024-09-21T01:46:37.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.526+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff510076a10 con 0x7ff510103b60 2024-09-21T01:46:37.528 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:37 smithi029 ceph-mon[31228]: pgmap v308: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:37.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.527+0000 7ff4ff7fe640 1 -- 172.21.15.29:0/889257894 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7ff5000e3e20 con 0x7ff510103b60 2024-09-21T01:46:37.529 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:37.529 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:46:37.529 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:46:37.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 msgr2=0x7ff4f80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:37.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 0x7ff4f80b8ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff510109200 tx=0x7ff504008040 comp rx=0 tx=0).stop 2024-09-21T01:46:37.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 msgr2=0x7ff510075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:37.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff500005940 tx=0x7ff5000042f0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 shutdown_connections 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4f80b65a0 0x7ff4f80b8ab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510109df0 0x7ff510076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5101094a0 0x7ff510075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 --2- 172.21.15.29:0/889257894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff510103b60 0x7ff510075710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 >> 172.21.15.29:0/889257894 conn(0x7ff51007b850 msgr2=0x7ff510107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 shutdown_connections 2024-09-21T01:46:37.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:37.530+0000 7ff514ffa640 1 -- 172.21.15.29:0/889257894 wait complete. 2024-09-21T01:46:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:37 smithi066 ceph-mon[35001]: pgmap v308: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:37 smithi190 ceph-mon[34368]: pgmap v308: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:38.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:46:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:38 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/889257894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:38.577 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:38 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/889257894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:38 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/889257894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 -- 172.21.15.29:0/1095357420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9718104840 msgr2=0x7f9718104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 --2- 172.21.15.29:0/1095357420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9718104840 0x7f9718104c50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9708009ca0 tx=0x7f970802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 -- 172.21.15.29:0/1095357420 shutdown_connections 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 --2- 172.21.15.29:0/1095357420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f971810ddf0 0x7f9718110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 --2- 172.21.15.29:0/1095357420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 --2- 172.21.15.29:0/1095357420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9718104840 0x7f9718104c50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 -- 172.21.15.29:0/1095357420 >> 172.21.15.29:0/1095357420 conn(0x7f9718079730 msgr2=0x7f9718079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:38.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 -- 172.21.15.29:0/1095357420 shutdown_connections 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.962+0000 7f971fbde640 1 -- 172.21.15.29:0/1095357420 wait complete. 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 Processor -- start 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 -- start start 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 0x7f971819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113ac0 con 0x7f971810ddf0 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113940 con 0x7f9718104840 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971fbde640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113c40 con 0x7f9718105270 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971e17b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971e17b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51646/0 (socket says 172.21.15.29:51646) 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971e17b640 1 -- 172.21.15.29:0/3081534122 learned_addr learned my addr 172.21.15.29:0/3081534122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:38.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.963+0000 7f971d97a640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 0x7f971819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971d179640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 -- 172.21.15.29:0/3081534122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 msgr2=0x7f971819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 -- 172.21.15.29:0/3081534122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 msgr2=0x7f971819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 0x7f971819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 -- 172.21.15.29:0/3081534122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97181a03a0 con 0x7f971810ddf0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971d97a640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 0x7f971819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971d179640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971e17b640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f971000c970 tx=0x7f971000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710007bb0 con 0x7f971810ddf0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97181a0690 con 0x7f971810ddf0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9710007d50 con 0x7f971810ddf0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710004e60 con 0x7f971810ddf0 2024-09-21T01:46:38.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.964+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a6eb0 con 0x7f971810ddf0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.965+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718104c50 con 0x7f971810ddf0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.967+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9710005000 con 0x7f971810ddf0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.968+0000 7f9716ffd640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 0x7f97000b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.968+0000 7f971d97a640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 0x7f97000b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.968+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f97100e7600 con 0x7f971810ddf0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.968+0000 7f971d97a640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 0x7f97000b8ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9708000f80 tx=0x7f9708031b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:38.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:38.971+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9710014030 con 0x7f971810ddf0 2024-09-21T01:46:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.160+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f971819b620 con 0x7f971810ddf0 2024-09-21T01:46:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.161+0000 7f9716ffd640 1 -- 172.21.15.29:0/3081534122 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7f97100add40 con 0x7f971810ddf0 2024-09-21T01:46:39.163 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:39.163 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.163+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 msgr2=0x7f97000b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.163+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 0x7f97000b8ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9708000f80 tx=0x7f9708031b10 comp rx=0 tx=0).stop 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.163+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 msgr2=0x7f971819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.163+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f971000c970 tx=0x7f971000ce30 comp rx=0 tx=0).stop 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 shutdown_connections 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97000b65d0 0x7f97000b8ae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f971810ddf0 0x7f971819fd30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:39.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9718105270 0x7f971819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 --2- 172.21.15.29:0/3081534122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9718104840 0x7f971819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 >> 172.21.15.29:0/3081534122 conn(0x7f9718079730 msgr2=0x7f9718107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 shutdown_connections 2024-09-21T01:46:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:39.164+0000 7f971fbde640 1 -- 172.21.15.29:0/3081534122 wait complete. 2024-09-21T01:46:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:39 smithi066 ceph-mon[35001]: pgmap v309: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:39 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3081534122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:39.584 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:39 smithi029 ceph-mon[31228]: pgmap v309: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:39.584 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:39 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3081534122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:39 smithi190 ceph-mon[34368]: pgmap v309: 97 pgs: 97 active+clean; 628 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:39 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3081534122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:40.028 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T01:46:40.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:46:40.294 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:40.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 -- 172.21.15.29:0/480162379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 msgr2=0x7f1d04102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 --2- 172.21.15.29:0/480162379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d04102e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009900 tx=0x7f1cf402a410 comp rx=0 tx=0).stop 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 -- 172.21.15.29:0/480162379 shutdown_connections 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 --2- 172.21.15.29:0/480162379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d04108d10 0x7f1d0410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 --2- 172.21.15.29:0/480162379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d041083c0 0x7f1d041087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 --2- 172.21.15.29:0/480162379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d04102e90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 -- 172.21.15.29:0/480162379 >> 172.21.15.29:0/480162379 conn(0x7f1d04079740 msgr2=0x7f1d04079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 -- 172.21.15.29:0/480162379 shutdown_connections 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.799+0000 7f1d0984d640 1 -- 172.21.15.29:0/480162379 wait complete. 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 Processor -- start 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 -- start start 2024-09-21T01:46:40.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 0x7f1d0419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d04108d10 0x7f1d0419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112980 con 0x7f1d04102a80 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112800 con 0x7f1d04108d10 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d0984d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112b00 con 0x7f1d041083c0 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51666/0 (socket says 172.21.15.29:51666) 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.800+0000 7f1d02ffd640 1 -- 172.21.15.29:0/1147133024 learned_addr learned my addr 172.21.15.29:0/1147133024 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d037fe640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d04108d10 0x7f1d0419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d027fc640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 0x7f1d0419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 -- 172.21.15.29:0/1147133024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 msgr2=0x7f1d0419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 0x7f1d0419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 -- 172.21.15.29:0/1147133024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d04108d10 msgr2=0x7f1d0419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:40.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d04108d10 0x7f1d0419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 -- 172.21.15.29:0/1147133024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d041a03b0 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d027fc640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 0x7f1d0419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d02ffd640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006010 tx=0x7f1cf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf403f030 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf4003800 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d041a0640 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4035070 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.801+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a0b20 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.802+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04103d00 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.804+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1cf4003cb0 con 0x7f1d04102a80 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.805+0000 7f1d0884b640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 0x7f1cec0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.805+0000 7f1d027fc640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 0x7f1cec0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:40.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.805+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1cf4108080 con 0x7f1d04102a80 2024-09-21T01:46:40.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.805+0000 7f1d027fc640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 0x7f1cec0b8ae0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d04100ce0 tx=0x7f1cf8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:40.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.808+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1cf40ce760 con 0x7f1d04102a80 2024-09-21T01:46:40.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:40.995+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d04106360 con 0x7f1cec0b65d0 2024-09-21T01:46:41.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.002+0000 7f1d0884b640 1 -- 172.21.15.29:0/1147133024 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7f1d04106360 con 0x7f1cec0b65d0 2024-09-21T01:46:41.005 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:41.007 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:46:41.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 msgr2=0x7f1cec0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 0x7f1cec0b8ae0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d04100ce0 tx=0x7f1cf8005d30 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 msgr2=0x7f1d0419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006010 tx=0x7f1cf40042f0 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 shutdown_connections 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1cec0b65d0 0x7f1cec0b8ae0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d04108d10 0x7f1d0419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d041083c0 0x7f1d0419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 --2- 172.21.15.29:0/1147133024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d04102a80 0x7f1d0419a500 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 >> 172.21.15.29:0/1147133024 conn(0x7f1d04079740 msgr2=0x7f1d04101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 shutdown_connections 2024-09-21T01:46:41.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:41.005+0000 7f1d0984d640 1 -- 172.21.15.29:0/1147133024 wait complete. 2024-09-21T01:46:41.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:41 smithi066 ceph-mon[35001]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 15 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:41 smithi029 ceph-mon[31228]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 15 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:41 smithi190 ceph-mon[34368]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 15 op/s; 72 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:41.859 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-21T01:46:40.097007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651665,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":279,"num_read_kb":234,"num_write":359,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402752,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539072,"statfs":{"total":1151940427776,"available":1151528009728,"internally_reserved":0,"allocated":12140544,"data_stored":6726339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46272,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":102,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002010"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225240+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:46:33.225240+0000","last_peered":"2024-09-21T01:46:33.225240+0000","last_clean":"2024-09-21T01:46:33.225240+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:46:33.225240+0000","last_undegraded":"2024-09-21T01:46:33.225240+0000","last_fullsized":"2024-09-21T01:46:33.225240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260760+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:46:33.260760+0000","last_peered":"2024-09-21T01:46:33.260760+0000","last_clean":"2024-09-21T01:46:33.260760+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:46:33.260760+0000","last_undegraded":"2024-09-21T01:46:33.260760+0000","last_fullsized":"2024-09-21T01:46:33.260760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260728+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:46:33.260728+0000","last_peered":"2024-09-21T01:46:33.260728+0000","last_clean":"2024-09-21T01:46:33.260728+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:46:33.260728+0000","last_undegraded":"2024-09-21T01:46:33.260728+0000","last_fullsized":"2024-09-21T01:46:33.260728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234921+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:46:33.234921+0000","last_peered":"2024-09-21T01:46:33.234921+0000","last_clean":"2024-09-21T01:46:33.234921+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:46:33.234921+0000","last_undegraded":"2024-09-21T01:46:33.234921+0000","last_fullsized":"2024-09-21T01:46:33.234921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228187+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:46:33.228187+0000","last_peered":"2024-09-21T01:46:33.228187+0000","last_clean":"2024-09-21T01:46:33.228187+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:46:33.228187+0000","last_undegraded":"2024-09-21T01:46:33.228187+0000","last_fullsized":"2024-09-21T01:46:33.228187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225298+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:46:33.225298+0000","last_peered":"2024-09-21T01:46:33.225298+0000","last_clean":"2024-09-21T01:46:33.225298+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:46:33.225298+0000","last_undegraded":"2024-09-21T01:46:33.225298+0000","last_fullsized":"2024-09-21T01:46:33.225298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231465+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:46:33.231465+0000","last_peered":"2024-09-21T01:46:33.231465+0000","last_clean":"2024-09-21T01:46:33.231465+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:46:33.231465+0000","last_undegraded":"2024-09-21T01:46:33.231465+0000","last_fullsized":"2024-09-21T01:46:33.231465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231501+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:46:33.231501+0000","last_peered":"2024-09-21T01:46:33.231501+0000","last_clean":"2024-09-21T01:46:33.231501+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:46:33.231501+0000","last_undegraded":"2024-09-21T01:46:33.231501+0000","last_fullsized":"2024-09-21T01:46:33.231501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228282+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:46:33.228282+0000","last_peered":"2024-09-21T01:46:33.228282+0000","last_clean":"2024-09-21T01:46:33.228282+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:46:33.228282+0000","last_undegraded":"2024-09-21T01:46:33.228282+0000","last_fullsized":"2024-09-21T01:46:33.228282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226004+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:46:33.226004+0000","last_peered":"2024-09-21T01:46:33.226004+0000","last_clean":"2024-09-21T01:46:33.226004+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:46:33.226004+0000","last_undegraded":"2024-09-21T01:46:33.226004+0000","last_fullsized":"2024-09-21T01:46:33.226004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231742+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:46:33.231742+0000","last_peered":"2024-09-21T01:46:33.231742+0000","last_clean":"2024-09-21T01:46:33.231742+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:46:33.231742+0000","last_undegraded":"2024-09-21T01:46:33.231742+0000","last_fullsized":"2024-09-21T01:46:33.231742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237387+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:46:33.237387+0000","last_peered":"2024-09-21T01:46:33.237387+0000","last_clean":"2024-09-21T01:46:33.237387+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:46:33.237387+0000","last_undegraded":"2024-09-21T01:46:33.237387+0000","last_fullsized":"2024-09-21T01:46:33.237387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226996+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:46:33.226996+0000","last_peered":"2024-09-21T01:46:33.226996+0000","last_clean":"2024-09-21T01:46:33.226996+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:46:33.226996+0000","last_undegraded":"2024-09-21T01:46:33.226996+0000","last_fullsized":"2024-09-21T01:46:33.226996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260670+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:46:33.260670+0000","last_peered":"2024-09-21T01:46:33.260670+0000","last_clean":"2024-09-21T01:46:33.260670+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:46:33.260670+0000","last_undegraded":"2024-09-21T01:46:33.260670+0000","last_fullsized":"2024-09-21T01:46:33.260670+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260568+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:46:33.260568+0000","last_peered":"2024-09-21T01:46:33.260568+0000","last_clean":"2024-09-21T01:46:33.260568+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:46:33.260568+0000","last_undegraded":"2024-09-21T01:46:33.260568+0000","last_fullsized":"2024-09-21T01:46:33.260568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227507+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:46:33.227507+0000","last_peered":"2024-09-21T01:46:33.227507+0000","last_clean":"2024-09-21T01:46:33.227507+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:46:33.227507+0000","last_undegraded":"2024-09-21T01:46:33.227507+0000","last_fullsized":"2024-09-21T01:46:33.227507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225461+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:46:33.225461+0000","last_peered":"2024-09-21T01:46:33.225461+0000","last_clean":"2024-09-21T01:46:33.225461+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:46:33.225461+0000","last_undegraded":"2024-09-21T01:46:33.225461+0000","last_fullsized":"2024-09-21T01:46:33.225461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226126+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:46:33.226126+0000","last_peered":"2024-09-21T01:46:33.226126+0000","last_clean":"2024-09-21T01:46:33.226126+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:46:33.226126+0000","last_undegraded":"2024-09-21T01:46:33.226126+0000","last_fullsized":"2024-09-21T01:46:33.226126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230980+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:46:33.230980+0000","last_peered":"2024-09-21T01:46:33.230980+0000","last_clean":"2024-09-21T01:46:33.230980+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:46:33.230980+0000","last_undegraded":"2024-09-21T01:46:33.230980+0000","last_fullsized":"2024-09-21T01:46:33.230980+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224911+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:46:33.224911+0000","last_peered":"2024-09-21T01:46:33.224911+0000","last_clean":"2024-09-21T01:46:33.224911+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:46:33.224911+0000","last_undegraded":"2024-09-21T01:46:33.224911+0000","last_fullsized":"2024-09-21T01:46:33.224911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227097+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:46:33.227097+0000","last_peered":"2024-09-21T01:46:33.227097+0000","last_clean":"2024-09-21T01:46:33.227097+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:46:33.227097+0000","last_undegraded":"2024-09-21T01:46:33.227097+0000","last_fullsized":"2024-09-21T01:46:33.227097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227796+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:46:33.227796+0000","last_peered":"2024-09-21T01:46:33.227796+0000","last_clean":"2024-09-21T01:46:33.227796+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:46:33.227796+0000","last_undegraded":"2024-09-21T01:46:33.227796+0000","last_fullsized":"2024-09-21T01:46:33.227796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223435+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:46:33.223435+0000","last_peered":"2024-09-21T01:46:33.223435+0000","last_clean":"2024-09-21T01:46:33.223435+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:46:33.223435+0000","last_undegraded":"2024-09-21T01:46:33.223435+0000","last_fullsized":"2024-09-21T01:46:33.223435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227979+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:46:33.227979+0000","last_peered":"2024-09-21T01:46:33.227979+0000","last_clean":"2024-09-21T01:46:33.227979+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:46:33.227979+0000","last_undegraded":"2024-09-21T01:46:33.227979+0000","last_fullsized":"2024-09-21T01:46:33.227979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224001+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:46:33.224001+0000","last_peered":"2024-09-21T01:46:33.224001+0000","last_clean":"2024-09-21T01:46:33.224001+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:46:33.224001+0000","last_undegraded":"2024-09-21T01:46:33.224001+0000","last_fullsized":"2024-09-21T01:46:33.224001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232919+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:46:33.232919+0000","last_peered":"2024-09-21T01:46:33.232919+0000","last_clean":"2024-09-21T01:46:33.232919+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:46:33.232919+0000","last_undegraded":"2024-09-21T01:46:33.232919+0000","last_fullsized":"2024-09-21T01:46:33.232919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224984+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:46:33.224984+0000","last_peered":"2024-09-21T01:46:33.224984+0000","last_clean":"2024-09-21T01:46:33.224984+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:46:33.224984+0000","last_undegraded":"2024-09-21T01:46:33.224984+0000","last_fullsized":"2024-09-21T01:46:33.224984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226981+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:46:33.226981+0000","last_peered":"2024-09-21T01:46:33.226981+0000","last_clean":"2024-09-21T01:46:33.226981+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:46:33.226981+0000","last_undegraded":"2024-09-21T01:46:33.226981+0000","last_fullsized":"2024-09-21T01:46:33.226981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260782+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:46:33.260782+0000","last_peered":"2024-09-21T01:46:33.260782+0000","last_clean":"2024-09-21T01:46:33.260782+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:46:33.260782+0000","last_undegraded":"2024-09-21T01:46:33.260782+0000","last_fullsized":"2024-09-21T01:46:33.260782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230552+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:46:33.230552+0000","last_peered":"2024-09-21T01:46:33.230552+0000","last_clean":"2024-09-21T01:46:33.230552+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:46:33.230552+0000","last_undegraded":"2024-09-21T01:46:33.230552+0000","last_fullsized":"2024-09-21T01:46:33.230552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230632+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:46:33.230632+0000","last_peered":"2024-09-21T01:46:33.230632+0000","last_clean":"2024-09-21T01:46:33.230632+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:46:33.230632+0000","last_undegraded":"2024-09-21T01:46:33.230632+0000","last_fullsized":"2024-09-21T01:46:33.230632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224723+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:46:33.224723+0000","last_peered":"2024-09-21T01:46:33.224723+0000","last_clean":"2024-09-21T01:46:33.224723+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:46:33.224723+0000","last_undegraded":"2024-09-21T01:46:33.224723+0000","last_fullsized":"2024-09-21T01:46:33.224723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228787+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:46:33.228787+0000","last_peered":"2024-09-21T01:46:33.228787+0000","last_clean":"2024-09-21T01:46:33.228787+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:46:33.228787+0000","last_undegraded":"2024-09-21T01:46:33.228787+0000","last_fullsized":"2024-09-21T01:46:33.228787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.238800+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:46:37.238800+0000","last_peered":"2024-09-21T01:46:37.238800+0000","last_clean":"2024-09-21T01:46:37.238800+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:46:37.238800+0000","last_undegraded":"2024-09-21T01:46:37.238800+0000","last_fullsized":"2024-09-21T01:46:37.238800+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228873+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:46:33.228873+0000","last_peered":"2024-09-21T01:46:33.228873+0000","last_clean":"2024-09-21T01:46:33.228873+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:46:33.228873+0000","last_undegraded":"2024-09-21T01:46:33.228873+0000","last_fullsized":"2024-09-21T01:46:33.228873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223419+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:46:33.223419+0000","last_peered":"2024-09-21T01:46:33.223419+0000","last_clean":"2024-09-21T01:46:33.223419+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:46:33.223419+0000","last_undegraded":"2024-09-21T01:46:33.223419+0000","last_fullsized":"2024-09-21T01:46:33.223419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225008+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:46:33.225008+0000","last_peered":"2024-09-21T01:46:33.225008+0000","last_clean":"2024-09-21T01:46:33.225008+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:46:33.225008+0000","last_undegraded":"2024-09-21T01:46:33.225008+0000","last_fullsized":"2024-09-21T01:46:33.225008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.239618+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:46:38.239618+0000","last_peered":"2024-09-21T01:46:38.239618+0000","last_clean":"2024-09-21T01:46:38.239618+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:46:38.239618+0000","last_undegraded":"2024-09-21T01:46:38.239618+0000","last_fullsized":"2024-09-21T01:46:38.239618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228320+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:46:33.228320+0000","last_peered":"2024-09-21T01:46:33.228320+0000","last_clean":"2024-09-21T01:46:33.228320+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:46:33.228320+0000","last_undegraded":"2024-09-21T01:46:33.228320+0000","last_fullsized":"2024-09-21T01:46:33.228320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260933+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:46:33.260933+0000","last_peered":"2024-09-21T01:46:33.260933+0000","last_clean":"2024-09-21T01:46:33.260933+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:46:33.260933+0000","last_undegraded":"2024-09-21T01:46:33.260933+0000","last_fullsized":"2024-09-21T01:46:33.260933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227879+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:46:33.227879+0000","last_peered":"2024-09-21T01:46:33.227879+0000","last_clean":"2024-09-21T01:46:33.227879+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:46:33.227879+0000","last_undegraded":"2024-09-21T01:46:33.227879+0000","last_fullsized":"2024-09-21T01:46:33.227879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":90,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.243460+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:46:38.243460+0000","last_peered":"2024-09-21T01:46:38.243460+0000","last_clean":"2024-09-21T01:46:38.243460+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:46:38.243460+0000","last_undegraded":"2024-09-21T01:46:38.243460+0000","last_fullsized":"2024-09-21T01:46:38.243460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54250,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":72,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229895+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:46:33.229895+0000","last_peered":"2024-09-21T01:46:33.229895+0000","last_clean":"2024-09-21T01:46:33.229895+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:46:33.229895+0000","last_undegraded":"2024-09-21T01:46:33.229895+0000","last_fullsized":"2024-09-21T01:46:33.229895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226066+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:46:33.226066+0000","last_peered":"2024-09-21T01:46:33.226066+0000","last_clean":"2024-09-21T01:46:33.226066+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:46:33.226066+0000","last_undegraded":"2024-09-21T01:46:33.226066+0000","last_fullsized":"2024-09-21T01:46:33.226066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232179+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:46:33.232179+0000","last_peered":"2024-09-21T01:46:33.232179+0000","last_clean":"2024-09-21T01:46:33.232179+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:46:33.232179+0000","last_undegraded":"2024-09-21T01:46:33.232179+0000","last_fullsized":"2024-09-21T01:46:33.232179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:36.241632+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:46:36.241632+0000","last_peered":"2024-09-21T01:46:36.241632+0000","last_clean":"2024-09-21T01:46:36.241632+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:46:36.241632+0000","last_undegraded":"2024-09-21T01:46:36.241632+0000","last_fullsized":"2024-09-21T01:46:36.241632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231720+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:46:33.231720+0000","last_peered":"2024-09-21T01:46:33.231720+0000","last_clean":"2024-09-21T01:46:33.231720+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:46:33.231720+0000","last_undegraded":"2024-09-21T01:46:33.231720+0000","last_fullsized":"2024-09-21T01:46:33.231720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227856+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:46:33.227856+0000","last_peered":"2024-09-21T01:46:33.227856+0000","last_clean":"2024-09-21T01:46:33.227856+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:46:33.227856+0000","last_undegraded":"2024-09-21T01:46:33.227856+0000","last_fullsized":"2024-09-21T01:46:33.227856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234181+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:46:33.234181+0000","last_peered":"2024-09-21T01:46:33.234181+0000","last_clean":"2024-09-21T01:46:33.234181+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:46:33.234181+0000","last_undegraded":"2024-09-21T01:46:33.234181+0000","last_fullsized":"2024-09-21T01:46:33.234181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231777+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:46:33.231777+0000","last_peered":"2024-09-21T01:46:33.231777+0000","last_clean":"2024-09-21T01:46:33.231777+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:46:33.231777+0000","last_undegraded":"2024-09-21T01:46:33.231777+0000","last_fullsized":"2024-09-21T01:46:33.231777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231571+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:46:33.231571+0000","last_peered":"2024-09-21T01:46:33.231571+0000","last_clean":"2024-09-21T01:46:33.231571+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:46:33.231571+0000","last_undegraded":"2024-09-21T01:46:33.231571+0000","last_fullsized":"2024-09-21T01:46:33.231571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":44,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:34.236228+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:46:34.236228+0000","last_peered":"2024-09-21T01:46:34.236228+0000","last_clean":"2024-09-21T01:46:34.236228+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:46:34.236228+0000","last_undegraded":"2024-09-21T01:46:34.236228+0000","last_fullsized":"2024-09-21T01:46:34.236228+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231672+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:46:33.231672+0000","last_peered":"2024-09-21T01:46:33.231672+0000","last_clean":"2024-09-21T01:46:33.231672+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:46:33.231672+0000","last_undegraded":"2024-09-21T01:46:33.231672+0000","last_fullsized":"2024-09-21T01:46:33.231672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231609+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231609+0000","last_peered":"2024-09-21T01:46:33.231609+0000","last_clean":"2024-09-21T01:46:33.231609+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231609+0000","last_undegraded":"2024-09-21T01:46:33.231609+0000","last_fullsized":"2024-09-21T01:46:33.231609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237072+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237072+0000","last_peered":"2024-09-21T01:46:33.237072+0000","last_clean":"2024-09-21T01:46:33.237072+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237072+0000","last_undegraded":"2024-09-21T01:46:33.237072+0000","last_fullsized":"2024-09-21T01:46:33.237072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223710+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:46:33.223710+0000","last_peered":"2024-09-21T01:46:33.223710+0000","last_clean":"2024-09-21T01:46:33.223710+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:46:33.223710+0000","last_undegraded":"2024-09-21T01:46:33.223710+0000","last_fullsized":"2024-09-21T01:46:33.223710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231445+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231445+0000","last_peered":"2024-09-21T01:46:33.231445+0000","last_clean":"2024-09-21T01:46:33.231445+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231445+0000","last_undegraded":"2024-09-21T01:46:33.231445+0000","last_fullsized":"2024-09-21T01:46:33.231445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226072+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:46:33.226072+0000","last_peered":"2024-09-21T01:46:33.226072+0000","last_clean":"2024-09-21T01:46:33.226072+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:46:33.226072+0000","last_undegraded":"2024-09-21T01:46:33.226072+0000","last_fullsized":"2024-09-21T01:46:33.226072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.242350+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:46:37.242350+0000","last_peered":"2024-09-21T01:46:37.242350+0000","last_clean":"2024-09-21T01:46:37.242350+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:46:37.242350+0000","last_undegraded":"2024-09-21T01:46:37.242350+0000","last_fullsized":"2024-09-21T01:46:37.242350+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237841+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:46:33.237841+0000","last_peered":"2024-09-21T01:46:33.237841+0000","last_clean":"2024-09-21T01:46:33.237841+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:46:33.237841+0000","last_undegraded":"2024-09-21T01:46:33.237841+0000","last_fullsized":"2024-09-21T01:46:33.237841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"74'6","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.239429+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:46:38.239429+0000","last_peered":"2024-09-21T01:46:38.239429+0000","last_clean":"2024-09-21T01:46:38.239429+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:46:38.239429+0000","last_undegraded":"2024-09-21T01:46:38.239429+0000","last_fullsized":"2024-09-21T01:46:38.239429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225616+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:46:33.225616+0000","last_peered":"2024-09-21T01:46:33.225616+0000","last_clean":"2024-09-21T01:46:33.225616+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:46:33.225616+0000","last_undegraded":"2024-09-21T01:46:33.225616+0000","last_fullsized":"2024-09-21T01:46:33.225616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225703+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225703+0000","last_peered":"2024-09-21T01:46:33.225703+0000","last_clean":"2024-09-21T01:46:33.225703+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225703+0000","last_undegraded":"2024-09-21T01:46:33.225703+0000","last_fullsized":"2024-09-21T01:46:33.225703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225654+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:46:33.225654+0000","last_peered":"2024-09-21T01:46:33.225654+0000","last_clean":"2024-09-21T01:46:33.225654+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:46:33.225654+0000","last_undegraded":"2024-09-21T01:46:33.225654+0000","last_fullsized":"2024-09-21T01:46:33.225654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230530+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230530+0000","last_peered":"2024-09-21T01:46:33.230530+0000","last_clean":"2024-09-21T01:46:33.230530+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230530+0000","last_undegraded":"2024-09-21T01:46:33.230530+0000","last_fullsized":"2024-09-21T01:46:33.230530+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227173+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:46:33.227173+0000","last_peered":"2024-09-21T01:46:33.227173+0000","last_clean":"2024-09-21T01:46:33.227173+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:46:33.227173+0000","last_undegraded":"2024-09-21T01:46:33.227173+0000","last_fullsized":"2024-09-21T01:46:33.227173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:35.858088+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:35.858088+0000","last_peered":"2024-09-21T01:46:35.858088+0000","last_clean":"2024-09-21T01:46:35.858088+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:35.858088+0000","last_undegraded":"2024-09-21T01:46:35.858088+0000","last_fullsized":"2024-09-21T01:46:35.858088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224103+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:46:33.224103+0000","last_peered":"2024-09-21T01:46:33.224103+0000","last_clean":"2024-09-21T01:46:33.224103+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:46:33.224103+0000","last_undegraded":"2024-09-21T01:46:33.224103+0000","last_fullsized":"2024-09-21T01:46:33.224103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237584+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237584+0000","last_peered":"2024-09-21T01:46:33.237584+0000","last_clean":"2024-09-21T01:46:33.237584+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237584+0000","last_undegraded":"2024-09-21T01:46:33.237584+0000","last_fullsized":"2024-09-21T01:46:33.237584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260770+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:46:33.260770+0000","last_peered":"2024-09-21T01:46:33.260770+0000","last_clean":"2024-09-21T01:46:33.260770+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:46:33.260770+0000","last_undegraded":"2024-09-21T01:46:33.260770+0000","last_fullsized":"2024-09-21T01:46:33.260770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228572+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:46:33.228572+0000","last_peered":"2024-09-21T01:46:33.228572+0000","last_clean":"2024-09-21T01:46:33.228572+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:46:33.228572+0000","last_undegraded":"2024-09-21T01:46:33.228572+0000","last_fullsized":"2024-09-21T01:46:33.228572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224910+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:46:33.224910+0000","last_peered":"2024-09-21T01:46:33.224910+0000","last_clean":"2024-09-21T01:46:33.224910+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:46:33.224910+0000","last_undegraded":"2024-09-21T01:46:33.224910+0000","last_fullsized":"2024-09-21T01:46:33.224910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223770+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:46:33.223770+0000","last_peered":"2024-09-21T01:46:33.223770+0000","last_clean":"2024-09-21T01:46:33.223770+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:46:33.223770+0000","last_undegraded":"2024-09-21T01:46:33.223770+0000","last_fullsized":"2024-09-21T01:46:33.223770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231959+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:46:33.231959+0000","last_peered":"2024-09-21T01:46:33.231959+0000","last_clean":"2024-09-21T01:46:33.231959+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:46:33.231959+0000","last_undegraded":"2024-09-21T01:46:33.231959+0000","last_fullsized":"2024-09-21T01:46:33.231959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227958+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:46:33.227958+0000","last_peered":"2024-09-21T01:46:33.227958+0000","last_clean":"2024-09-21T01:46:33.227958+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:46:33.227958+0000","last_undegraded":"2024-09-21T01:46:33.227958+0000","last_fullsized":"2024-09-21T01:46:33.227958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227724+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:46:33.227724+0000","last_peered":"2024-09-21T01:46:33.227724+0000","last_clean":"2024-09-21T01:46:33.227724+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:46:33.227724+0000","last_undegraded":"2024-09-21T01:46:33.227724+0000","last_fullsized":"2024-09-21T01:46:33.227724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260935+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:46:33.260935+0000","last_peered":"2024-09-21T01:46:33.260935+0000","last_clean":"2024-09-21T01:46:33.260935+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:46:33.260935+0000","last_undegraded":"2024-09-21T01:46:33.260935+0000","last_fullsized":"2024-09-21T01:46:33.260935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224309+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224309+0000","last_peered":"2024-09-21T01:46:33.224309+0000","last_clean":"2024-09-21T01:46:33.224309+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224309+0000","last_undegraded":"2024-09-21T01:46:33.224309+0000","last_fullsized":"2024-09-21T01:46:33.224309+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224094+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:46:33.224094+0000","last_peered":"2024-09-21T01:46:33.224094+0000","last_clean":"2024-09-21T01:46:33.224094+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:46:33.224094+0000","last_undegraded":"2024-09-21T01:46:33.224094+0000","last_fullsized":"2024-09-21T01:46:33.224094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.237058+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:46:37.237058+0000","last_peered":"2024-09-21T01:46:37.237058+0000","last_clean":"2024-09-21T01:46:37.237058+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:46:37.237058+0000","last_undegraded":"2024-09-21T01:46:37.237058+0000","last_fullsized":"2024-09-21T01:46:37.237058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":203,"num_read_kb":170,"num_write":234,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":183189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776148,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1044083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":991630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":398842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822706,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32616,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712536,"statfs":{"total":95995035648,"available":95961636864,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40344,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704808,"statfs":{"total":95995035648,"available":95953723392,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":453834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":454005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:46:41.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:46:42.126 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:42.528 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:42 smithi029 ceph-mon[31228]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:42.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:42 smithi066 ceph-mon[35001]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 -- 172.21.15.29:0/1373448238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 msgr2=0x7f035810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 --2- 172.21.15.29:0/1373448238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f035810d8d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f034c009900 tx=0x7f034c02a410 comp rx=0 tx=0).stop 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 -- 172.21.15.29:0/1373448238 shutdown_connections 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 --2- 172.21.15.29:0/1373448238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f035810de60 0x7f0358110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 --2- 172.21.15.29:0/1373448238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f035810d8d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 --2- 172.21.15.29:0/1373448238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358104a30 0x7f0358104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 -- 172.21.15.29:0/1373448238 >> 172.21.15.29:0/1373448238 conn(0x7f03580797b0 msgr2=0x7f0358079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 -- 172.21.15.29:0/1373448238 shutdown_connections 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 -- 172.21.15.29:0/1373448238 wait complete. 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.620+0000 7f035d4c1640 1 Processor -- start 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 -- start start 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0358104a30 0x7f0358102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f035810de60 0x7f03581031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0358113aa0 con 0x7f0358105380 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0358113920 con 0x7f035810de60 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f035d4c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0358113c20 con 0x7f0358104a30 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36596/0 (socket says 172.21.15.29:36596) 2024-09-21T01:46:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 -- 172.21.15.29:0/221090806 learned_addr learned my addr 172.21.15.29:0/221090806 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f0356ffd640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0358104a30 0x7f0358102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03577fe640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f035810de60 0x7f03581031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 -- 172.21.15.29:0/221090806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0358104a30 msgr2=0x7f0358102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0358104a30 0x7f0358102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.621+0000 7f03567fc640 1 -- 172.21.15.29:0/221090806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f035810de60 msgr2=0x7f03581031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f03567fc640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f035810de60 0x7f03581031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f03567fc640 1 -- 172.21.15.29:0/221090806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03581a87b0 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f03567fc640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f034c006010 tx=0x7f034c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c03f030 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03581a8a40 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c003aa0 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c035070 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.622+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581a8f70 con 0x7f0358105380 2024-09-21T01:46:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.623+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0358104e40 con 0x7f0358105380 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.625+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f034c042030 con 0x7f0358105380 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.625+0000 7f033ffff640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 0x7f03400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.626+0000 7f0356ffd640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 0x7f03400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.626+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f034c107d70 con 0x7f0358105380 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.626+0000 7f0356ffd640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 0x7f03400b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03581041c0 tx=0x7f0348009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:42.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.629+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f034c0ce4b0 con 0x7f0358105380 2024-09-21T01:46:42.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:42 smithi190 ceph-mon[34368]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:42.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.817+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0358063800 con 0x7f03400b65a0 2024-09-21T01:46:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.822+0000 7f033ffff640 1 -- 172.21.15.29:0/221090806 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258753 (secure 0 0 0) 0x7f0358063800 con 0x7f03400b65a0 2024-09-21T01:46:42.827 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:42.828 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:46:42.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 msgr2=0x7f03400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:42.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 0x7f03400b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03581041c0 tx=0x7f0348009500 comp rx=0 tx=0).stop 2024-09-21T01:46:42.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 msgr2=0x7f0358102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:42.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f034c006010 tx=0x7f034c004060 comp rx=0 tx=0).stop 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 shutdown_connections 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f03400b65a0 0x7f03400b8ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f035810de60 0x7f03581031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0358105380 0x7f0358102cb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 --2- 172.21.15.29:0/221090806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0358104a30 0x7f0358102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:42.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 >> 172.21.15.29:0/221090806 conn(0x7f03580797b0 msgr2=0x7f03580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:42.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 shutdown_connections 2024-09-21T01:46:42.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:42.825+0000 7f035d4c1640 1 -- 172.21.15.29:0/221090806 wait complete. 2024-09-21T01:46:43.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:43 smithi066 ceph-mon[35001]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s; 58 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:43.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:43 smithi066 ceph-mon[35001]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:43.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:43 smithi029 ceph-mon[31228]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s; 58 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:43.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:43 smithi029 ceph-mon[31228]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:43.653 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-21T01:46:42.097540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651665,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":284,"num_read_kb":234,"num_write":359,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402748,"kb_used_data":11860,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539076,"statfs":{"total":1151940427776,"available":1151528013824,"internally_reserved":0,"allocated":12144640,"data_stored":6730596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46272,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.876678"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225240+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:46:33.225240+0000","last_peered":"2024-09-21T01:46:33.225240+0000","last_clean":"2024-09-21T01:46:33.225240+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:46:33.225240+0000","last_undegraded":"2024-09-21T01:46:33.225240+0000","last_fullsized":"2024-09-21T01:46:33.225240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260760+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:46:33.260760+0000","last_peered":"2024-09-21T01:46:33.260760+0000","last_clean":"2024-09-21T01:46:33.260760+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:46:33.260760+0000","last_undegraded":"2024-09-21T01:46:33.260760+0000","last_fullsized":"2024-09-21T01:46:33.260760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260728+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:46:33.260728+0000","last_peered":"2024-09-21T01:46:33.260728+0000","last_clean":"2024-09-21T01:46:33.260728+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:46:33.260728+0000","last_undegraded":"2024-09-21T01:46:33.260728+0000","last_fullsized":"2024-09-21T01:46:33.260728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234921+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:46:33.234921+0000","last_peered":"2024-09-21T01:46:33.234921+0000","last_clean":"2024-09-21T01:46:33.234921+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:46:33.234921+0000","last_undegraded":"2024-09-21T01:46:33.234921+0000","last_fullsized":"2024-09-21T01:46:33.234921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228187+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:46:33.228187+0000","last_peered":"2024-09-21T01:46:33.228187+0000","last_clean":"2024-09-21T01:46:33.228187+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:46:33.228187+0000","last_undegraded":"2024-09-21T01:46:33.228187+0000","last_fullsized":"2024-09-21T01:46:33.228187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225298+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:46:33.225298+0000","last_peered":"2024-09-21T01:46:33.225298+0000","last_clean":"2024-09-21T01:46:33.225298+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:46:33.225298+0000","last_undegraded":"2024-09-21T01:46:33.225298+0000","last_fullsized":"2024-09-21T01:46:33.225298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231465+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:46:33.231465+0000","last_peered":"2024-09-21T01:46:33.231465+0000","last_clean":"2024-09-21T01:46:33.231465+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:46:33.231465+0000","last_undegraded":"2024-09-21T01:46:33.231465+0000","last_fullsized":"2024-09-21T01:46:33.231465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231501+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:46:33.231501+0000","last_peered":"2024-09-21T01:46:33.231501+0000","last_clean":"2024-09-21T01:46:33.231501+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:46:33.231501+0000","last_undegraded":"2024-09-21T01:46:33.231501+0000","last_fullsized":"2024-09-21T01:46:33.231501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228282+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:46:33.228282+0000","last_peered":"2024-09-21T01:46:33.228282+0000","last_clean":"2024-09-21T01:46:33.228282+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:46:33.228282+0000","last_undegraded":"2024-09-21T01:46:33.228282+0000","last_fullsized":"2024-09-21T01:46:33.228282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226004+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:46:33.226004+0000","last_peered":"2024-09-21T01:46:33.226004+0000","last_clean":"2024-09-21T01:46:33.226004+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:46:33.226004+0000","last_undegraded":"2024-09-21T01:46:33.226004+0000","last_fullsized":"2024-09-21T01:46:33.226004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231742+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:46:33.231742+0000","last_peered":"2024-09-21T01:46:33.231742+0000","last_clean":"2024-09-21T01:46:33.231742+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:46:33.231742+0000","last_undegraded":"2024-09-21T01:46:33.231742+0000","last_fullsized":"2024-09-21T01:46:33.231742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237387+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:46:33.237387+0000","last_peered":"2024-09-21T01:46:33.237387+0000","last_clean":"2024-09-21T01:46:33.237387+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:46:33.237387+0000","last_undegraded":"2024-09-21T01:46:33.237387+0000","last_fullsized":"2024-09-21T01:46:33.237387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226996+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:46:33.226996+0000","last_peered":"2024-09-21T01:46:33.226996+0000","last_clean":"2024-09-21T01:46:33.226996+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:46:33.226996+0000","last_undegraded":"2024-09-21T01:46:33.226996+0000","last_fullsized":"2024-09-21T01:46:33.226996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260670+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:46:33.260670+0000","last_peered":"2024-09-21T01:46:33.260670+0000","last_clean":"2024-09-21T01:46:33.260670+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:46:33.260670+0000","last_undegraded":"2024-09-21T01:46:33.260670+0000","last_fullsized":"2024-09-21T01:46:33.260670+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260568+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:46:33.260568+0000","last_peered":"2024-09-21T01:46:33.260568+0000","last_clean":"2024-09-21T01:46:33.260568+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:46:33.260568+0000","last_undegraded":"2024-09-21T01:46:33.260568+0000","last_fullsized":"2024-09-21T01:46:33.260568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227507+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:46:33.227507+0000","last_peered":"2024-09-21T01:46:33.227507+0000","last_clean":"2024-09-21T01:46:33.227507+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:46:33.227507+0000","last_undegraded":"2024-09-21T01:46:33.227507+0000","last_fullsized":"2024-09-21T01:46:33.227507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225461+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:46:33.225461+0000","last_peered":"2024-09-21T01:46:33.225461+0000","last_clean":"2024-09-21T01:46:33.225461+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:46:33.225461+0000","last_undegraded":"2024-09-21T01:46:33.225461+0000","last_fullsized":"2024-09-21T01:46:33.225461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226126+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:46:33.226126+0000","last_peered":"2024-09-21T01:46:33.226126+0000","last_clean":"2024-09-21T01:46:33.226126+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:46:33.226126+0000","last_undegraded":"2024-09-21T01:46:33.226126+0000","last_fullsized":"2024-09-21T01:46:33.226126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230980+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:46:33.230980+0000","last_peered":"2024-09-21T01:46:33.230980+0000","last_clean":"2024-09-21T01:46:33.230980+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:46:33.230980+0000","last_undegraded":"2024-09-21T01:46:33.230980+0000","last_fullsized":"2024-09-21T01:46:33.230980+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224911+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:46:33.224911+0000","last_peered":"2024-09-21T01:46:33.224911+0000","last_clean":"2024-09-21T01:46:33.224911+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:46:33.224911+0000","last_undegraded":"2024-09-21T01:46:33.224911+0000","last_fullsized":"2024-09-21T01:46:33.224911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227097+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:46:33.227097+0000","last_peered":"2024-09-21T01:46:33.227097+0000","last_clean":"2024-09-21T01:46:33.227097+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:46:33.227097+0000","last_undegraded":"2024-09-21T01:46:33.227097+0000","last_fullsized":"2024-09-21T01:46:33.227097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227796+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:46:33.227796+0000","last_peered":"2024-09-21T01:46:33.227796+0000","last_clean":"2024-09-21T01:46:33.227796+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:46:33.227796+0000","last_undegraded":"2024-09-21T01:46:33.227796+0000","last_fullsized":"2024-09-21T01:46:33.227796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223435+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:46:33.223435+0000","last_peered":"2024-09-21T01:46:33.223435+0000","last_clean":"2024-09-21T01:46:33.223435+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:46:33.223435+0000","last_undegraded":"2024-09-21T01:46:33.223435+0000","last_fullsized":"2024-09-21T01:46:33.223435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227979+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:46:33.227979+0000","last_peered":"2024-09-21T01:46:33.227979+0000","last_clean":"2024-09-21T01:46:33.227979+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:46:33.227979+0000","last_undegraded":"2024-09-21T01:46:33.227979+0000","last_fullsized":"2024-09-21T01:46:33.227979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224001+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:46:33.224001+0000","last_peered":"2024-09-21T01:46:33.224001+0000","last_clean":"2024-09-21T01:46:33.224001+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:46:33.224001+0000","last_undegraded":"2024-09-21T01:46:33.224001+0000","last_fullsized":"2024-09-21T01:46:33.224001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232919+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:46:33.232919+0000","last_peered":"2024-09-21T01:46:33.232919+0000","last_clean":"2024-09-21T01:46:33.232919+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:46:33.232919+0000","last_undegraded":"2024-09-21T01:46:33.232919+0000","last_fullsized":"2024-09-21T01:46:33.232919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224984+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:46:33.224984+0000","last_peered":"2024-09-21T01:46:33.224984+0000","last_clean":"2024-09-21T01:46:33.224984+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:46:33.224984+0000","last_undegraded":"2024-09-21T01:46:33.224984+0000","last_fullsized":"2024-09-21T01:46:33.224984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226981+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:46:33.226981+0000","last_peered":"2024-09-21T01:46:33.226981+0000","last_clean":"2024-09-21T01:46:33.226981+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:46:33.226981+0000","last_undegraded":"2024-09-21T01:46:33.226981+0000","last_fullsized":"2024-09-21T01:46:33.226981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260782+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:46:33.260782+0000","last_peered":"2024-09-21T01:46:33.260782+0000","last_clean":"2024-09-21T01:46:33.260782+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:46:33.260782+0000","last_undegraded":"2024-09-21T01:46:33.260782+0000","last_fullsized":"2024-09-21T01:46:33.260782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230552+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:46:33.230552+0000","last_peered":"2024-09-21T01:46:33.230552+0000","last_clean":"2024-09-21T01:46:33.230552+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:46:33.230552+0000","last_undegraded":"2024-09-21T01:46:33.230552+0000","last_fullsized":"2024-09-21T01:46:33.230552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230632+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:46:33.230632+0000","last_peered":"2024-09-21T01:46:33.230632+0000","last_clean":"2024-09-21T01:46:33.230632+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:46:33.230632+0000","last_undegraded":"2024-09-21T01:46:33.230632+0000","last_fullsized":"2024-09-21T01:46:33.230632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224723+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:46:33.224723+0000","last_peered":"2024-09-21T01:46:33.224723+0000","last_clean":"2024-09-21T01:46:33.224723+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:46:33.224723+0000","last_undegraded":"2024-09-21T01:46:33.224723+0000","last_fullsized":"2024-09-21T01:46:33.224723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228787+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:46:33.228787+0000","last_peered":"2024-09-21T01:46:33.228787+0000","last_clean":"2024-09-21T01:46:33.228787+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:46:33.228787+0000","last_undegraded":"2024-09-21T01:46:33.228787+0000","last_fullsized":"2024-09-21T01:46:33.228787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.238800+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:46:37.238800+0000","last_peered":"2024-09-21T01:46:37.238800+0000","last_clean":"2024-09-21T01:46:37.238800+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:46:37.238800+0000","last_undegraded":"2024-09-21T01:46:37.238800+0000","last_fullsized":"2024-09-21T01:46:37.238800+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228873+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:46:33.228873+0000","last_peered":"2024-09-21T01:46:33.228873+0000","last_clean":"2024-09-21T01:46:33.228873+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:46:33.228873+0000","last_undegraded":"2024-09-21T01:46:33.228873+0000","last_fullsized":"2024-09-21T01:46:33.228873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223419+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:46:33.223419+0000","last_peered":"2024-09-21T01:46:33.223419+0000","last_clean":"2024-09-21T01:46:33.223419+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:46:33.223419+0000","last_undegraded":"2024-09-21T01:46:33.223419+0000","last_fullsized":"2024-09-21T01:46:33.223419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225008+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:46:33.225008+0000","last_peered":"2024-09-21T01:46:33.225008+0000","last_clean":"2024-09-21T01:46:33.225008+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:46:33.225008+0000","last_undegraded":"2024-09-21T01:46:33.225008+0000","last_fullsized":"2024-09-21T01:46:33.225008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.239618+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:46:38.239618+0000","last_peered":"2024-09-21T01:46:38.239618+0000","last_clean":"2024-09-21T01:46:38.239618+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:46:38.239618+0000","last_undegraded":"2024-09-21T01:46:38.239618+0000","last_fullsized":"2024-09-21T01:46:38.239618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228320+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:46:33.228320+0000","last_peered":"2024-09-21T01:46:33.228320+0000","last_clean":"2024-09-21T01:46:33.228320+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:46:33.228320+0000","last_undegraded":"2024-09-21T01:46:33.228320+0000","last_fullsized":"2024-09-21T01:46:33.228320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260933+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:46:33.260933+0000","last_peered":"2024-09-21T01:46:33.260933+0000","last_clean":"2024-09-21T01:46:33.260933+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:46:33.260933+0000","last_undegraded":"2024-09-21T01:46:33.260933+0000","last_fullsized":"2024-09-21T01:46:33.260933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227879+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:46:33.227879+0000","last_peered":"2024-09-21T01:46:33.227879+0000","last_clean":"2024-09-21T01:46:33.227879+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:46:33.227879+0000","last_undegraded":"2024-09-21T01:46:33.227879+0000","last_fullsized":"2024-09-21T01:46:33.227879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":90,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.243460+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:46:38.243460+0000","last_peered":"2024-09-21T01:46:38.243460+0000","last_clean":"2024-09-21T01:46:38.243460+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:46:38.243460+0000","last_undegraded":"2024-09-21T01:46:38.243460+0000","last_fullsized":"2024-09-21T01:46:38.243460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54250,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":72,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229895+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:46:33.229895+0000","last_peered":"2024-09-21T01:46:33.229895+0000","last_clean":"2024-09-21T01:46:33.229895+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:46:33.229895+0000","last_undegraded":"2024-09-21T01:46:33.229895+0000","last_fullsized":"2024-09-21T01:46:33.229895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226066+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:46:33.226066+0000","last_peered":"2024-09-21T01:46:33.226066+0000","last_clean":"2024-09-21T01:46:33.226066+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:46:33.226066+0000","last_undegraded":"2024-09-21T01:46:33.226066+0000","last_fullsized":"2024-09-21T01:46:33.226066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232179+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:46:33.232179+0000","last_peered":"2024-09-21T01:46:33.232179+0000","last_clean":"2024-09-21T01:46:33.232179+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:46:33.232179+0000","last_undegraded":"2024-09-21T01:46:33.232179+0000","last_fullsized":"2024-09-21T01:46:33.232179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:36.241632+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:46:36.241632+0000","last_peered":"2024-09-21T01:46:36.241632+0000","last_clean":"2024-09-21T01:46:36.241632+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:46:36.241632+0000","last_undegraded":"2024-09-21T01:46:36.241632+0000","last_fullsized":"2024-09-21T01:46:36.241632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231720+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:46:33.231720+0000","last_peered":"2024-09-21T01:46:33.231720+0000","last_clean":"2024-09-21T01:46:33.231720+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:46:33.231720+0000","last_undegraded":"2024-09-21T01:46:33.231720+0000","last_fullsized":"2024-09-21T01:46:33.231720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227856+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:46:33.227856+0000","last_peered":"2024-09-21T01:46:33.227856+0000","last_clean":"2024-09-21T01:46:33.227856+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:46:33.227856+0000","last_undegraded":"2024-09-21T01:46:33.227856+0000","last_fullsized":"2024-09-21T01:46:33.227856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234181+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:46:33.234181+0000","last_peered":"2024-09-21T01:46:33.234181+0000","last_clean":"2024-09-21T01:46:33.234181+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:46:33.234181+0000","last_undegraded":"2024-09-21T01:46:33.234181+0000","last_fullsized":"2024-09-21T01:46:33.234181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231777+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:46:33.231777+0000","last_peered":"2024-09-21T01:46:33.231777+0000","last_clean":"2024-09-21T01:46:33.231777+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:46:33.231777+0000","last_undegraded":"2024-09-21T01:46:33.231777+0000","last_fullsized":"2024-09-21T01:46:33.231777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231571+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:46:33.231571+0000","last_peered":"2024-09-21T01:46:33.231571+0000","last_clean":"2024-09-21T01:46:33.231571+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:46:33.231571+0000","last_undegraded":"2024-09-21T01:46:33.231571+0000","last_fullsized":"2024-09-21T01:46:33.231571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:39.242156+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:46:39.242156+0000","last_peered":"2024-09-21T01:46:39.242156+0000","last_clean":"2024-09-21T01:46:39.242156+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:46:39.242156+0000","last_undegraded":"2024-09-21T01:46:39.242156+0000","last_fullsized":"2024-09-21T01:46:39.242156+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231672+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:46:33.231672+0000","last_peered":"2024-09-21T01:46:33.231672+0000","last_clean":"2024-09-21T01:46:33.231672+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:46:33.231672+0000","last_undegraded":"2024-09-21T01:46:33.231672+0000","last_fullsized":"2024-09-21T01:46:33.231672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231609+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231609+0000","last_peered":"2024-09-21T01:46:33.231609+0000","last_clean":"2024-09-21T01:46:33.231609+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231609+0000","last_undegraded":"2024-09-21T01:46:33.231609+0000","last_fullsized":"2024-09-21T01:46:33.231609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237072+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237072+0000","last_peered":"2024-09-21T01:46:33.237072+0000","last_clean":"2024-09-21T01:46:33.237072+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237072+0000","last_undegraded":"2024-09-21T01:46:33.237072+0000","last_fullsized":"2024-09-21T01:46:33.237072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223710+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:46:33.223710+0000","last_peered":"2024-09-21T01:46:33.223710+0000","last_clean":"2024-09-21T01:46:33.223710+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:46:33.223710+0000","last_undegraded":"2024-09-21T01:46:33.223710+0000","last_fullsized":"2024-09-21T01:46:33.223710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231445+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231445+0000","last_peered":"2024-09-21T01:46:33.231445+0000","last_clean":"2024-09-21T01:46:33.231445+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231445+0000","last_undegraded":"2024-09-21T01:46:33.231445+0000","last_fullsized":"2024-09-21T01:46:33.231445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226072+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:46:33.226072+0000","last_peered":"2024-09-21T01:46:33.226072+0000","last_clean":"2024-09-21T01:46:33.226072+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:46:33.226072+0000","last_undegraded":"2024-09-21T01:46:33.226072+0000","last_fullsized":"2024-09-21T01:46:33.226072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.242350+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:46:37.242350+0000","last_peered":"2024-09-21T01:46:37.242350+0000","last_clean":"2024-09-21T01:46:37.242350+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:46:37.242350+0000","last_undegraded":"2024-09-21T01:46:37.242350+0000","last_fullsized":"2024-09-21T01:46:37.242350+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237841+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:46:33.237841+0000","last_peered":"2024-09-21T01:46:33.237841+0000","last_clean":"2024-09-21T01:46:33.237841+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:46:33.237841+0000","last_undegraded":"2024-09-21T01:46:33.237841+0000","last_fullsized":"2024-09-21T01:46:33.237841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"74'6","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:38.239429+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:46:38.239429+0000","last_peered":"2024-09-21T01:46:38.239429+0000","last_clean":"2024-09-21T01:46:38.239429+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:46:38.239429+0000","last_undegraded":"2024-09-21T01:46:38.239429+0000","last_fullsized":"2024-09-21T01:46:38.239429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225616+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:46:33.225616+0000","last_peered":"2024-09-21T01:46:33.225616+0000","last_clean":"2024-09-21T01:46:33.225616+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:46:33.225616+0000","last_undegraded":"2024-09-21T01:46:33.225616+0000","last_fullsized":"2024-09-21T01:46:33.225616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225703+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225703+0000","last_peered":"2024-09-21T01:46:33.225703+0000","last_clean":"2024-09-21T01:46:33.225703+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225703+0000","last_undegraded":"2024-09-21T01:46:33.225703+0000","last_fullsized":"2024-09-21T01:46:33.225703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225654+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:46:33.225654+0000","last_peered":"2024-09-21T01:46:33.225654+0000","last_clean":"2024-09-21T01:46:33.225654+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:46:33.225654+0000","last_undegraded":"2024-09-21T01:46:33.225654+0000","last_fullsized":"2024-09-21T01:46:33.225654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230530+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230530+0000","last_peered":"2024-09-21T01:46:33.230530+0000","last_clean":"2024-09-21T01:46:33.230530+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230530+0000","last_undegraded":"2024-09-21T01:46:33.230530+0000","last_fullsized":"2024-09-21T01:46:33.230530+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227173+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:46:33.227173+0000","last_peered":"2024-09-21T01:46:33.227173+0000","last_clean":"2024-09-21T01:46:33.227173+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:46:33.227173+0000","last_undegraded":"2024-09-21T01:46:33.227173+0000","last_fullsized":"2024-09-21T01:46:33.227173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:35.858088+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:35.858088+0000","last_peered":"2024-09-21T01:46:35.858088+0000","last_clean":"2024-09-21T01:46:35.858088+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:35.858088+0000","last_undegraded":"2024-09-21T01:46:35.858088+0000","last_fullsized":"2024-09-21T01:46:35.858088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224103+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:46:33.224103+0000","last_peered":"2024-09-21T01:46:33.224103+0000","last_clean":"2024-09-21T01:46:33.224103+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:46:33.224103+0000","last_undegraded":"2024-09-21T01:46:33.224103+0000","last_fullsized":"2024-09-21T01:46:33.224103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237584+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237584+0000","last_peered":"2024-09-21T01:46:33.237584+0000","last_clean":"2024-09-21T01:46:33.237584+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237584+0000","last_undegraded":"2024-09-21T01:46:33.237584+0000","last_fullsized":"2024-09-21T01:46:33.237584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260770+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:46:33.260770+0000","last_peered":"2024-09-21T01:46:33.260770+0000","last_clean":"2024-09-21T01:46:33.260770+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:46:33.260770+0000","last_undegraded":"2024-09-21T01:46:33.260770+0000","last_fullsized":"2024-09-21T01:46:33.260770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228572+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:46:33.228572+0000","last_peered":"2024-09-21T01:46:33.228572+0000","last_clean":"2024-09-21T01:46:33.228572+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:46:33.228572+0000","last_undegraded":"2024-09-21T01:46:33.228572+0000","last_fullsized":"2024-09-21T01:46:33.228572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224910+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:46:33.224910+0000","last_peered":"2024-09-21T01:46:33.224910+0000","last_clean":"2024-09-21T01:46:33.224910+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:46:33.224910+0000","last_undegraded":"2024-09-21T01:46:33.224910+0000","last_fullsized":"2024-09-21T01:46:33.224910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223770+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:46:33.223770+0000","last_peered":"2024-09-21T01:46:33.223770+0000","last_clean":"2024-09-21T01:46:33.223770+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:46:33.223770+0000","last_undegraded":"2024-09-21T01:46:33.223770+0000","last_fullsized":"2024-09-21T01:46:33.223770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231959+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:46:33.231959+0000","last_peered":"2024-09-21T01:46:33.231959+0000","last_clean":"2024-09-21T01:46:33.231959+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:46:33.231959+0000","last_undegraded":"2024-09-21T01:46:33.231959+0000","last_fullsized":"2024-09-21T01:46:33.231959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227958+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:46:33.227958+0000","last_peered":"2024-09-21T01:46:33.227958+0000","last_clean":"2024-09-21T01:46:33.227958+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:46:33.227958+0000","last_undegraded":"2024-09-21T01:46:33.227958+0000","last_fullsized":"2024-09-21T01:46:33.227958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227724+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:46:33.227724+0000","last_peered":"2024-09-21T01:46:33.227724+0000","last_clean":"2024-09-21T01:46:33.227724+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:46:33.227724+0000","last_undegraded":"2024-09-21T01:46:33.227724+0000","last_fullsized":"2024-09-21T01:46:33.227724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260935+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:46:33.260935+0000","last_peered":"2024-09-21T01:46:33.260935+0000","last_clean":"2024-09-21T01:46:33.260935+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:46:33.260935+0000","last_undegraded":"2024-09-21T01:46:33.260935+0000","last_fullsized":"2024-09-21T01:46:33.260935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224309+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224309+0000","last_peered":"2024-09-21T01:46:33.224309+0000","last_clean":"2024-09-21T01:46:33.224309+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224309+0000","last_undegraded":"2024-09-21T01:46:33.224309+0000","last_fullsized":"2024-09-21T01:46:33.224309+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224094+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:46:33.224094+0000","last_peered":"2024-09-21T01:46:33.224094+0000","last_clean":"2024-09-21T01:46:33.224094+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:46:33.224094+0000","last_undegraded":"2024-09-21T01:46:33.224094+0000","last_fullsized":"2024-09-21T01:46:33.224094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:37.237058+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:46:37.237058+0000","last_peered":"2024-09-21T01:46:37.237058+0000","last_clean":"2024-09-21T01:46:37.237058+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:46:37.237058+0000","last_undegraded":"2024-09-21T01:46:37.237058+0000","last_fullsized":"2024-09-21T01:46:37.237058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":208,"num_read_kb":170,"num_write":234,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":187446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776148,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1044083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266723,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":991630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":398842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822706,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32616,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712536,"statfs":{"total":95995035648,"available":95961636864,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215164,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":458091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":454005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:46:43.657 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T01:46:43.658 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T01:46:43.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T01:46:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:43 smithi190 ceph-mon[34368]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s; 58 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:43 smithi190 ceph-mon[34368]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:46:43.918 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:44.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.476+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/995366756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 msgr2=0x7fc2f8103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:44.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.476+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/995366756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f8103290 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a480 comp rx=0 tx=0).stop 2024-09-21T01:46:44.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/995366756 shutdown_connections 2024-09-21T01:46:44.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/995366756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/995366756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f8103290 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/995366756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f8102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/995366756 >> 172.21.15.29:0/995366756 conn(0x7fc2f80fd760 msgr2=0x7fc2f80ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/995366756 shutdown_connections 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/995366756 wait complete. 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.477+0000 7fc2fe0fc640 1 Processor -- start 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 -- start start 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f810b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f810e2a0 con 0x7fc2f8102e80 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f810e120 con 0x7fc2f8103820 2024-09-21T01:46:44.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2fe0fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f810e420 con 0x7fc2f8101df0 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36618/0 (socket says 172.21.15.29:36618) 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/1795238550 learned_addr learned my addr 172.21.15.29:0/1795238550 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f77fe640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f810b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f7fff640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/1795238550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 msgr2=0x7fc2f810b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f810b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/1795238550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 msgr2=0x7fc2f81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/1795238550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f81a1aa0 con 0x7fc2f8102e80 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f77fe640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f810b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.478+0000 7fc2f7fff640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec006010 tx=0x7fc2ec004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:44.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec03f030 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f81a1d30 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec003cb0 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f81a21b0 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.479+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec035070 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.480+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f8108fd0 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.482+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc2ec042030 con 0x7fc2f8102e80 2024-09-21T01:46:44.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.482+0000 7fc2f4ff9640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 0x7fc2e00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:44.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.483+0000 7fc2f77fe640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 0x7fc2e00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:44.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.483+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2ec108290 con 0x7fc2f8102e80 2024-09-21T01:46:44.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.483+0000 7fc2f77fe640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 0x7fc2e00b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80058c0 tx=0x7fc2e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:44.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.485+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2ec0ce9d0 con 0x7fc2f8102e80 2024-09-21T01:46:44.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.744+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc2f8063fc0 con 0x7fc2f8102e80 2024-09-21T01:46:44.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.748+0000 7fc2f4ff9640 1 -- 172.21.15.29:0/1795238550 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc2ec0d2db0 con 0x7fc2f8102e80 2024-09-21T01:46:44.749 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-09-21T01:46:44.749 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCEJe5mYNmHLBAAFM+YJk0UO/JehemiD1pyHA== 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 msgr2=0x7fc2e00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 0x7fc2e00b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80058c0 tx=0x7fc2e8005850 comp rx=0 tx=0).stop 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 msgr2=0x7fc2f819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec006010 tx=0x7fc2ec004340 comp rx=0 tx=0).stop 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 shutdown_connections 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2e00b65a0 0x7fc2e00b8ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2f8103820 0x7fc2f81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f8102e80 0x7fc2f819ed50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 --2- 172.21.15.29:0/1795238550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f8101df0 0x7fc2f810b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:44.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 >> 172.21.15.29:0/1795238550 conn(0x7fc2f80fd760 msgr2=0x7fc2f81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:44.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 shutdown_connections 2024-09-21T01:46:44.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:44.751+0000 7fc2fe0fc640 1 -- 172.21.15.29:0/1795238550 wait complete. 2024-09-21T01:46:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:45 smithi066 ceph-mon[35001]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 15 op/s; 53 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:45 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:46:45.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:45 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:46:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:45 smithi029 ceph-mon[31228]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 15 op/s; 53 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:45 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:46:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:45 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:46:45.668 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:46:45.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T01:46:45.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T01:46:45.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T01:46:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:45 smithi190 ceph-mon[34368]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 15 op/s; 53 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T01:46:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:45 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:46:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:45 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1795238550' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:46:46.022 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 -- 172.21.15.29:0/2147950112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c1094a0 msgr2=0x7f885c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 --2- 172.21.15.29:0/2147950112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c1094a0 0x7f885c1098b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8850009900 tx=0x7f885002a410 comp rx=0 tx=0).stop 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 -- 172.21.15.29:0/2147950112 shutdown_connections 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 --2- 172.21.15.29:0/2147950112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c109df0 0x7f885c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 --2- 172.21.15.29:0/2147950112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c1094a0 0x7f885c1098b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 --2- 172.21.15.29:0/2147950112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c103b60 0x7f885c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 -- 172.21.15.29:0/2147950112 >> 172.21.15.29:0/2147950112 conn(0x7f885c07b850 msgr2=0x7f885c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 -- 172.21.15.29:0/2147950112 shutdown_connections 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 -- 172.21.15.29:0/2147950112 wait complete. 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.513+0000 7f8862a85640 1 Processor -- start 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 -- start start 2024-09-21T01:46:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c103b60 0x7f885c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 0x7f885c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113ba0 con 0x7f885c109df0 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113a20 con 0x7f885c1094a0 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8862a85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113d20 con 0x7f885c103b60 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36650/0 (socket says 172.21.15.29:36650) 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 -- 172.21.15.29:0/1842848490 learned_addr learned my addr 172.21.15.29:0/1842848490 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8860821640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c103b60 0x7f885c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f885bfff640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 0x7f885c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 -- 172.21.15.29:0/1842848490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c103b60 msgr2=0x7f885c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c103b60 0x7f885c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 -- 172.21.15.29:0/1842848490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 msgr2=0x7f885c19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 0x7f885c19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f8861022640 1 -- 172.21.15.29:0/1842848490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1a05e0 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.514+0000 7f885bfff640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 0x7f885c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8861022640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f885400b970 tx=0x7f885400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854019070 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a08d0 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8854014440 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540075d0 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.515+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a7030 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.516+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c103ff0 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.517+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f885401c070 con 0x7f885c109df0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.518+0000 7f8859ffb640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 0x7f88440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.518+0000 7f8860821640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 0x7f88440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:46.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.518+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f88540e6890 con 0x7f885c109df0 2024-09-21T01:46:46.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.519+0000 7f8860821640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 0x7f88440b8b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f884c009990 tx=0x7f884c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:46.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.522+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88540acfd0 con 0x7f885c109df0 2024-09-21T01:46:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.714+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f885c10b630 con 0x7f885c109df0 2024-09-21T01:46:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.714+0000 7f8859ffb640 1 -- 172.21.15.29:0/1842848490 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v54) ==== 86+0+133 (secure 0 0 0) 0x7f88540b13b0 con 0x7f885c109df0 2024-09-21T01:46:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:46.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 msgr2=0x7f88440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:46.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 0x7f88440b8b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f884c009990 tx=0x7f884c009110 comp rx=0 tx=0).stop 2024-09-21T01:46:46.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 msgr2=0x7f885c19feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:46.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f885400b970 tx=0x7f885400bd70 comp rx=0 tx=0).stop 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 shutdown_connections 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88440b6620 0x7f88440b8b30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f885c109df0 0x7f885c19feb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c1094a0 0x7f885c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 --2- 172.21.15.29:0/1842848490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f885c103b60 0x7f885c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 >> 172.21.15.29:0/1842848490 conn(0x7f885c07b850 msgr2=0x7f885c107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 shutdown_connections 2024-09-21T01:46:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:46.717+0000 7f8862a85640 1 -- 172.21.15.29:0/1842848490 wait complete. 2024-09-21T01:46:47.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:47 smithi190 ceph-mon[34368]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T01:46:47.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:47 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1842848490' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:46:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:47 smithi066 ceph-mon[35001]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T01:46:47.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:47 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1842848490' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:46:47.556 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:47 smithi029 ceph-mon[31228]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T01:46:47.556 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:47 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1842848490' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:46:47.589 INFO:teuthology.orchestra.run.smithi029.stdout:[{"version":54,"timestamp":"2024-09-21T01:46:00.323178+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T01:46:47.589 INFO:tasks.ceph_manager:config epoch is 54 2024-09-21T01:46:47.589 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T01:46:47.589 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T01:46:47.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T01:46:47.853 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- 172.21.15.29:0/2521778090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c103b60 msgr2=0x7f797c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- 172.21.15.29:0/2521778090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c103b60 0x7f797c103f70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f796c009ca0 tx=0x7f796c02a8e0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- 172.21.15.29:0/2521778090 shutdown_connections 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- 172.21.15.29:0/2521778090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 0x7f797c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- 172.21.15.29:0/2521778090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c1094a0 0x7f797c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- 172.21.15.29:0/2521778090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c103b60 0x7f797c103f70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- 172.21.15.29:0/2521778090 >> 172.21.15.29:0/2521778090 conn(0x7f797c07b850 msgr2=0x7f797c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- 172.21.15.29:0/2521778090 shutdown_connections 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- 172.21.15.29:0/2521778090 wait complete. 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 Processor -- start 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- start start 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c103b60 0x7f797c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 0x7f797c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f797c113b70 con 0x7f797c1094a0 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1139f0 con 0x7f797c109df0 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7983553640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f797c113cf0 con 0x7f797c103b60 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7980aee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7980aee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36674/0 (socket says 172.21.15.29:36674) 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7980aee640 1 -- 172.21.15.29:0/1476641658 learned_addr learned my addr 172.21.15.29:0/1476641658 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f79812ef640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c103b60 0x7f797c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.372+0000 7f7981af0640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 0x7f797c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 -- 172.21.15.29:0/1476641658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c103b60 msgr2=0x7f797c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c103b60 0x7f797c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 -- 172.21.15.29:0/1476641658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 msgr2=0x7f797c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 0x7f797c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 -- 172.21.15.29:0/1476641658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c1a8880 con 0x7f797c1094a0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7980aee640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7970006fd0 tx=0x7f797000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970011830 con 0x7f797c1094a0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c1a8b70 con 0x7f797c1094a0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7970016070 con 0x7f797c1094a0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970010e00 con 0x7f797c1094a0 2024-09-21T01:46:48.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.373+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1a9060 con 0x7f797c1094a0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.374+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c103ff0 con 0x7f797c1094a0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.377+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f79700119d0 con 0x7f797c1094a0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.378+0000 7f797a7fc640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 0x7f79640b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.378+0000 7f79812ef640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 0x7f79640b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.378+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f79700e6fe0 con 0x7f797c1094a0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.378+0000 7f79812ef640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 0x7f79640b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098d0 tx=0x7f796c009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.380+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79700ad720 con 0x7f797c1094a0 2024-09-21T01:46:48.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.620+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f797c076a10 con 0x7f797c1094a0 2024-09-21T01:46:48.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.625+0000 7f797a7fc640 1 -- 172.21.15.29:0/1476641658 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+296434 (secure 0 0 0) 0x7f797c076a10 con 0x7f797c1094a0 2024-09-21T01:46:48.629 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:48.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 msgr2=0x7f79640b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:48.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 0x7f79640b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098d0 tx=0x7f796c009940 comp rx=0 tx=0).stop 2024-09-21T01:46:48.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 msgr2=0x7f797c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:48.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7970006fd0 tx=0x7f797000b8e0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 shutdown_connections 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79640b65a0 0x7f79640b8ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f797c109df0 0x7f797c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f797c1094a0 0x7f797c075c50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 --2- 172.21.15.29:0/1476641658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c103b60 0x7f797c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 >> 172.21.15.29:0/1476641658 conn(0x7f797c07b850 msgr2=0x7f797c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 shutdown_connections 2024-09-21T01:46:48.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:48.628+0000 7f7983553640 1 -- 172.21.15.29:0/1476641658 wait complete. 2024-09-21T01:46:49.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:49 smithi029 ceph-mon[31228]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-21T01:46:49.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:49 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1476641658' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:46:49.429 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":796732990},{"type":"v1","addr":"172.21.15.29:6801","nonce":796732990}]},"active_addr":"172.21.15.29:6801/796732990","active_change":"2024-09-21T01:38:31.990721+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24125,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.29:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.29:0","nonce":1262382996}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.29:0","nonce":516499598}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.29:0","nonce":1821711658}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.29:0","nonce":3738586778}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.29:0","nonce":2649307272}]}]} 2024-09-21T01:46:49.435 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T01:46:49.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T01:46:49.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:46:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:49 smithi066 ceph-mon[35001]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-21T01:46:49.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:49 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1476641658' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:46:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:49 smithi190 ceph-mon[34368]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-21T01:46:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:49 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1476641658' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 -- 172.21.15.29:0/1780518572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec108d00 msgr2=0x7fc6ec10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 --2- 172.21.15.29:0/1780518572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec108d00 0x7fc6ec10f530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009ca0 tx=0x7fc6e402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 -- 172.21.15.29:0/1780518572 shutdown_connections 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 --2- 172.21.15.29:0/1780518572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec108d00 0x7fc6ec10f530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 --2- 172.21.15.29:0/1780518572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 0x7fc6ec1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 --2- 172.21.15.29:0/1780518572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec102a70 0x7fc6ec102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 -- 172.21.15.29:0/1780518572 >> 172.21.15.29:0/1780518572 conn(0x7fc6ec079710 msgr2=0x7fc6ec079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:50.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 -- 172.21.15.29:0/1780518572 shutdown_connections 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.234+0000 7fc6f1293640 1 -- 172.21.15.29:0/1780518572 wait complete. 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.235+0000 7fc6f1293640 1 Processor -- start 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.235+0000 7fc6f1293640 1 -- start start 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 0x7fc6ec0ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 0x7fc6ec100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1129c0 con 0x7fc6ec102a70 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec112840 con 0x7fc6ec108d00 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6f1293640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec112b40 con 0x7fc6ec1083b0 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:50.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36684/0 (socket says 172.21.15.29:36684) 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 -- 172.21.15.29:0/3681317942 learned_addr learned my addr 172.21.15.29:0/3681317942 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6ea7fc640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 0x7fc6ec0ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eb7fe640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 0x7fc6ec100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 -- 172.21.15.29:0/3681317942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 msgr2=0x7fc6ec0ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 0x7fc6ec0ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 -- 172.21.15.29:0/3681317942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 msgr2=0x7fc6ec100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 0x7fc6ec100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 -- 172.21.15.29:0/3681317942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec1a86e0 con 0x7fc6ec102a70 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eb7fe640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 0x7fc6ec100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6eaffd640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c890 tx=0x7fc6dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.236+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004220 con 0x7fc6ec102a70 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.237+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec1a89d0 con 0x7fc6ec102a70 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.237+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc0043c0 con 0x7fc6ec102a70 2024-09-21T01:46:50.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.237+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004a00 con 0x7fc6ec102a70 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.237+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec1a8f10 con 0x7fc6ec102a70 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.238+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec103cf0 con 0x7fc6ec102a70 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.239+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc6dc0079b0 con 0x7fc6ec102a70 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.240+0000 7fc6d3fff640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 0x7fc6d40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.240+0000 7fc6ea7fc640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 0x7fc6d40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.240+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc6dc0e7600 con 0x7fc6ec102a70 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.241+0000 7fc6ea7fc640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 0x7fc6d40b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec06ca10 tx=0x7fc6e000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:50.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.243+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6dc014030 con 0x7fc6ec102a70 2024-09-21T01:46:50.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.437+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6ec1008f0 con 0x7fc6ec102a70 2024-09-21T01:46:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.439+0000 7fc6d3fff640 1 -- 172.21.15.29:0/3681317942 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7fc6dc0add40 con 0x7fc6ec102a70 2024-09-21T01:46:50.439 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:50.439 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.441+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 msgr2=0x7fc6d40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.441+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 0x7fc6d40b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec06ca10 tx=0x7fc6e000a2d0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.441+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 msgr2=0x7fc6ec0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.441+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c890 tx=0x7fc6dc00cd50 comp rx=0 tx=0).stop 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 shutdown_connections 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6d40b65a0 0x7fc6d40b8ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108d00 0x7fc6ec100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6ec1083b0 0x7fc6ec0ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 --2- 172.21.15.29:0/3681317942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec102a70 0x7fc6ec0ff5f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:50.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 >> 172.21.15.29:0/3681317942 conn(0x7fc6ec079710 msgr2=0x7fc6ec1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:50.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 shutdown_connections 2024-09-21T01:46:50.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:50.442+0000 7fc6f1293640 1 -- 172.21.15.29:0/3681317942 wait complete. 2024-09-21T01:46:51.311 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T01:46:51.311 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:46:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:51 smithi066 ceph-mon[35001]: pgmap v315: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-21T01:46:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:51 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3681317942' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:51.571 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:51.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:51 smithi029 ceph-mon[31228]: pgmap v315: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-21T01:46:51.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:51 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3681317942' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:51 smithi190 ceph-mon[34368]: pgmap v315: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-21T01:46:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:51 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3681317942' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 -- 172.21.15.29:0/3531157948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b81094a0 msgr2=0x7f18b81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3531157948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b81094a0 0x7f18b81098b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009900 tx=0x7f18ac02a410 comp rx=0 tx=0).stop 2024-09-21T01:46:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 -- 172.21.15.29:0/3531157948 shutdown_connections 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3531157948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3531157948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b81094a0 0x7f18b81098b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3531157948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8103b60 0x7f18b8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 -- 172.21.15.29:0/3531157948 >> 172.21.15.29:0/3531157948 conn(0x7f18b807b850 msgr2=0x7f18b807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 -- 172.21.15.29:0/3531157948 shutdown_connections 2024-09-21T01:46:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.087+0000 7f18bef8a640 1 -- 172.21.15.29:0/3531157948 wait complete. 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 Processor -- start 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 -- start start 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b81094a0 0x7f18b819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8113b00 con 0x7f18b8103b60 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8113980 con 0x7f18b8109df0 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bef8a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8113c80 con 0x7f18b81094a0 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bcd26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bcd26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33176/0 (socket says 172.21.15.29:33176) 2024-09-21T01:46:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bcd26640 1 -- 172.21.15.29:0/3765132451 learned_addr learned my addr 172.21.15.29:0/3765132451 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18b7fff640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b81094a0 0x7f18b819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.088+0000 7f18bd527640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 -- 172.21.15.29:0/3765132451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b81094a0 msgr2=0x7f18b819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b81094a0 0x7f18b819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 -- 172.21.15.29:0/3765132451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 msgr2=0x7f18b819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 -- 172.21.15.29:0/3765132451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81a0380 con 0x7f18b8103b60 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bd527640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bcd26640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c920 tx=0x7f18a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007c70 con 0x7f18b8103b60 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8007e10 con 0x7f18b8103b60 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a0670 con 0x7f18b8103b60 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8018820 con 0x7f18b8103b60 2024-09-21T01:46:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.089+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a6e90 con 0x7f18b8103b60 2024-09-21T01:46:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.090+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8103f70 con 0x7f18b8103b60 2024-09-21T01:46:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.092+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f18a800f450 con 0x7f18b8103b60 2024-09-21T01:46:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.092+0000 7f18b5ffb640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 0x7f18a00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.093+0000 7f18b7fff640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 0x7f18a00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.093+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f18a80e75c0 con 0x7f18b8103b60 2024-09-21T01:46:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.093+0000 7f18b7fff640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 0x7f18a00b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02aa00 tx=0x7f18ac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.097+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18a8014030 con 0x7f18b8103b60 2024-09-21T01:46:52.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.287+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18b810bc10 con 0x7f18b8103b60 2024-09-21T01:46:52.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.289+0000 7f18b5ffb640 1 -- 172.21.15.29:0/3765132451 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7f18b810bc10 con 0x7f18b8103b60 2024-09-21T01:46:52.289 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:46:52.289 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:46:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 msgr2=0x7f18a00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 0x7f18a00b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02aa00 tx=0x7f18ac005a20 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 msgr2=0x7f18b819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c920 tx=0x7f18a800cde0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 shutdown_connections 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a00b65d0 0x7f18a00b8ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8109df0 0x7f18b819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b81094a0 0x7f18b819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 --2- 172.21.15.29:0/3765132451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8103b60 0x7f18b819a4d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 >> 172.21.15.29:0/3765132451 conn(0x7f18b807b850 msgr2=0x7f18b8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 shutdown_connections 2024-09-21T01:46:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:52.291+0000 7f18bef8a640 1 -- 172.21.15.29:0/3765132451 wait complete. 2024-09-21T01:46:53.108 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T01:46:53.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T01:46:53.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T01:46:53.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T01:46:53.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T01:46:53.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T01:46:53.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T01:46:53.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T01:46:53.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T01:46:53.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T01:46:53.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T01:46:53.113 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T01:46:53.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:53 smithi029 ceph-mon[31228]: pgmap v316: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-21T01:46:53.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:53 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3765132451' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:53 smithi066 ceph-mon[35001]: pgmap v316: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-21T01:46:53.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:53 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3765132451' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:53 smithi190 ceph-mon[34368]: pgmap v316: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-21T01:46:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:53 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3765132451' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:46:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.910 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.915 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.915 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.924 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.925 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.928 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:55.129 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:46:55.353 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:55 smithi029 ceph-mon[31228]: pgmap v317: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 -- 172.21.15.29:0/1119856842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 msgr2=0x7f4d6c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 --2- 172.21.15.29:0/1119856842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c1066e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009900 tx=0x7f4d5402a410 comp rx=0 tx=0).stop 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 -- 172.21.15.29:0/1119856842 shutdown_connections 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 --2- 172.21.15.29:0/1119856842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d6c106c20 0x7f4d6c110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 --2- 172.21.15.29:0/1119856842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c1066e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 --2- 172.21.15.29:0/1119856842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 0x7f4d6c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 -- 172.21.15.29:0/1119856842 >> 172.21.15.29:0/1119856842 conn(0x7f4d6c077cc0 msgr2=0x7f4d6c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 -- 172.21.15.29:0/1119856842 shutdown_connections 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.331+0000 7f4d714bb640 1 -- 172.21.15.29:0/1119856842 wait complete. 2024-09-21T01:46:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 Processor -- start 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 -- start start 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 0x7f4d6c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d6c106c20 0x7f4d6c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113ad0 con 0x7f4d6c1062d0 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113950 con 0x7f4d6c102080 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d714bb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113c50 con 0x7f4d6c106c20 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33214/0 (socket says 172.21.15.29:33214) 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d6a7fc640 1 -- 172.21.15.29:0/3387961662 learned_addr learned my addr 172.21.15.29:0/3387961662 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d6affd640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 0x7f4d6c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.332+0000 7f4d6b7fe640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d6c106c20 0x7f4d6c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 -- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d6c106c20 msgr2=0x7f4d6c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d6c106c20 0x7f4d6c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 -- 172.21.15.29:0/3387961662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 msgr2=0x7f4d6c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 0x7f4d6c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a0370 con 0x7f4d6c1062d0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6affd640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c102080 0x7f4d6c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d6a7fc640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c19aa60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005940 tx=0x7f4d540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5403f030 con 0x7f4d6c1062d0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a0600 con 0x7f4d6c1062d0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d54003cb0 con 0x7f4d6c1062d0 2024-09-21T01:46:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54035070 con 0x7f4d6c1062d0 2024-09-21T01:46:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.333+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a0ae0 con 0x7f4d6c1062d0 2024-09-21T01:46:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.334+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d6c063b50 con 0x7f4d6c1062d0 2024-09-21T01:46:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.336+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4d54042030 con 0x7f4d6c1062d0 2024-09-21T01:46:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.336+0000 7f4d63fff640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4d580b65d0 0x7f4d580b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.336+0000 7f4d6affd640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4d580b65d0 0x7f4d580b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.336+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4d54108130 con 0x7f4d6c1062d0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d63fff640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] conn(0x7f4d580c00c0 0x7f4d580c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d580c2c80 con 0x7f4d580c00c0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4d54108580 con 0x7f4d6c1062d0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d6affd640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4d580b65d0 0x7f4d580b8ae0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005d00 tx=0x7f4d64005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d6b7fe640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] conn(0x7f4d580c00c0 0x7f4d580c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.337+0000 7f4d6b7fe640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] conn(0x7f4d580c00c0 0x7f4d580c25b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.338+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== osd.8 v2:172.21.15.190:6816/1425209392 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d580c2c80 con 0x7f4d580c00c0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.352+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 --> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d6c063f50 con 0x7f4d580c00c0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d63fff640 1 -- 172.21.15.29:0/3387961662 <== osd.8 v2:172.21.15.190:6816/1425209392 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d6c063f50 con 0x7f4d580c00c0 2024-09-21T01:46:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] conn(0x7f4d580c00c0 msgr2=0x7f4d580c25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] conn(0x7f4d580c00c0 0x7f4d580c25b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4d580b65d0 msgr2=0x7f4d580b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4d580b65d0 0x7f4d580b8ae0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005d00 tx=0x7f4d64005c90 comp rx=0 tx=0).stop 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 msgr2=0x7f4d6c19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 --2- 172.21.15.29:0/3387961662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d6c1062d0 0x7f4d6c19aa60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005940 tx=0x7f4d540042f0 comp rx=0 tx=0).stop 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d6b7fe640 1 -- 172.21.15.29:0/3387961662 reap_dead start 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 shutdown_connections 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 >> 172.21.15.29:0/3387961662 conn(0x7f4d6c077cc0 msgr2=0x7f4d6c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 shutdown_connections 2024-09-21T01:46:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:55.353+0000 7f4d714bb640 1 -- 172.21.15.29:0/3387961662 wait complete. 2024-09-21T01:46:55.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:55 smithi066 ceph-mon[35001]: pgmap v317: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:46:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:55 smithi190 ceph-mon[34368]: pgmap v317: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:46:57.250 INFO:teuthology.orchestra.run.smithi029.stdout:227633266727 2024-09-21T01:46:57.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T01:46:57.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 -- 172.21.15.29:0/1594076852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 msgr2=0x7fc65c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 --2- 172.21.15.29:0/1594076852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c1087e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc650009900 tx=0x7fc65002a410 comp rx=0 tx=0).stop 2024-09-21T01:46:57.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 -- 172.21.15.29:0/1594076852 shutdown_connections 2024-09-21T01:46:57.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 --2- 172.21.15.29:0/1594076852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 0x7fc65c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 --2- 172.21.15.29:0/1594076852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c1087e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 --2- 172.21.15.29:0/1594076852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc65c102ae0 0x7fc65c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 -- 172.21.15.29:0/1594076852 >> 172.21.15.29:0/1594076852 conn(0x7fc65c079760 msgr2=0x7fc65c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 -- 172.21.15.29:0/1594076852 shutdown_connections 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.680+0000 7fc6619ef640 1 -- 172.21.15.29:0/1594076852 wait complete. 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 Processor -- start 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 -- start start 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc65c102ae0 0x7fc65c105110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c105650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 0x7fc65c105b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1129f0 con 0x7fc65c1083d0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c112870 con 0x7fc65c102ae0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc6619ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c112b70 con 0x7fc65c108d20 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc65a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc65a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33234/0 (socket says 172.21.15.29:33234) 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.681+0000 7fc65a7fc640 1 -- 172.21.15.29:0/3968321403 learned_addr learned my addr 172.21.15.29:0/3968321403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65affd640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc65c102ae0 0x7fc65c105110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65b7fe640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 0x7fc65c105b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 -- 172.21.15.29:0/3968321403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 msgr2=0x7fc65c105b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 0x7fc65c105b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 -- 172.21.15.29:0/3968321403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc65c102ae0 msgr2=0x7fc65c105110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc65c102ae0 0x7fc65c105110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c1a87e0 con 0x7fc65c1083d0 2024-09-21T01:46:57.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65b7fe640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc65c108d20 0x7fc65c105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc65a7fc640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c105650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc650031a90 tx=0x7fc650004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65003f030 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc650003cb0 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1a8a70 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650035070 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.682+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a8fd0 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.683+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc65c0ffc20 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.685+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc650042030 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.685+0000 7fc6609ed640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6440b65a0 0x7fc6440b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc65affd640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6440b65a0 0x7fc6440b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc650107f90 con 0x7fc65c1083d0 2024-09-21T01:46:57.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc6609ed640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] conn(0x7fc6440c0090 0x7fc6440c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc650005640 con 0x7fc6440c0090 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc6501083e0 con 0x7fc65c1083d0 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc65b7fe640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] conn(0x7fc6440c0090 0x7fc6440c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc65affd640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6440b65a0 0x7fc6440b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc65c106b60 tx=0x7fc64c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.686+0000 7fc65b7fe640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] conn(0x7fc6440c0090 0x7fc6440c2580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.687+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== osd.6 v2:172.21.15.29:6818/797110744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc650005640 con 0x7fc6440c0090 2024-09-21T01:46:57.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 --> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc65c06b350 con 0x7fc6440c0090 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6609ed640 1 -- 172.21.15.29:0/3968321403 <== osd.6 v2:172.21.15.29:6818/797110744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc65c06b350 con 0x7fc6440c0090 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] conn(0x7fc6440c0090 msgr2=0x7fc6440c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] conn(0x7fc6440c0090 0x7fc6440c2580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6440b65a0 msgr2=0x7fc6440b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6440b65a0 0x7fc6440b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc65c106b60 tx=0x7fc64c008040 comp rx=0 tx=0).stop 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 msgr2=0x7fc65c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 --2- 172.21.15.29:0/3968321403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc65c1083d0 0x7fc65c105650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc650031a90 tx=0x7fc650004270 comp rx=0 tx=0).stop 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc65b7fe640 1 -- 172.21.15.29:0/3968321403 reap_dead start 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 shutdown_connections 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.701+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 >> 172.21.15.29:0/3968321403 conn(0x7fc65c079760 msgr2=0x7fc65c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:57.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.702+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 shutdown_connections 2024-09-21T01:46:57.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:56.702+0000 7fc6619ef640 1 -- 172.21.15.29:0/3968321403 wait complete. 2024-09-21T01:46:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:57 smithi066 ceph-mon[35001]: pgmap v318: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:57 smithi029 ceph-mon[31228]: pgmap v318: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:57 smithi190 ceph-mon[34368]: pgmap v318: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 -- 172.21.15.29:0/1409372554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 msgr2=0x7f30a4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 --2- 172.21.15.29:0/1409372554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a4110630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f309c009ca0 tx=0x7f309c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 -- 172.21.15.29:0/1409372554 shutdown_connections 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 --2- 172.21.15.29:0/1409372554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a4110630 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 --2- 172.21.15.29:0/1409372554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a41094b0 0x7f30a41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 --2- 172.21.15.29:0/1409372554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30a4103b70 0x7f30a4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 -- 172.21.15.29:0/1409372554 >> 172.21.15.29:0/1409372554 conn(0x7f30a40ff5e0 msgr2=0x7f30a4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 -- 172.21.15.29:0/1409372554 shutdown_connections 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.862+0000 7f30aadf6640 1 -- 172.21.15.29:0/1409372554 wait complete. 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 Processor -- start 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 -- start start 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30a4103b70 0x7f30a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a41094b0 0x7f30a419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4113ac0 con 0x7f30a4109e00 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4113940 con 0x7f30a4103b70 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30aadf6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4113c40 con 0x7f30a41094b0 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30a9393640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30a9393640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33260/0 (socket says 172.21.15.29:33260) 2024-09-21T01:46:57.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30a9393640 1 -- 172.21.15.29:0/2152640925 learned_addr learned my addr 172.21.15.29:0/2152640925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30a8b92640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30a4103b70 0x7f30a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.863+0000 7f30a3fff640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a41094b0 0x7f30a419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 -- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a41094b0 msgr2=0x7f30a419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a41094b0 0x7f30a419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 -- 172.21.15.29:0/2152640925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30a4103b70 msgr2=0x7f30a419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30a4103b70 0x7f30a419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41a0420 con 0x7f30a4109e00 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a9393640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a419fcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f309c0098d0 tx=0x7f309c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c003b70 con 0x7f30a4109e00 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c003d10 con 0x7f30a4109e00 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a06b0 con 0x7f30a4109e00 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c02fa20 con 0x7f30a4109e00 2024-09-21T01:46:57.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.864+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a6e70 con 0x7f30a4109e00 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.865+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30a4063b50 con 0x7f30a4109e00 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.867+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f309c02fbc0 con 0x7f30a4109e00 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a1ffb640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f308c0b6620 0x7f308c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a8b92640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f308c0b6620 0x7f308c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f309c108a00 con 0x7f30a4109e00 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a1ffb640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] conn(0x7f308c0c0110 0x7f308c0c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f308c0c2cd0 con 0x7f308c0c0110 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f309c108db0 con 0x7f30a4109e00 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a8b92640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f308c0b6620 0x7f308c0b8b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3094009bd0 tx=0x7f3094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.868+0000 7f30a3fff640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] conn(0x7f308c0c0110 0x7f308c0c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.869+0000 7f30a3fff640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] conn(0x7f308c0c0110 0x7f308c0c2600 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:57.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.870+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== osd.11 v2:172.21.15.190:6824/3508320462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f308c0c2cd0 con 0x7f308c0c0110 2024-09-21T01:46:57.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.884+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 --> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30a4063f50 con 0x7f308c0c0110 2024-09-21T01:46:57.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30a1ffb640 1 -- 172.21.15.29:0/2152640925 <== osd.11 v2:172.21.15.190:6824/3508320462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30a4063f50 con 0x7f308c0c0110 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] conn(0x7f308c0c0110 msgr2=0x7f308c0c2600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] conn(0x7f308c0c0110 0x7f308c0c2600 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f308c0b6620 msgr2=0x7f308c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f308c0b6620 0x7f308c0b8b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3094009bd0 tx=0x7f3094005f90 comp rx=0 tx=0).stop 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 msgr2=0x7f30a419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 --2- 172.21.15.29:0/2152640925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4109e00 0x7f30a419fcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f309c0098d0 tx=0x7f309c0317b0 comp rx=0 tx=0).stop 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30a9393640 1 -- 172.21.15.29:0/2152640925 reap_dead start 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 shutdown_connections 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.885+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 >> 172.21.15.29:0/2152640925 conn(0x7f30a40ff5e0 msgr2=0x7f30a4107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.886+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 shutdown_connections 2024-09-21T01:46:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.886+0000 7f30aadf6640 1 -- 172.21.15.29:0/2152640925 wait complete. 2024-09-21T01:46:58.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 -- 172.21.15.29:0/2284841118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470102a60 msgr2=0x7fd470102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 --2- 172.21.15.29:0/2284841118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470102a60 0x7fd470102e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd460009900 tx=0x7fd46002a410 comp rx=0 tx=0).stop 2024-09-21T01:46:58.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 -- 172.21.15.29:0/2284841118 shutdown_connections 2024-09-21T01:46:58.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 --2- 172.21.15.29:0/2284841118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd470108cf0 0x7fd47010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 --2- 172.21.15.29:0/2284841118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4701083a0 0x7fd4701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 --2- 172.21.15.29:0/2284841118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470102a60 0x7fd470102e70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 -- 172.21.15.29:0/2284841118 >> 172.21.15.29:0/2284841118 conn(0x7fd470079730 msgr2=0x7fd470079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 -- 172.21.15.29:0/2284841118 shutdown_connections 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.618+0000 7fd47485b640 1 -- 172.21.15.29:0/2284841118 wait complete. 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.619+0000 7fd47485b640 1 Processor -- start 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.619+0000 7fd47485b640 1 -- start start 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd470102a60 0x7fd4701993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4701083a0 0x7fd470199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 0x7fd47019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112960 con 0x7fd470108cf0 2024-09-21T01:46:58.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701127e0 con 0x7fd470102a60 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd47485b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112ae0 con 0x7fd4701083a0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 0x7fd47019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 0x7fd47019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33280/0 (socket says 172.21.15.29:33280) 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 -- 172.21.15.29:0/1711050766 learned_addr learned my addr 172.21.15.29:0/1711050766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46e59c640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd470102a60 0x7fd4701993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46dd9b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4701083a0 0x7fd470199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 -- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4701083a0 msgr2=0x7fd470199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4701083a0 0x7fd470199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 -- 172.21.15.29:0/1711050766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd470102a60 msgr2=0x7fd4701993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd470102a60 0x7fd4701993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47019f260 con 0x7fd470108cf0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd46ed9d640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 0x7fd47019ebf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd468007c30 tx=0x7fd46800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:58.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468019020 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.620+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468007e40 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.621+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47019f550 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.621+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468005560 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.621+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47006ae60 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.621+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd47006b410 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.623+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd46800eb60 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd45f7fe640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd4580b6650 0x7fd4580b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd46e59c640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd4580b6650 0x7fd4580b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd4680e9630 con 0x7fd470108cf0 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd45f7fe640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] conn(0x7fd4580c0140 0x7fd4580c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd46801de70 con 0x7fd4580c0140 2024-09-21T01:46:58.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd4680e9a80 con 0x7fd470108cf0 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd46dd9b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] conn(0x7fd4580c0140 0x7fd4580c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.624+0000 7fd46e59c640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd4580b6650 0x7fd4580b8b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd460031440 tx=0x7fd460005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.625+0000 7fd46dd9b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] conn(0x7fd4580c0140 0x7fd4580c2630 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.626+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== osd.5 v2:172.21.15.190:6808/4247249906 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd46801de70 con 0x7fd4580c0140 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.639+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 --> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd47006b890 con 0x7fd4580c0140 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd45f7fe640 1 -- 172.21.15.29:0/1711050766 <== osd.5 v2:172.21.15.190:6808/4247249906 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd47006b890 con 0x7fd4580c0140 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] conn(0x7fd4580c0140 msgr2=0x7fd4580c2630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] conn(0x7fd4580c0140 0x7fd4580c2630 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd4580b6650 msgr2=0x7fd4580b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd4580b6650 0x7fd4580b8b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd460031440 tx=0x7fd460005e80 comp rx=0 tx=0).stop 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 msgr2=0x7fd47019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 --2- 172.21.15.29:0/1711050766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd470108cf0 0x7fd47019ebf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd468007c30 tx=0x7fd46800e420 comp rx=0 tx=0).stop 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd46ed9d640 1 -- 172.21.15.29:0/1711050766 reap_dead start 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 shutdown_connections 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 >> 172.21.15.29:0/1711050766 conn(0x7fd470079730 msgr2=0x7fd470101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 shutdown_connections 2024-09-21T01:46:58.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:58.640+0000 7fd47485b640 1 -- 172.21.15.29:0/1711050766 wait complete. 2024-09-21T01:46:59.498 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:46:59 smithi029 ceph-mon[31228]: pgmap v319: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:59.529 INFO:teuthology.orchestra.run.smithi029.stdout:154618822711 2024-09-21T01:46:59.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T01:46:59.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:46:59 smithi066 ceph-mon[35001]: pgmap v319: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:46:59 smithi190 ceph-mon[34368]: pgmap v319: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:46:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 -- 172.21.15.29:0/4136358687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b410ddf0 msgr2=0x7ff2b4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/4136358687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b410ddf0 0x7ff2b4110300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009900 tx=0x7ff2ac02a410 comp rx=0 tx=0).stop 2024-09-21T01:46:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 -- 172.21.15.29:0/4136358687 shutdown_connections 2024-09-21T01:46:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/4136358687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b410ddf0 0x7ff2b4110300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/4136358687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4105270 0x7ff2b410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/4136358687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4104840 0x7ff2b4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 -- 172.21.15.29:0/4136358687 >> 172.21.15.29:0/4136358687 conn(0x7ff2b4079730 msgr2=0x7ff2b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 -- 172.21.15.29:0/4136358687 shutdown_connections 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.815+0000 7ff2bacb2640 1 -- 172.21.15.29:0/4136358687 wait complete. 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 Processor -- start 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 -- start start 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 0x7ff2b419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4105270 0x7ff2b419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b410ddf0 0x7ff2b419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4113a70 con 0x7ff2b4104840 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41138f0 con 0x7ff2b410ddf0 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2bacb2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4113bf0 con 0x7ff2b4105270 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2b8a4e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 0x7ff2b419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2b8a4e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 0x7ff2b419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33194/0 (socket says 172.21.15.29:33194) 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.816+0000 7ff2b8a4e640 1 -- 172.21.15.29:0/2631978951 learned_addr learned my addr 172.21.15.29:0/2631978951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b3fff640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4105270 0x7ff2b419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b924f640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b410ddf0 0x7ff2b419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 -- 172.21.15.29:0/2631978951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4105270 msgr2=0x7ff2b419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4105270 0x7ff2b419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 -- 172.21.15.29:0/2631978951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b410ddf0 msgr2=0x7ff2b419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b410ddf0 0x7ff2b419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41a0330 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b924f640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b410ddf0 0x7ff2b419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b8a4e640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 0x7ff2b419a3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00d770 tx=0x7ff29c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c017070 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41a0620 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c009c30 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c003cc0 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.817+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a6d80 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.818+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2b4106d30 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.820+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff29c003e60 con 0x7ff2b4104840 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b1ffb640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2a00b65a0 0x7ff2a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b3fff640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2a00b65a0 0x7ff2a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:59.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff29c0e7300 con 0x7ff2b4104840 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b1ffb640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] conn(0x7ff2a00c0090 0x7ff2a00c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2a00c2c50 con 0x7ff2a00c0090 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff29c0e76b0 con 0x7ff2b4104840 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b924f640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] conn(0x7ff2a00c0090 0x7ff2a00c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.821+0000 7ff2b3fff640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2a00b65a0 0x7ff2a00b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2b410fbc0 tx=0x7ff2a4009810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.822+0000 7ff2b924f640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] conn(0x7ff2a00c0090 0x7ff2a00c2580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:46:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.822+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== osd.9 v2:172.21.15.29:6826/246841085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2a00c2c50 con 0x7ff2a00c0090 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.836+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 --> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2b406b500 con 0x7ff2a00c0090 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.836+0000 7ff2b1ffb640 1 -- 172.21.15.29:0/2631978951 <== osd.9 v2:172.21.15.29:6826/246841085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff2b406b500 con 0x7ff2a00c0090 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] conn(0x7ff2a00c0090 msgr2=0x7ff2a00c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] conn(0x7ff2a00c0090 0x7ff2a00c2580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2a00b65a0 msgr2=0x7ff2a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2a00b65a0 0x7ff2a00b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2b410fbc0 tx=0x7ff2a4009810 comp rx=0 tx=0).stop 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 msgr2=0x7ff2b419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 --2- 172.21.15.29:0/2631978951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2b4104840 0x7ff2b419a3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00d770 tx=0x7ff29c00dc30 comp rx=0 tx=0).stop 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2b924f640 1 -- 172.21.15.29:0/2631978951 reap_dead start 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 shutdown_connections 2024-09-21T01:46:59.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 >> 172.21.15.29:0/2631978951 conn(0x7ff2b4079730 msgr2=0x7ff2b41062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:46:59.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 shutdown_connections 2024-09-21T01:46:59.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:54.837+0000 7ff2bacb2640 1 -- 172.21.15.29:0/2631978951 wait complete. 2024-09-21T01:47:00.530 INFO:teuthology.orchestra.run.smithi029.stdout:292057776152 2024-09-21T01:47:00.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/2216679684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105b70 msgr2=0x7fc3f810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/2216679684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105b70 0x7fc3f810f600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009ca0 tx=0x7fc3f002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/2216679684 shutdown_connections 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/2216679684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105b70 0x7fc3f810f600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/2216679684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3f8105220 0x7fc3f8105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/2216679684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f8103210 0x7fc3f8103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/2216679684 >> 172.21.15.29:0/2216679684 conn(0x7fc3f8079ea0 msgr2=0x7fc3f807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/2216679684 shutdown_connections 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.743+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/2216679684 wait complete. 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.744+0000 7fc3ff1bd640 1 Processor -- start 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.744+0000 7fc3ff1bd640 1 -- start start 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.744+0000 7fc3ff1bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f8103210 0x7fc3f8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.744+0000 7fc3ff1bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 0x7fc3f8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.744+0000 7fc3ff1bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3f8105b70 0x7fc3f819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3ff1bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8112a60 con 0x7fc3f8105220 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3ff1bd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81128e0 con 0x7fc3f8105b70 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3ff1bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8112be0 con 0x7fc3f8103210 2024-09-21T01:47:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 0x7fc3f8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 0x7fc3f8199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33242/0 (socket says 172.21.15.29:33242) 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 -- 172.21.15.29:0/3313067308 learned_addr learned my addr 172.21.15.29:0/3313067308 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3fd75a640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3f8105b70 0x7fc3f819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3fcf59640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f8103210 0x7fc3f8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 -- 172.21.15.29:0/3313067308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f8103210 msgr2=0x7fc3f8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f8103210 0x7fc3f8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 -- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3f8105b70 msgr2=0x7fc3f819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3f8105b70 0x7fc3f819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f819f3e0 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f7fff640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 0x7fc3f8199a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00eb00 tx=0x7fc3ec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec01c070 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec00bbd0 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f819f6d0 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.745+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec016610 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.746+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f8101e70 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.746+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3f8102360 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.748+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc3ec01f070 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3f5ffb640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3e00b6620 0x7fc3e00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3fcf59640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3e00b6620 0x7fc3e00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc3ec0e8f00 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3f5ffb640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] conn(0x7fc3e00c0110 0x7fc3e00c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3ec016e80 con 0x7fc3e00c0110 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc3ec0e92b0 con 0x7fc3f8105220 2024-09-21T01:47:01.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.749+0000 7fc3fcf59640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3e00b6620 0x7fc3e00b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806bb90 tx=0x7fc3e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.750+0000 7fc3fd75a640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] conn(0x7fc3e00c0110 0x7fc3e00c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.750+0000 7fc3fd75a640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] conn(0x7fc3e00c0110 0x7fc3e00c2600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.751+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== osd.1 v2:172.21.15.66:6800/2183970412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3ec016e80 con 0x7fc3e00c0110 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.765+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 --> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3f8102760 con 0x7fc3e00c0110 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3f5ffb640 1 -- 172.21.15.29:0/3313067308 <== osd.1 v2:172.21.15.66:6800/2183970412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc3f8102760 con 0x7fc3e00c0110 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] conn(0x7fc3e00c0110 msgr2=0x7fc3e00c2600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] conn(0x7fc3e00c0110 0x7fc3e00c2600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3e00b6620 msgr2=0x7fc3e00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc3e00b6620 0x7fc3e00b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806bb90 tx=0x7fc3e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 msgr2=0x7fc3f8199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 --2- 172.21.15.29:0/3313067308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8105220 0x7fc3f8199a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00eb00 tx=0x7fc3ec009ec0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3fd75a640 1 -- 172.21.15.29:0/3313067308 reap_dead start 2024-09-21T01:47:01.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 shutdown_connections 2024-09-21T01:47:01.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 >> 172.21.15.29:0/3313067308 conn(0x7fc3f8079ea0 msgr2=0x7fc3f81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:01.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 shutdown_connections 2024-09-21T01:47:01.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:46:57.766+0000 7fc3ff1bd640 1 -- 172.21.15.29:0/3313067308 wait complete. 2024-09-21T01:47:01.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 -- 172.21.15.29:0/59268396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964110190 msgr2=0x7f79641125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 --2- 172.21.15.29:0/59268396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964110190 0x7f79641125b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f795c009900 tx=0x7f795c02a480 comp rx=0 tx=0).stop 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 -- 172.21.15.29:0/59268396 shutdown_connections 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 --2- 172.21.15.29:0/59268396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964110190 0x7f79641125b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 --2- 172.21.15.29:0/59268396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7964102f30 0x7f796410fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 --2- 172.21.15.29:0/59268396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79641025e0 0x7f79641029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 -- 172.21.15.29:0/59268396 >> 172.21.15.29:0/59268396 conn(0x7f7964079710 msgr2=0x7f7964079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 -- 172.21.15.29:0/59268396 shutdown_connections 2024-09-21T01:47:01.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.510+0000 7f796be3c640 1 -- 172.21.15.29:0/59268396 wait complete. 2024-09-21T01:47:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 Processor -- start 2024-09-21T01:47:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 -- start start 2024-09-21T01:47:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79641025e0 0x7f796419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 0x7f796419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7964110190 0x7f79641a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7964115c80 con 0x7f7964102f30 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7964115b00 con 0x7f7964110190 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f796be3c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7964115e00 con 0x7f79641025e0 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f79693d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 0x7f796419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f79693d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 0x7f796419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33300/0 (socket says 172.21.15.29:33300) 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.511+0000 7f79693d7640 1 -- 172.21.15.29:0/1358699235 learned_addr learned my addr 172.21.15.29:0/1358699235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f796a3d9640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7964110190 0x7f79641a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f7969bd8640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79641025e0 0x7f796419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 -- 172.21.15.29:0/1358699235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79641025e0 msgr2=0x7f796419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79641025e0 0x7f796419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 -- 172.21.15.29:0/1358699235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7964110190 msgr2=0x7f79641a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7964110190 0x7f79641a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79641a2480 con 0x7f7964102f30 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f7969bd8640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79641025e0 0x7f796419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f796a3d9640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7964110190 0x7f79641a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f79693d7640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 0x7f796419cb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f795800c920 tx=0x7f795800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958007c70 con 0x7f7964102f30 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641a2770 con 0x7f7964102f30 2024-09-21T01:47:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7958007e10 con 0x7f7964102f30 2024-09-21T01:47:01.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958018820 con 0x7f7964102f30 2024-09-21T01:47:01.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.512+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a8f90 con 0x7f7964102f30 2024-09-21T01:47:01.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.513+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f796410a700 con 0x7f7964102f30 2024-09-21T01:47:01.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.515+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f795800f450 con 0x7f7964102f30 2024-09-21T01:47:01.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7962ffd640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f794c0b65d0 0x7f794c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7969bd8640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f794c0b65d0 0x7f794c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f79580e7a60 con 0x7f7964102f30 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7962ffd640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] conn(0x7f794c0c00c0 0x7f794c0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f795801ee50 con 0x7f794c0c00c0 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f79580e7e10 con 0x7f7964102f30 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f796a3d9640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] conn(0x7f794c0c00c0 0x7f794c0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.516+0000 7f7969bd8640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f794c0b65d0 0x7f794c0b8ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7954007e70 tx=0x7f7954007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.517+0000 7f796a3d9640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] conn(0x7f794c0c00c0 0x7f794c0c25b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.517+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== osd.0 v2:172.21.15.29:6802/271350255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f795801ee50 con 0x7f794c0c00c0 2024-09-21T01:47:01.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.531+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 --> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f796406b390 con 0x7f794c0c00c0 2024-09-21T01:47:01.531 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:01 smithi029 ceph-mon[31228]: pgmap v320: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:47:01.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.531+0000 7f7962ffd640 1 -- 172.21.15.29:0/1358699235 <== osd.0 v2:172.21.15.29:6802/271350255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f796406b390 con 0x7f794c0c00c0 2024-09-21T01:47:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] conn(0x7f794c0c00c0 msgr2=0x7f794c0c25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] conn(0x7f794c0c00c0 0x7f794c0c25b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f794c0b65d0 msgr2=0x7f794c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f794c0b65d0 0x7f794c0b8ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7954007e70 tx=0x7f7954007850 comp rx=0 tx=0).stop 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 msgr2=0x7f796419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 --2- 172.21.15.29:0/1358699235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7964102f30 0x7f796419cb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f795800c920 tx=0x7f795800cde0 comp rx=0 tx=0).stop 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796a3d9640 1 -- 172.21.15.29:0/1358699235 reap_dead start 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 shutdown_connections 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 >> 172.21.15.29:0/1358699235 conn(0x7f7964079710 msgr2=0x7f7964109ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 shutdown_connections 2024-09-21T01:47:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:01.532+0000 7f796be3c640 1 -- 172.21.15.29:0/1358699235 wait complete. 2024-09-21T01:47:01.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:01 smithi066 ceph-mon[35001]: pgmap v320: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:47:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:01 smithi190 ceph-mon[34368]: pgmap v320: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:47:02.019 INFO:teuthology.orchestra.run.smithi029.stdout:249108103201 2024-09-21T01:47:02.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T01:47:02.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 -- 172.21.15.29:0/2203177392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc103b60 msgr2=0x7f6ccc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/2203177392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc103b60 0x7f6ccc103f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:02.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 -- 172.21.15.29:0/2203177392 shutdown_connections 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/2203177392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc109df0 0x7f6ccc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/2203177392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ccc1094a0 0x7f6ccc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/2203177392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc103b60 0x7f6ccc103f70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 -- 172.21.15.29:0/2203177392 >> 172.21.15.29:0/2203177392 conn(0x7f6ccc07b850 msgr2=0x7f6ccc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 -- 172.21.15.29:0/2203177392 shutdown_connections 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.504+0000 7f6cd09b5640 1 -- 172.21.15.29:0/2203177392 wait complete. 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 Processor -- start 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 -- start start 2024-09-21T01:47:02.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ccc103b60 0x7f6ccc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc1094a0 0x7f6ccc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 0x7f6ccc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc113ab0 con 0x7f6ccc109df0 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc113930 con 0x7f6ccc1094a0 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cd09b5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc113c30 con 0x7f6ccc103b60 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 0x7f6ccc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 0x7f6ccc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46852/0 (socket says 172.21.15.29:46852) 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 -- 172.21.15.29:0/4156276692 learned_addr learned my addr 172.21.15.29:0/4156276692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cca59c640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ccc103b60 0x7f6ccc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6cc9d9b640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc1094a0 0x7f6ccc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 -- 172.21.15.29:0/4156276692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ccc103b60 msgr2=0x7f6ccc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ccc103b60 0x7f6ccc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 -- 172.21.15.29:0/4156276692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc1094a0 msgr2=0x7f6ccc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.505+0000 7f6ccad9d640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc1094a0 0x7f6ccc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6ccad9d640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc1a0490 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cc9d9b640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ccc1094a0 0x7f6ccc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6ccad9d640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 0x7f6ccc19fd60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009fd0 tx=0x7f6cc400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4017070 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1a0780 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc4003e20 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4010460 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.506+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1a6ee0 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.507+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ccc063b50 con 0x7f6ccc109df0 2024-09-21T01:47:02.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.509+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6cc4003970 con 0x7f6ccc109df0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.509+0000 7f6cbb7fe640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6cb40b65a0 0x7f6cb40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cca59c640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6cb40b65a0 0x7f6cb40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6cc40e7550 con 0x7f6ccc109df0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cbb7fe640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] conn(0x7f6cb40c0090 0x7f6cb40c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cb40c2c50 con 0x7f6cb40c0090 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6cc40e79a0 con 0x7f6ccc109df0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cc9d9b640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] conn(0x7f6cb40c0090 0x7f6cb40c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cca59c640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6cb40b65a0 0x7f6cb40b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc031440 tx=0x7f6cbc009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.510+0000 7f6cc9d9b640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] conn(0x7f6cb40c0090 0x7f6cb40c2580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:02.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.511+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== osd.3 v2:172.21.15.29:6810/1440460638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cb40c2c50 con 0x7f6cb40c0090 2024-09-21T01:47:02.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.525+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 --> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ccc063fd0 con 0x7f6cb40c0090 2024-09-21T01:47:02.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.525+0000 7f6cbb7fe640 1 -- 172.21.15.29:0/4156276692 <== osd.3 v2:172.21.15.29:6810/1440460638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ccc063fd0 con 0x7f6cb40c0090 2024-09-21T01:47:02.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.525+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] conn(0x7f6cb40c0090 msgr2=0x7f6cb40c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.525+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] conn(0x7f6cb40c0090 0x7f6cb40c2580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:02.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6cb40b65a0 msgr2=0x7f6cb40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6cb40b65a0 0x7f6cb40b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc031440 tx=0x7f6cbc009560 comp rx=0 tx=0).stop 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 msgr2=0x7f6ccc19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 --2- 172.21.15.29:0/4156276692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ccc109df0 0x7f6ccc19fd60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009fd0 tx=0x7f6cc400b260 comp rx=0 tx=0).stop 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6ccad9d640 1 -- 172.21.15.29:0/4156276692 reap_dead start 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 shutdown_connections 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 >> 172.21.15.29:0/4156276692 conn(0x7f6ccc07b850 msgr2=0x7f6ccc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 shutdown_connections 2024-09-21T01:47:02.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:02.526+0000 7f6cd09b5640 1 -- 172.21.15.29:0/4156276692 wait complete. 2024-09-21T01:47:03.160 INFO:teuthology.orchestra.run.smithi029.stdout:55834574922 2024-09-21T01:47:03.160 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T01:47:03.386 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:03 smithi029 ceph-mon[31228]: pgmap v321: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:03.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:03 smithi066 ceph-mon[35001]: pgmap v321: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:03.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:03 smithi190 ceph-mon[34368]: pgmap v321: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 -- 172.21.15.29:0/2916504580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa60108cf0 msgr2=0x7ffa6010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 --2- 172.21.15.29:0/2916504580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa60108cf0 0x7ffa6010f520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009ca0 tx=0x7ffa5002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 -- 172.21.15.29:0/2916504580 shutdown_connections 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 --2- 172.21.15.29:0/2916504580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa60108cf0 0x7ffa6010f520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 --2- 172.21.15.29:0/2916504580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa601083a0 0x7ffa601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 --2- 172.21.15.29:0/2916504580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa60102a60 0x7ffa60102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 -- 172.21.15.29:0/2916504580 >> 172.21.15.29:0/2916504580 conn(0x7ffa60079730 msgr2=0x7ffa60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 -- 172.21.15.29:0/2916504580 shutdown_connections 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.858+0000 7ffa674c5640 1 -- 172.21.15.29:0/2916504580 wait complete. 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 Processor -- start 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 -- start start 2024-09-21T01:47:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa60102a60 0x7ffa601993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 0x7ffa60199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa60108cf0 0x7ffa6019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601129c0 con 0x7ffa601083a0 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60112840 con 0x7ffa60108cf0 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa674c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60112b40 con 0x7ffa60102a60 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa64a60640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 0x7ffa60199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa64a60640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 0x7ffa60199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46906/0 (socket says 172.21.15.29:46906) 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.859+0000 7ffa64a60640 1 -- 172.21.15.29:0/1165077573 learned_addr learned my addr 172.21.15.29:0/1165077573 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa65a62640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa60108cf0 0x7ffa6019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa65261640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa60102a60 0x7ffa601993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 -- 172.21.15.29:0/1165077573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa60102a60 msgr2=0x7ffa601993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa60102a60 0x7ffa601993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 -- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa60108cf0 msgr2=0x7ffa6019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa60108cf0 0x7ffa6019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6019f360 con 0x7ffa601083a0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa65261640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa60102a60 0x7ffa601993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa64a60640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 0x7ffa60199990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b450 tx=0x7ffa5400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54015020 con 0x7ffa601083a0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa54011d40 con 0x7ffa601083a0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6019f650 con 0x7ffa601083a0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400f340 con 0x7ffa601083a0 2024-09-21T01:47:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.860+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a5db0 con 0x7ffa601083a0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.861+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa60063b10 con 0x7ffa601083a0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.863+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ffa54018020 con 0x7ffa601083a0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.863+0000 7ffa5e7fc640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa480b65a0 0x7ffa480b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa65261640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa480b65a0 0x7ffa480b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffa540e7520 con 0x7ffa601083a0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa5e7fc640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] conn(0x7ffa480c0090 0x7ffa480c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa54021e70 con 0x7ffa480c0090 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ffa540e7970 con 0x7ffa601083a0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa65261640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa480b65a0 0x7ffa480b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa580058c0 tx=0x7ffa58005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.864+0000 7ffa65a62640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] conn(0x7ffa480c0090 0x7ffa480c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.865+0000 7ffa65a62640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] conn(0x7ffa480c0090 0x7ffa480c2580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.866+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== osd.10 v2:172.21.15.66:6824/1772263510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa54021e70 con 0x7ffa480c0090 2024-09-21T01:47:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.879+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 --> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa60063f90 con 0x7ffa480c0090 2024-09-21T01:47:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa5e7fc640 1 -- 172.21.15.29:0/1165077573 <== osd.10 v2:172.21.15.66:6824/1772263510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa60063f90 con 0x7ffa480c0090 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] conn(0x7ffa480c0090 msgr2=0x7ffa480c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] conn(0x7ffa480c0090 0x7ffa480c2580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa480b65a0 msgr2=0x7ffa480b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa480b65a0 0x7ffa480b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa580058c0 tx=0x7ffa58005850 comp rx=0 tx=0).stop 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 msgr2=0x7ffa60199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 --2- 172.21.15.29:0/1165077573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa601083a0 0x7ffa60199990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b450 tx=0x7ffa5400b910 comp rx=0 tx=0).stop 2024-09-21T01:47:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa65a62640 1 -- 172.21.15.29:0/1165077573 reap_dead start 2024-09-21T01:47:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 shutdown_connections 2024-09-21T01:47:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 >> 172.21.15.29:0/1165077573 conn(0x7ffa60079730 msgr2=0x7ffa601052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 shutdown_connections 2024-09-21T01:47:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.880+0000 7ffa674c5640 1 -- 172.21.15.29:0/1165077573 wait complete. 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 -- 172.21.15.29:0/4170059192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c102ac0 msgr2=0x7f793c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 --2- 172.21.15.29:0/4170059192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c102ac0 0x7f793c102ed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f792c009900 tx=0x7f792c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 -- 172.21.15.29:0/4170059192 shutdown_connections 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 --2- 172.21.15.29:0/4170059192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f793c108d00 0x7f793c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 --2- 172.21.15.29:0/4170059192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793c1083b0 0x7f793c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 --2- 172.21.15.29:0/4170059192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c102ac0 0x7f793c102ed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 -- 172.21.15.29:0/4170059192 >> 172.21.15.29:0/4170059192 conn(0x7f793c079790 msgr2=0x7f793c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 -- 172.21.15.29:0/4170059192 shutdown_connections 2024-09-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.350+0000 7f79417d9640 1 -- 172.21.15.29:0/4170059192 wait complete. 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 Processor -- start 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 -- start start 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f793c102ac0 0x7f793c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793c1083b0 0x7f793c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 0x7f793c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.351+0000 7f79417d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f793c1129c0 con 0x7f793c108d00 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f79417d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f793c112840 con 0x7f793c1083b0 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f79417d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f793c112b40 con 0x7f793c102ac0 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 0x7f793c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 0x7f793c19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46870/0 (socket says 172.21.15.29:46870) 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 -- 172.21.15.29:0/91883865 learned_addr learned my addr 172.21.15.29:0/91883865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793a7fc640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793c1083b0 0x7f793c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793affd640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f793c102ac0 0x7f793c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 -- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f793c102ac0 msgr2=0x7f793c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f793c102ac0 0x7f793c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 -- 172.21.15.29:0/91883865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793c1083b0 msgr2=0x7f793c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793c1083b0 0x7f793c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1a6590 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.352+0000 7f793b7fe640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 0x7f793c19be10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f793400cce0 tx=0x7f793400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.353+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400f450 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.353+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c1a6880 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.353+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793400faa0 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.353+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934018a80 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.353+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c1a6d70 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.354+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f793c063ad0 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.355+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f793400fc50 con 0x7f793c108d00 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.356+0000 7f7923fff640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79240b65d0 0x7f79240b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.356+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f79340e7e10 con 0x7f793c108d00 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.356+0000 7f793affd640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79240b65d0 0x7f79240b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.357+0000 7f7923fff640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] conn(0x7f79240c00c0 0x7f79240c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.357+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79240c2c80 con 0x7f79240c00c0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.357+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f79340e81c0 con 0x7f793c108d00 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.357+0000 7f793affd640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79240b65d0 0x7f79240b8ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f792c000c00 tx=0x7f792c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.357+0000 7f793a7fc640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] conn(0x7f79240c00c0 0x7f79240c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.358+0000 7f793a7fc640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] conn(0x7f79240c00c0 0x7f79240c25b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.361+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== osd.2 v2:172.21.15.190:6800/2887309374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79240c2c80 con 0x7f79240c00c0 2024-09-21T01:47:04.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.375+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 --> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f793c063f50 con 0x7f79240c00c0 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f7923fff640 1 -- 172.21.15.29:0/91883865 <== osd.2 v2:172.21.15.190:6800/2887309374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f793c063f50 con 0x7f79240c00c0 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] conn(0x7f79240c00c0 msgr2=0x7f79240c25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] conn(0x7f79240c00c0 0x7f79240c25b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79240b65d0 msgr2=0x7f79240b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f79240b65d0 0x7f79240b8ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f792c000c00 tx=0x7f792c031400 comp rx=0 tx=0).stop 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 msgr2=0x7f793c19be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.376+0000 7f79417d9640 1 --2- 172.21.15.29:0/91883865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f793c108d00 0x7f793c19be10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f793400cce0 tx=0x7f793400a620 comp rx=0 tx=0).stop 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.377+0000 7f793b7fe640 1 -- 172.21.15.29:0/91883865 reap_dead start 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.377+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 shutdown_connections 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.377+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 >> 172.21.15.29:0/91883865 conn(0x7f793c079790 msgr2=0x7f793c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.377+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 shutdown_connections 2024-09-21T01:47:04.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.377+0000 7f79417d9640 1 -- 172.21.15.29:0/91883865 wait complete. 2024-09-21T01:47:04.722 INFO:teuthology.orchestra.run.smithi029.stdout:77309411399 2024-09-21T01:47:04.722 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T01:47:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 -- 172.21.15.29:0/1484603140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8103b60 msgr2=0x7f15e8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 --2- 172.21.15.29:0/1484603140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8103b60 0x7f15e8103f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009900 tx=0x7f15d802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 -- 172.21.15.29:0/1484603140 shutdown_connections 2024-09-21T01:47:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 --2- 172.21.15.29:0/1484603140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15e8109df0 0x7f15e8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 --2- 172.21.15.29:0/1484603140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 0x7f15e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 --2- 172.21.15.29:0/1484603140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8103b60 0x7f15e8103f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 -- 172.21.15.29:0/1484603140 >> 172.21.15.29:0/1484603140 conn(0x7f15e807b850 msgr2=0x7f15e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 -- 172.21.15.29:0/1484603140 shutdown_connections 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.763+0000 7f15ecce7640 1 -- 172.21.15.29:0/1484603140 wait complete. 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 Processor -- start 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 -- start start 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15e8103b60 0x7f15e819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 0x7f15e819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 0x7f15e819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113ab0 con 0x7f15e8109df0 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113930 con 0x7f15e8103b60 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15ecce7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113c30 con 0x7f15e81094a0 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15e6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 0x7f15e819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15e6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 0x7f15e819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46882/0 (socket says 172.21.15.29:46882) 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15e6d9d640 1 -- 172.21.15.29:0/3630948795 learned_addr learned my addr 172.21.15.29:0/3630948795 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15e5d9b640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 0x7f15e819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.764+0000 7f15e659c640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15e8103b60 0x7f15e819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 -- 172.21.15.29:0/3630948795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 msgr2=0x7f15e819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 0x7f15e819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 -- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15e8103b60 msgr2=0x7f15e819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15e8103b60 0x7f15e819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81a03d0 con 0x7f15e8109df0 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e5d9b640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15e81094a0 0x7f15e819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15e6d9d640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 0x7f15e819fd60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15e000cce0 tx=0x7f15e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000f450 con 0x7f15e8109df0 2024-09-21T01:47:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81a06c0 con 0x7f15e8109df0 2024-09-21T01:47:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e000faa0 con 0x7f15e8109df0 2024-09-21T01:47:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.765+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a6ee0 con 0x7f15e8109df0 2024-09-21T01:47:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.766+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0018b10 con 0x7f15e8109df0 2024-09-21T01:47:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.766+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15e8063b50 con 0x7f15e8109df0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.769+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f15e0018cb0 con 0x7f15e8109df0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.769+0000 7f15d77fe640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15d00b65d0 0x7f15d00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15e659c640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15d00b65d0 0x7f15d00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f15e00e7bf0 con 0x7f15e8109df0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15d77fe640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] conn(0x7f15d00c00c0 0x7f15d00c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15d00c2c80 con 0x7f15d00c00c0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f15e00e8040 con 0x7f15e8109df0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15e659c640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15d00b65d0 0x7f15d00b8ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f15d8031440 tx=0x7f15d8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.770+0000 7f15e5d9b640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] conn(0x7f15d00c00c0 0x7f15d00c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.771+0000 7f15e5d9b640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] conn(0x7f15d00c00c0 0x7f15d00c25b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.772+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== osd.7 v2:172.21.15.66:6816/3795777782 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15d00c2c80 con 0x7f15d00c00c0 2024-09-21T01:47:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.785+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 --> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15e8063fd0 con 0x7f15d00c00c0 2024-09-21T01:47:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15d77fe640 1 -- 172.21.15.29:0/3630948795 <== osd.7 v2:172.21.15.66:6816/3795777782 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f15e8063fd0 con 0x7f15d00c00c0 2024-09-21T01:47:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] conn(0x7f15d00c00c0 msgr2=0x7f15d00c25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] conn(0x7f15d00c00c0 0x7f15d00c25b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15d00b65d0 msgr2=0x7f15d00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f15d00b65d0 0x7f15d00b8ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f15d8031440 tx=0x7f15d8005e80 comp rx=0 tx=0).stop 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 msgr2=0x7f15e819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 --2- 172.21.15.29:0/3630948795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e8109df0 0x7f15e819fd60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15e000cce0 tx=0x7f15e000a620 comp rx=0 tx=0).stop 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15e6d9d640 1 -- 172.21.15.29:0/3630948795 reap_dead start 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 shutdown_connections 2024-09-21T01:47:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 >> 172.21.15.29:0/3630948795 conn(0x7f15e807b850 msgr2=0x7f15e8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:04.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 shutdown_connections 2024-09-21T01:47:04.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:03.786+0000 7f15ecce7640 1 -- 172.21.15.29:0/3630948795 wait complete. 2024-09-21T01:47:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:05 smithi066 ceph-mon[35001]: pgmap v322: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:47:05.574 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:05 smithi029 ceph-mon[31228]: pgmap v322: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:47:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:05 smithi190 ceph-mon[34368]: pgmap v322: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:47:06.163 INFO:teuthology.orchestra.run.smithi029.stdout:176093659183 2024-09-21T01:47:06.163 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T01:47:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2577545117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 msgr2=0x7f1d4410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2577545117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4410d8b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009900 tx=0x7f1d3802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2577545117 shutdown_connections 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2577545117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d4410ddf0 0x7f1d44110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2577545117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4410d8b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2577545117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d44104840 0x7f1d44104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2577545117 >> 172.21.15.29:0/2577545117 conn(0x7f1d44079730 msgr2=0x7f1d44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2577545117 shutdown_connections 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2577545117 wait complete. 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.417+0000 7f1d49e8e640 1 Processor -- start 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 -- start start 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d44104840 0x7f1d4419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d4410ddf0 0x7f1d4419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44113a70 con 0x7f1d44105270 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441138f0 con 0x7f1d4410ddf0 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d49e8e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44113bf0 con 0x7f1d44104840 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d42ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d42ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46928/0 (socket says 172.21.15.29:46928) 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d42ffd640 1 -- 172.21.15.29:0/2248343221 learned_addr learned my addr 172.21.15.29:0/2248343221 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d437fe640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d44104840 0x7f1d4419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.418+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d4410ddf0 0x7f1d4419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 -- 172.21.15.29:0/2248343221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d44104840 msgr2=0x7f1d4419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d44104840 0x7f1d4419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 -- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d4410ddf0 msgr2=0x7f1d4419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d4410ddf0 0x7f1d4419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441a0330 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d42ffd640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4419aa20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d3802a930 tx=0x7f1d38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3803f030 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a05c0 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38003aa0 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38035070 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.419+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a0af0 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.420+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d44063b50 con 0x7f1d44105270 2024-09-21T01:47:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.423+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1d38042030 con 0x7f1d44105270 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.423+0000 7f1d40ff9640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d2c0b65d0 0x7f1d2c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d38107f20 con 0x7f1d44105270 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d437fe640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d2c0b65d0 0x7f1d2c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d40ff9640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] conn(0x7f1d2c0c00c0 0x7f1d2c0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d38009080 con 0x7f1d2c0c00c0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1d38108370 con 0x7f1d44105270 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] conn(0x7f1d2c0c00c0 0x7f1d2c0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.424+0000 7f1d437fe640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d2c0b65d0 0x7f1d2c0b8ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d4406bc00 tx=0x7f1d3400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.425+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] conn(0x7f1d2c0c00c0 0x7f1d2c0c25b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.426+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== osd.4 v2:172.21.15.66:6808/2683216793 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d38009080 con 0x7f1d2c0c00c0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.439+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 --> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d44063f50 con 0x7f1d2c0c00c0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d40ff9640 1 -- 172.21.15.29:0/2248343221 <== osd.4 v2:172.21.15.66:6808/2683216793 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d44063f50 con 0x7f1d2c0c00c0 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] conn(0x7f1d2c0c00c0 msgr2=0x7f1d2c0c25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] conn(0x7f1d2c0c00c0 0x7f1d2c0c25b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d2c0b65d0 msgr2=0x7f1d2c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d2c0b65d0 0x7f1d2c0b8ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d4406bc00 tx=0x7f1d3400b390 comp rx=0 tx=0).stop 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 msgr2=0x7f1d4419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 --2- 172.21.15.29:0/2248343221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d44105270 0x7f1d4419aa20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d3802a930 tx=0x7f1d38004270 comp rx=0 tx=0).stop 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d43fff640 1 -- 172.21.15.29:0/2248343221 reap_dead start 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 shutdown_connections 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.440+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 >> 172.21.15.29:0/2248343221 conn(0x7f1d44079730 msgr2=0x7f1d44100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.441+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 shutdown_connections 2024-09-21T01:47:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:05.441+0000 7f1d49e8e640 1 -- 172.21.15.29:0/2248343221 wait complete. 2024-09-21T01:47:07.051 INFO:teuthology.orchestra.run.smithi029.stdout:201863462956 2024-09-21T01:47:07.052 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T01:47:07.312 INFO:teuthology.orchestra.run.smithi029.stdout:128849018939 2024-09-21T01:47:07.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T01:47:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:07 smithi190 ceph-mon[34368]: pgmap v323: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:07 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:47:07.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:07 smithi066 ceph-mon[35001]: pgmap v323: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:07.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:07 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:47:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:07 smithi029 ceph-mon[31228]: pgmap v323: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:07 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:47:07.766 INFO:teuthology.orchestra.run.smithi029.stdout:103079215169 2024-09-21T01:47:07.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T01:47:08.431 INFO:teuthology.orchestra.run.smithi029.stdout:34359738446 2024-09-21T01:47:08.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T01:47:08.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:08.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:08.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:08.978 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:08 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:08.978 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:08 smithi029 ceph-mon[31228]: pgmap v324: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:47:09.018 INFO:teuthology.orchestra.run.smithi029.stdout:270582939677 2024-09-21T01:47:09.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T01:47:09.137 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.138 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:08 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:08 smithi190 ceph-mon[34368]: pgmap v324: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:47:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:08 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:08 smithi066 ceph-mon[35001]: pgmap v324: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:47:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- 172.21.15.29:0/291330375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c104890 msgr2=0x7ff92c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 --2- 172.21.15.29:0/291330375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c104890 0x7ff92c104ca0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009900 tx=0x7ff91c02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- 172.21.15.29:0/291330375 shutdown_connections 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 --2- 172.21.15.29:0/291330375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c10de40 0x7ff92c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 --2- 172.21.15.29:0/291330375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c1052c0 0x7ff92c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 --2- 172.21.15.29:0/291330375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c104890 0x7ff92c104ca0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- 172.21.15.29:0/291330375 >> 172.21.15.29:0/291330375 conn(0x7ff92c079710 msgr2=0x7ff92c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- 172.21.15.29:0/291330375 shutdown_connections 2024-09-21T01:47:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- 172.21.15.29:0/291330375 wait complete. 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 Processor -- start 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.694+0000 7ff9324d0640 1 -- start start 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c104890 0x7ff92c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c10de40 0x7ff92c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c113b60 con 0x7ff92c1052c0 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1139e0 con 0x7ff92c10de40 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff9324d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c113ce0 con 0x7ff92c104890 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46946/0 (socket says 172.21.15.29:46946) 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 -- 172.21.15.29:0/143080306 learned_addr learned my addr 172.21.15.29:0/143080306 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92bfff640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c104890 0x7ff92c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 -- 172.21.15.29:0/143080306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c104890 msgr2=0x7ff92c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c104890 0x7ff92c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff930a6d640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c10de40 0x7ff92c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 -- 172.21.15.29:0/143080306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c10de40 msgr2=0x7ff92c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c10de40 0x7ff92c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.695+0000 7ff92b7fe640 1 -- 172.21.15.29:0/143080306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c1a0500 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff92b7fe640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b450 tx=0x7ff92000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920015020 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c1a07f0 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff920011d40 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92000f340 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.696+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1a6f50 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.697+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c06c1e0 con 0x7ff92c1052c0 2024-09-21T01:47:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.699+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff920018020 con 0x7ff92c1052c0 2024-09-21T01:47:09.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.699+0000 7ff9297fa640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 0x7ff9140b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:09.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.700+0000 7ff92bfff640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 0x7ff9140b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:09.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.700+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff9200e7400 con 0x7ff92c1052c0 2024-09-21T01:47:09.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.700+0000 7ff92bfff640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 0x7ff9140b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff91c031440 tx=0x7ff91c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:09.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.703+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff92001f020 con 0x7ff92c1052c0 2024-09-21T01:47:09.723 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.723 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.723 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.723 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.724 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.727 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.727 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.729 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:09.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.941+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff92c104d20 con 0x7ff92c1052c0 2024-09-21T01:47:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.942+0000 7ff9297fa640 1 -- 172.21.15.29:0/143080306 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff92c104d20 con 0x7ff92c1052c0 2024-09-21T01:47:09.942 INFO:teuthology.orchestra.run.smithi029.stdout:103079215170 2024-09-21T01:47:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 msgr2=0x7ff9140b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 0x7ff9140b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff91c031440 tx=0x7ff91c005e80 comp rx=0 tx=0).stop 2024-09-21T01:47:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 msgr2=0x7ff92c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b450 tx=0x7ff92000b910 comp rx=0 tx=0).stop 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 shutdown_connections 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9140b65a0 0x7ff9140b8ab0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c10de40 0x7ff92c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c1052c0 0x7ff92c19ab30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 --2- 172.21.15.29:0/143080306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff92c104890 0x7ff92c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 >> 172.21.15.29:0/143080306 conn(0x7ff92c079710 msgr2=0x7ff92c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 shutdown_connections 2024-09-21T01:47:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:09.944+0000 7ff9324d0640 1 -- 172.21.15.29:0/143080306 wait complete. 2024-09-21T01:47:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:10 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/143080306' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:47:10.321 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:10 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/143080306' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:47:10.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:10 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/143080306' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:47:11.134 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:11 smithi029 ceph-mon[31228]: pgmap v325: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:47:11.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 -- 172.21.15.29:0/2035323273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48104850 msgr2=0x7f8b48104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/2035323273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48104850 0x7f8b48104c60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 -- 172.21.15.29:0/2035323273 shutdown_connections 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/2035323273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 0x7f8b48110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/2035323273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48105280 0x7f8b4810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/2035323273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48104850 0x7f8b48104c60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 -- 172.21.15.29:0/2035323273 >> 172.21.15.29:0/2035323273 conn(0x7f8b48079730 msgr2=0x7f8b48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 -- 172.21.15.29:0/2035323273 shutdown_connections 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.085+0000 7f8b4ee94640 1 -- 172.21.15.29:0/2035323273 wait complete. 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 Processor -- start 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 -- start start 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 0x7f8b4819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:11.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 0x7f8b4819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48113aa0 con 0x7f8b48105280 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48113920 con 0x7f8b4810de00 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4ee94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48113c20 con 0x7f8b48104850 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46972/0 (socket says 172.21.15.29:46972) 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 -- 172.21.15.29:0/3018064391 learned_addr learned my addr 172.21.15.29:0/3018064391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4cc30640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 0x7f8b4819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b4d431640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 0x7f8b4819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 -- 172.21.15.29:0/3018064391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 msgr2=0x7f8b4819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 0x7f8b4819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 -- 172.21.15.29:0/3018064391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 msgr2=0x7f8b4819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 0x7f8b4819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.086+0000 7f8b47fff640 1 -- 172.21.15.29:0/3018064391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b481a0440 con 0x7f8b48105280 2024-09-21T01:47:11.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b4cc30640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 0x7f8b4819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b47fff640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00d750 tx=0x7f8b3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c017070 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b481a0730 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c0054d0 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004c30 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.087+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a6e90 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.088+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4806c180 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.090+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8b3c004dd0 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.090+0000 7f8b45ffb640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 0x7f8b300b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.090+0000 7f8b4cc30640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 0x7f8b300b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.090+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8b3c0e6f60 con 0x7f8b48105280 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.091+0000 7f8b4cc30640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 0x7f8b300b8ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38032cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:11.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.095+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b3c0ad6a0 con 0x7f8b48105280 2024-09-21T01:47:11.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.288+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8b4819b600 con 0x7f8b48105280 2024-09-21T01:47:11.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.289+0000 7f8b45ffb640 1 -- 172.21.15.29:0/3018064391 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b3c0b1a80 con 0x7f8b48105280 2024-09-21T01:47:11.289 INFO:teuthology.orchestra.run.smithi029.stdout:176093659186 2024-09-21T01:47:11.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 msgr2=0x7f8b300b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:11.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 0x7f8b300b8ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38032cc0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 msgr2=0x7f8b4819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:11.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00d750 tx=0x7f8b3c00dc10 comp rx=0 tx=0).stop 2024-09-21T01:47:11.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 shutdown_connections 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8b300b65a0 0x7f8b300b8ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4810de00 0x7f8b4819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b48105280 0x7f8b4819aa70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 --2- 172.21.15.29:0/3018064391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b48104850 0x7f8b4819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 >> 172.21.15.29:0/3018064391 conn(0x7f8b48079730 msgr2=0x7f8b48100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 shutdown_connections 2024-09-21T01:47:11.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:11.291+0000 7f8b4ee94640 1 -- 172.21.15.29:0/3018064391 wait complete. 2024-09-21T01:47:11.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:11 smithi066 ceph-mon[35001]: pgmap v325: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:47:11.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:11 smithi190 ceph-mon[34368]: pgmap v325: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:47:12.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:12 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3018064391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:47:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:12 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3018064391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:47:12.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:12 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3018064391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 -- 172.21.15.29:0/973224644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8660108cf0 msgr2=0x7f866010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 --2- 172.21.15.29:0/973224644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8660108cf0 0x7f866010f520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8658009ca0 tx=0x7f865802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 -- 172.21.15.29:0/973224644 shutdown_connections 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 --2- 172.21.15.29:0/973224644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8660108cf0 0x7f866010f520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 --2- 172.21.15.29:0/973224644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86601083a0 0x7f86601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 --2- 172.21.15.29:0/973224644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 0x7f8660102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 -- 172.21.15.29:0/973224644 >> 172.21.15.29:0/973224644 conn(0x7f8660079730 msgr2=0x7f8660079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 -- 172.21.15.29:0/973224644 shutdown_connections 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.717+0000 7f8664c85640 1 -- 172.21.15.29:0/973224644 wait complete. 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 Processor -- start 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 -- start start 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 0x7f866019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8660108cf0 0x7f866019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8660112960 con 0x7f86601083a0 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86601127e0 con 0x7f8660108cf0 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.718+0000 7f8664c85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8660112ae0 con 0x7f8660102a60 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32988/0 (socket says 172.21.15.29:32988) 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 -- 172.21.15.29:0/304326981 learned_addr learned my addr 172.21.15.29:0/304326981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:12.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f865ed9d640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8660108cf0 0x7f866019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f865e59c640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 0x7f866019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 -- 172.21.15.29:0/304326981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 msgr2=0x7f866019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 0x7f866019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 -- 172.21.15.29:0/304326981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8660108cf0 msgr2=0x7f866019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8660108cf0 0x7f866019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 -- 172.21.15.29:0/304326981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86601a02f0 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f8657fff640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f864800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.719+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648017070 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.720+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86601a05e0 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.720+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86601a6e00 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.720+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f864800de50 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.720+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648003c30 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.721+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86400052c0 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.723+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8648009c30 con 0x7f86601083a0 2024-09-21T01:47:12.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.723+0000 7f86577fe640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 0x7f864c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.723+0000 7f865e59c640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 0x7f864c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.723+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f86480e7650 con 0x7f86601083a0 2024-09-21T01:47:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.724+0000 7f865e59c640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 0x7f864c0b8b60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f865000abe0 tx=0x7f865000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.731+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8648020030 con 0x7f86601083a0 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 -- 172.21.15.29:0/1309904357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7941083b0 msgr2=0x7fc7941087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 --2- 172.21.15.29:0/1309904357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7941083b0 0x7fc7941087c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 -- 172.21.15.29:0/1309904357 shutdown_connections 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 --2- 172.21.15.29:0/1309904357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc794108d00 0x7fc79410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 --2- 172.21.15.29:0/1309904357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7941083b0 0x7fc7941087c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 --2- 172.21.15.29:0/1309904357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794102a70 0x7fc794102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 -- 172.21.15.29:0/1309904357 >> 172.21.15.29:0/1309904357 conn(0x7fc794079730 msgr2=0x7fc794079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 -- 172.21.15.29:0/1309904357 shutdown_connections 2024-09-21T01:47:12.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.576+0000 7fc79a113640 1 -- 172.21.15.29:0/1309904357 wait complete. 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 Processor -- start 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 -- start start 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7941083b0 0x7fc794199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794108d00 0x7fc79419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc794112970 con 0x7fc794102a70 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941127f0 con 0x7fc794108d00 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc79a113640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc794112af0 con 0x7fc7941083b0 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32970/0 (socket says 172.21.15.29:32970) 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc7937fe640 1 -- 172.21.15.29:0/1487377788 learned_addr learned my addr 172.21.15.29:0/1487377788 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc78affd640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7941083b0 0x7fc794199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.577+0000 7fc793fff640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794108d00 0x7fc79419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 -- 172.21.15.29:0/1487377788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7941083b0 msgr2=0x7fc794199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7941083b0 0x7fc794199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 -- 172.21.15.29:0/1487377788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794108d00 msgr2=0x7fc79419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794108d00 0x7fc79419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 -- 172.21.15.29:0/1487377788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79419f230 con 0x7fc794102a70 2024-09-21T01:47:12.931 INFO:teuthology.orchestra.run.smithi029.stdout:55834574925 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7937fe640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c920 tx=0x7fc78400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784007c70 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79419f520 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc784007e10 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784018820 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.578+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941a5d40 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.579+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc794103cf0 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.581+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc78400f450 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.582+0000 7fc7917fa640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 0x7fc7800b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.582+0000 7fc78affd640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 0x7fc7800b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.582+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc7840e7860 con 0x7fc794102a70 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.583+0000 7fc78affd640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 0x7fc7800b8ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.586+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc784014030 con 0x7fc794102a70 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.792+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc794102e80 con 0x7fc794102a70 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.792+0000 7fc7917fa640 1 -- 172.21.15.29:0/1487377788 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc794102e80 con 0x7fc794102a70 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 msgr2=0x7fc7800b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 0x7fc7800b8ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c031400 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 msgr2=0x7fc794199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c920 tx=0x7fc78400cde0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 shutdown_connections 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc7800b65d0 0x7fc7800b8ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc794108d00 0x7fc79419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7941083b0 0x7fc794199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 --2- 172.21.15.29:0/1487377788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc794102a70 0x7fc794199380 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 >> 172.21.15.29:0/1487377788 conn(0x7fc794079730 msgr2=0x7fc794104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:12.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 shutdown_connections 2024-09-21T01:47:12.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.795+0000 7fc79a113640 1 -- 172.21.15.29:0/1487377788 wait complete. 2024-09-21T01:47:12.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.932+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8640005880 con 0x7f86601083a0 2024-09-21T01:47:12.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.932+0000 7f86577fe640 1 -- 172.21.15.29:0/304326981 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f86480addb0 con 0x7f86601083a0 2024-09-21T01:47:12.934 INFO:teuthology.orchestra.run.smithi029.stdout:292057776155 2024-09-21T01:47:12.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 msgr2=0x7f864c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 0x7f864c0b8b60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f865000abe0 tx=0x7f865000a2d0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 msgr2=0x7f866019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:12.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f864800dd20 comp rx=0 tx=0).stop 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 shutdown_connections 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f864c0b6650 0x7f864c0b8b60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8660108cf0 0x7f866019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86601083a0 0x7f866019a9e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 --2- 172.21.15.29:0/304326981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660102a60 0x7f866019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 >> 172.21.15.29:0/304326981 conn(0x7f8660079730 msgr2=0x7f866006b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 shutdown_connections 2024-09-21T01:47:12.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.935+0000 7f8664c85640 1 -- 172.21.15.29:0/304326981 wait complete. 2024-09-21T01:47:13.252 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:13 smithi029 ceph-mon[31228]: pgmap v326: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:13.252 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:13 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2639396241' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:47:13.252 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:13 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1487377788' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:47:13.252 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:13 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/304326981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:47:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:13 smithi066 ceph-mon[35001]: pgmap v326: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:13 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2639396241' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:47:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:13 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1487377788' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:47:13.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:13 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/304326981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:47:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:13 smithi190 ceph-mon[34368]: pgmap v326: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:13 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2639396241' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:47:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:13 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1487377788' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:47:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:13 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/304326981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:47:13.638 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574925 for osd.1 2024-09-21T01:47:13.638 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 -- 172.21.15.29:0/1620235459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 msgr2=0x7faa78108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 --2- 172.21.15.29:0/1620235459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa78108830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009900 tx=0x7faa6c02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 -- 172.21.15.29:0/1620235459 shutdown_connections 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 --2- 172.21.15.29:0/1620235459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 --2- 172.21.15.29:0/1620235459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa78108830 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 --2- 172.21.15.29:0/1620235459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 0x7faa78102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 -- 172.21.15.29:0/1620235459 >> 172.21.15.29:0/1620235459 conn(0x7faa78079770 msgr2=0x7faa78079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:13.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 -- 172.21.15.29:0/1620235459 shutdown_connections 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.464+0000 7faa7edc9640 1 -- 172.21.15.29:0/1620235459 wait complete. 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 Processor -- start 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 -- start start 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 0x7faa7819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112a30 con 0x7faa78108420 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa781128b0 con 0x7faa78108d70 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7edc9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112bb0 con 0x7faa78102ae0 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32958/0 (socket says 172.21.15.29:32958) 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa77fff640 1 -- 172.21.15.29:0/2639396241 learned_addr learned my addr 172.21.15.29:0/2639396241 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7cb65640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 0x7faa7819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa7d366640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.465+0000 7faa77fff640 1 -- 172.21.15.29:0/2639396241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 msgr2=0x7faa7819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa77fff640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 0x7faa7819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa77fff640 1 -- 172.21.15.29:0/2639396241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 msgr2=0x7faa7819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa77fff640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa77fff640 1 -- 172.21.15.29:0/2639396241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781a65b0 con 0x7faa78108420 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa7d366640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa77fff640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa6c02a930 tx=0x7faa6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:154618822713 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c03f030 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a6840 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c003aa0 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c035070 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.466+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a6d70 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.467+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78103d60 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.469+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7faa6c042030 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.470+0000 7faa75ffb640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 0x7faa600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.470+0000 7faa7cb65640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 0x7faa600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.470+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faa6c107a10 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.470+0000 7faa7cb65640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 0x7faa600b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faa7806c720 tx=0x7faa6800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.475+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa6c0ce0f0 con 0x7faa78108420 2024-09-21T01:47:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.669+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faa7810ab60 con 0x7faa78108420 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.669+0000 7faa75ffb640 1 -- 172.21.15.29:0/2639396241 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa6c0d24d0 con 0x7faa78108420 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 msgr2=0x7faa600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 0x7faa600b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faa7806c720 tx=0x7faa6800b390 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 msgr2=0x7faa7819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa6c02a930 tx=0x7faa6c004270 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 shutdown_connections 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa600b65a0 0x7faa600b8ab0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa78108d70 0x7faa7819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78108420 0x7faa7819b7e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 --2- 172.21.15.29:0/2639396241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa78102ae0 0x7faa7819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 >> 172.21.15.29:0/2639396241 conn(0x7faa78079770 msgr2=0x7faa781064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 shutdown_connections 2024-09-21T01:47:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:12.672+0000 7faa7edc9640 1 -- 172.21.15.29:0/2639396241 wait complete. 2024-09-21T01:47:14.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659183 got 176093659186 for osd.6 2024-09-21T01:47:14.701 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:15.312 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:15 smithi029 ceph-mon[31228]: pgmap v327: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 -- 172.21.15.29:0/1973834042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 msgr2=0x7f26101098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 --2- 172.21.15.29:0/1973834042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f26101098b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260402a410 comp rx=0 tx=0).stop 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 -- 172.21.15.29:0/1973834042 shutdown_connections 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 --2- 172.21.15.29:0/1973834042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610109df0 0x7f2610110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 --2- 172.21.15.29:0/1973834042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f26101098b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 --2- 172.21.15.29:0/1973834042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610103b60 0x7f2610103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 -- 172.21.15.29:0/1973834042 >> 172.21.15.29:0/1973834042 conn(0x7f261007b850 msgr2=0x7f261007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 -- 172.21.15.29:0/1973834042 shutdown_connections 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.311+0000 7f261614d640 1 -- 172.21.15.29:0/1973834042 wait complete. 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.312+0000 7f261614d640 1 Processor -- start 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 -- start start 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610103b60 0x7f261019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610109df0 0x7f261019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2610113ab0 con 0x7f26101094a0 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2610113930 con 0x7f2610109df0 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f261614d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2610113c30 con 0x7f2610103b60 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33014/0 (socket says 172.21.15.29:33014) 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 -- 172.21.15.29:0/1618610188 learned_addr learned my addr 172.21.15.29:0/1618610188 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260f7fe640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610103b60 0x7f261019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260ffff640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610109df0 0x7f261019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 -- 172.21.15.29:0/1618610188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610103b60 msgr2=0x7f261019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610103b60 0x7f261019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 -- 172.21.15.29:0/1618610188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610109df0 msgr2=0x7f261019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610109df0 0x7f261019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.313+0000 7f260effd640 1 -- 172.21.15.29:0/1618610188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101a03b0 con 0x7f26101094a0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f260effd640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f260402a930 tx=0x7f2604004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260403f030 con 0x7f26101094a0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2604003aa0 con 0x7f26101094a0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101a0640 con 0x7f26101094a0 2024-09-21T01:47:15.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604035070 con 0x7f26101094a0 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.314+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a0b70 con 0x7f26101094a0 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.315+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610103f70 con 0x7f26101094a0 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.317+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2604042030 con 0x7f26101094a0 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.317+0000 7f260cff9640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 0x7f25f80b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.318+0000 7f260f7fe640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 0x7f25f80b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.318+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f26041080a0 con 0x7f26101094a0 2024-09-21T01:47:15.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.318+0000 7f260f7fe640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 0x7f25f80b8b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f260000abe0 tx=0x7f260000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.321+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26101a0b70 con 0x7f26101094a0 2024-09-21T01:47:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:15 smithi190 ceph-mon[34368]: pgmap v327: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:15.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.517+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f261002e640 con 0x7f26101094a0 2024-09-21T01:47:15.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.521+0000 7f260cff9640 1 -- 172.21.15.29:0/1618610188 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f26040ce780 con 0x7f26101094a0 2024-09-21T01:47:15.522 INFO:teuthology.orchestra.run.smithi029.stdout:227633266730 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 msgr2=0x7f25f80b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 0x7f25f80b8b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f260000abe0 tx=0x7f260000a2d0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 msgr2=0x7f261019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f260402a930 tx=0x7f2604004270 comp rx=0 tx=0).stop 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 shutdown_connections 2024-09-21T01:47:15.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f25f80b6650 0x7f25f80b8b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2610109df0 0x7f261019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26101094a0 0x7f261019aaa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 --2- 172.21.15.29:0/1618610188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610103b60 0x7f261019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 >> 172.21.15.29:0/1618610188 conn(0x7f261007b850 msgr2=0x7f26101075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 shutdown_connections 2024-09-21T01:47:15.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.524+0000 7f261614d640 1 -- 172.21.15.29:0/1618610188 wait complete. 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 -- 172.21.15.29:0/3683108356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 msgr2=0x7f8900104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 --2- 172.21.15.29:0/3683108356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f8900104dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009900 tx=0x7f88f002a410 comp rx=0 tx=0).stop 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 -- 172.21.15.29:0/3683108356 shutdown_connections 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 --2- 172.21.15.29:0/3683108356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890010de10 0x7f8900110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 --2- 172.21.15.29:0/3683108356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8900105300 0x7f890010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 --2- 172.21.15.29:0/3683108356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f8900104dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 -- 172.21.15.29:0/3683108356 >> 172.21.15.29:0/3683108356 conn(0x7f8900079730 msgr2=0x7f8900079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 -- 172.21.15.29:0/3683108356 shutdown_connections 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.512+0000 7f8904e8a640 1 -- 172.21.15.29:0/3683108356 wait complete. 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.513+0000 7f8904e8a640 1 Processor -- start 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.513+0000 7f8904e8a640 1 -- start start 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.513+0000 7f8904e8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.513+0000 7f8904e8a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 0x7f8900102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.513+0000 7f8904e8a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f890010de10 0x7f8900103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f8904e8a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8900113a90 con 0x7f89001049b0 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f8904e8a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8900113910 con 0x7f890010de10 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f8904e8a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8900113c10 con 0x7f8900105300 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33034/0 (socket says 172.21.15.29:33034) 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 -- 172.21.15.29:0/2034387020 learned_addr learned my addr 172.21.15.29:0/2034387020 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:15.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fed9d640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f890010de10 0x7f8900103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fdd9b640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 0x7f8900102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 -- 172.21.15.29:0/2034387020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 msgr2=0x7f8900102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 0x7f8900102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 -- 172.21.15.29:0/2034387020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f890010de10 msgr2=0x7f8900103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f890010de10 0x7f8900103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 -- 172.21.15.29:0/2034387020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001a87c0 con 0x7f89001049b0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fdd9b640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 0x7f8900102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.514+0000 7f88fe59c640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88f0032fd0 tx=0x7f88f0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.515+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f003f030 con 0x7f89001049b0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.515+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f0004490 con 0x7f89001049b0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.515+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001a8a50 con 0x7f89001049b0 2024-09-21T01:47:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.515+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0035070 con 0x7f89001049b0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.515+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a8f30 con 0x7f89001049b0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.516+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8900104dc0 con 0x7f89001049b0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.517+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f88f0042030 con 0x7f89001049b0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.518+0000 7f88ef7fe640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 0x7f88e80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.518+0000 7f88fdd9b640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 0x7f88e80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.518+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f88f0107c70 con 0x7f89001049b0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.519+0000 7f88fdd9b640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 0x7f88e80b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f890007a030 tx=0x7f88f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.521+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88f00ce3b0 con 0x7f89001049b0 2024-09-21T01:47:15.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:15 smithi066 ceph-mon[35001]: pgmap v327: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:15.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.718+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f890002e640 con 0x7f89001049b0 2024-09-21T01:47:15.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.718+0000 7f88ef7fe640 1 -- 172.21.15.29:0/2034387020 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88f00d2790 con 0x7f89001049b0 2024-09-21T01:47:15.719 INFO:teuthology.orchestra.run.smithi029.stdout:77309411401 2024-09-21T01:47:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 msgr2=0x7f88e80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 0x7f88e80b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f890007a030 tx=0x7f88f4005f90 comp rx=0 tx=0).stop 2024-09-21T01:47:15.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 msgr2=0x7f89001026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88f0032fd0 tx=0x7f88f0031a20 comp rx=0 tx=0).stop 2024-09-21T01:47:15.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 shutdown_connections 2024-09-21T01:47:15.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88e80b65a0 0x7f88e80b8ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f890010de10 0x7f8900103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8900105300 0x7f8900102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 --2- 172.21.15.29:0/2034387020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89001049b0 0x7f89001026f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.720+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 >> 172.21.15.29:0/2034387020 conn(0x7f8900079730 msgr2=0x7f89001007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.721+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 shutdown_connections 2024-09-21T01:47:15.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.721+0000 7f8904e8a640 1 -- 172.21.15.29:0/2034387020 wait complete. 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/1595022943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac103b60 msgr2=0x7ff0ac103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/1595022943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac103b60 0x7ff0ac103f70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009900 tx=0x7ff09c02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/1595022943 shutdown_connections 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/1595022943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac109df0 0x7ff0ac110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/1595022943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac1094a0 0x7ff0ac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/1595022943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac103b60 0x7ff0ac103f70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/1595022943 >> 172.21.15.29:0/1595022943 conn(0x7ff0ac07b850 msgr2=0x7ff0ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/1595022943 shutdown_connections 2024-09-21T01:47:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.967+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/1595022943 wait complete. 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.968+0000 7ff0b1d7a640 1 Processor -- start 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.968+0000 7ff0b1d7a640 1 -- start start 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 0x7ff0ac19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac1094a0 0x7ff0ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac113b30 con 0x7ff0ac109df0 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1139b0 con 0x7ff0ac1094a0 2024-09-21T01:47:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0b1d7a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac113cb0 con 0x7ff0ac103b60 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 0x7ff0ac19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33046/0 (socket says 172.21.15.29:33046) 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0abfff640 1 -- 172.21.15.29:0/901739739 learned_addr learned my addr 172.21.15.29:0/901739739 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0aaffd640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac1094a0 0x7ff0ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 -- 172.21.15.29:0/901739739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac1094a0 msgr2=0x7ff0ac19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac1094a0 0x7ff0ac19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 -- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 msgr2=0x7ff0ac19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0ab7fe640 1 -- 172.21.15.29:0/901739739 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac1a0510 con 0x7ff0ac103b60 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.969+0000 7ff0abfff640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0ab7fe640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 0x7ff0ac19a5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff09c02a930 tx=0x7ff09c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c03f030 con 0x7ff0ac103b60 2024-09-21T01:47:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac1a07a0 con 0x7ff0ac103b60 2024-09-21T01:47:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c003aa0 con 0x7ff0ac103b60 2024-09-21T01:47:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c035070 con 0x7ff0ac103b60 2024-09-21T01:47:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.970+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1a6f60 con 0x7ff0ac103b60 2024-09-21T01:47:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.971+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac103f70 con 0x7ff0ac103b60 2024-09-21T01:47:15.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.973+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff09c042030 con 0x7ff0ac103b60 2024-09-21T01:47:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.974+0000 7ff0a8ff9640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 0x7ff0940b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.974+0000 7ff0aaffd640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 0x7ff0940b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.974+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff09c107cd0 con 0x7ff0ac103b60 2024-09-21T01:47:15.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.975+0000 7ff0aaffd640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 0x7ff0940b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac109200 tx=0x7ff0a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:15.977+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff09c0d27f0 con 0x7ff0ac103b60 2024-09-21T01:47:16.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.171+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff0ac10bbc0 con 0x7ff0ac103b60 2024-09-21T01:47:16.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.171+0000 7ff0a8ff9640 1 -- 172.21.15.29:0/901739739 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff09c0d7730 con 0x7ff0ac103b60 2024-09-21T01:47:16.172 INFO:teuthology.orchestra.run.smithi029.stdout:201863462958 2024-09-21T01:47:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 msgr2=0x7ff0940b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 0x7ff0940b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac109200 tx=0x7ff0a000b390 comp rx=0 tx=0).stop 2024-09-21T01:47:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 msgr2=0x7ff0ac19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 0x7ff0ac19a5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff09c02a930 tx=0x7ff09c004270 comp rx=0 tx=0).stop 2024-09-21T01:47:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 shutdown_connections 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff0940b65a0 0x7ff0940b8ab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0ac109df0 0x7ff0ac19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0ac1094a0 0x7ff0ac19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 --2- 172.21.15.29:0/901739739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ac103b60 0x7ff0ac19a5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 >> 172.21.15.29:0/901739739 conn(0x7ff0ac07b850 msgr2=0x7ff0ac107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 shutdown_connections 2024-09-21T01:47:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.174+0000 7ff0b1d7a640 1 -- 172.21.15.29:0/901739739 wait complete. 2024-09-21T01:47:16.271 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1618610188' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:47:16.271 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2034387020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:47:16.458 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266730 for osd.8 2024-09-21T01:47:16.458 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1618610188' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:47:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2034387020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:47:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1618610188' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:47:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2034387020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:47:16.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 -- 172.21.15.29:0/3193895012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 msgr2=0x7fea40110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 --2- 172.21.15.29:0/3193895012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea40110300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fea38009ca0 tx=0x7fea3802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 -- 172.21.15.29:0/3193895012 shutdown_connections 2024-09-21T01:47:16.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 --2- 172.21.15.29:0/3193895012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea40110300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 --2- 172.21.15.29:0/3193895012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea40105300 0x7fea4010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 --2- 172.21.15.29:0/3193895012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea401049b0 0x7fea40104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 -- 172.21.15.29:0/3193895012 >> 172.21.15.29:0/3193895012 conn(0x7fea40079730 msgr2=0x7fea40079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 -- 172.21.15.29:0/3193895012 shutdown_connections 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.637+0000 7fea45547640 1 -- 172.21.15.29:0/3193895012 wait complete. 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 Processor -- start 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 -- start start 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 0x7fea4019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea40105300 0x7fea4019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fea40113a70 con 0x7fea4010de10 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fea401138f0 con 0x7fea40105300 2024-09-21T01:47:16.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea45547640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea40113bf0 con 0x7fea401049b0 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33074/0 (socket says 172.21.15.29:33074) 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea3f7fe640 1 -- 172.21.15.29:0/3290171839 learned_addr learned my addr 172.21.15.29:0/3290171839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea3e7fc640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea40105300 0x7fea4019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.638+0000 7fea3effd640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 0x7fea4019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 -- 172.21.15.29:0/3290171839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 msgr2=0x7fea4019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 0x7fea4019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 -- 172.21.15.29:0/3290171839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea40105300 msgr2=0x7fea4019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea40105300 0x7fea4019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 -- 172.21.15.29:0/3290171839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea401a0390 con 0x7fea4010de10 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3effd640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 0x7fea4019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea3f7fe640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea38005cc0 tx=0x7fea3802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38003bf0 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea401a0620 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea38003d90 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3802fbc0 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.639+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401a6de0 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.640+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea40104dc0 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.643+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fea3802fd60 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.644+0000 7fea27fff640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 0x7fea280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.644+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fea38108b80 con 0x7fea4010de10 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.644+0000 7fea3effd640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 0x7fea280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.645+0000 7fea3effd640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 0x7fea280b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea4006bc60 tx=0x7fea3000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.646+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea380cf260 con 0x7fea4010de10 2024-09-21T01:47:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.841+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fea40108980 con 0x7fea4010de10 2024-09-21T01:47:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.842+0000 7fea27fff640 1 -- 172.21.15.29:0/3290171839 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea380d3640 con 0x7fea4010de10 2024-09-21T01:47:16.843 INFO:teuthology.orchestra.run.smithi029.stdout:249108103205 2024-09-21T01:47:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 msgr2=0x7fea280b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 0x7fea280b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea4006bc60 tx=0x7fea3000a560 comp rx=0 tx=0).stop 2024-09-21T01:47:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 msgr2=0x7fea4019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea38005cc0 tx=0x7fea3802f380 comp rx=0 tx=0).stop 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 shutdown_connections 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea280b65a0 0x7fea280b8ab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea4010de10 0x7fea4019fc60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea40105300 0x7fea4019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 --2- 172.21.15.29:0/3290171839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea401049b0 0x7fea4019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 >> 172.21.15.29:0/3290171839 conn(0x7fea40079730 msgr2=0x7fea40100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:16.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 shutdown_connections 2024-09-21T01:47:16.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.844+0000 7fea45547640 1 -- 172.21.15.29:0/3290171839 wait complete. 2024-09-21T01:47:17.472 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:17 smithi029 ceph-mon[31228]: pgmap v328: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:17.472 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/901739739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:47:17.472 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3290171839' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:47:17.472 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4057338251' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:47:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:17 smithi190 ceph-mon[34368]: pgmap v328: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/901739739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:47:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3290171839' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:47:17.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4057338251' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:47:17.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462956 got 201863462958 for osd.7 2024-09-21T01:47:17.500 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:17 smithi066 ceph-mon[35001]: pgmap v328: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/901739739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:47:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3290171839' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:47:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4057338251' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:47:18.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215169 got 103079215170 for osd.3 2024-09-21T01:47:18.663 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 -- 172.21.15.29:0/2404635396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 msgr2=0x7f13a410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 --2- 172.21.15.29:0/2404635396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a410d880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1394009900 tx=0x7f139402a410 comp rx=0 tx=0).stop 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 -- 172.21.15.29:0/2404635396 shutdown_connections 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 --2- 172.21.15.29:0/2404635396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 0x7f13a4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 --2- 172.21.15.29:0/2404635396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a410d880 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 --2- 172.21.15.29:0/2404635396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 0x7f13a4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.907+0000 7f13aba2f640 1 -- 172.21.15.29:0/2404635396 >> 172.21.15.29:0/2404635396 conn(0x7f13a4079750 msgr2=0x7f13a4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.908+0000 7f13aba2f640 1 -- 172.21.15.29:0/2404635396 shutdown_connections 2024-09-21T01:47:18.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.908+0000 7f13aba2f640 1 -- 172.21.15.29:0/2404635396 wait complete. 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.908+0000 7f13aba2f640 1 Processor -- start 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.908+0000 7f13aba2f640 1 -- start start 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 0x7f13a4102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 0x7f13a41031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4113aa0 con 0x7f13a4105330 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4113920 con 0x7f13a410de40 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13aba2f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4113c20 con 0x7f13a41049e0 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33096/0 (socket says 172.21.15.29:33096) 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 -- 172.21.15.29:0/4057338251 learned_addr learned my addr 172.21.15.29:0/4057338251 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a9fcc640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 0x7f13a41031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a97cb640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 0x7f13a4102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 -- 172.21.15.29:0/4057338251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 msgr2=0x7f13a4102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 0x7f13a4102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 -- 172.21.15.29:0/4057338251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 msgr2=0x7f13a41031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 0x7f13a41031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 -- 172.21.15.29:0/4057338251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41a8830 con 0x7f13a4105330 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.909+0000 7f13a8fca640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f139402a930 tx=0x7f1394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stdout:34359738449 2024-09-21T01:47:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.910+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139403f030 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.910+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a41a8ac0 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.910+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1394003aa0 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.910+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394035070 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.910+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41a8ff0 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.911+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a406c280 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.912+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1394042030 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.913+0000 7f139a7fc640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 0x7f13900b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.913+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1394107f00 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.913+0000 7f13a97cb640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 0x7f13900b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.914+0000 7f13a97cb640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 0x7f13900b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13a00058c0 tx=0x7f13a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:16.916+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13940d2a20 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.106+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f13a4104df0 con 0x7f13a4105330 2024-09-21T01:47:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.107+0000 7f139a7fc640 1 -- 172.21.15.29:0/4057338251 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13a4104df0 con 0x7f13a4105330 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 msgr2=0x7f13900b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 0x7f13900b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13a00058c0 tx=0x7f13a000a2d0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 msgr2=0x7f13a4102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f139402a930 tx=0x7f1394004270 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 shutdown_connections 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13900b65a0 0x7f13900b8ab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13a410de40 0x7f13a41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13a4105330 0x7f13a4102c60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 --2- 172.21.15.29:0/4057338251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13a41049e0 0x7f13a4102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 >> 172.21.15.29:0/4057338251 conn(0x7f13a4079750 msgr2=0x7f13a40ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 shutdown_connections 2024-09-21T01:47:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:17.109+0000 7f13aba2f640 1 -- 172.21.15.29:0/4057338251 wait complete. 2024-09-21T01:47:19.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 -- 172.21.15.29:0/2945289871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 msgr2=0x7fdad4112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:19.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 --2- 172.21.15.29:0/2945289871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad4112560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009ca0 tx=0x7fdacc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 -- 172.21.15.29:0/2945289871 shutdown_connections 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 --2- 172.21.15.29:0/2945289871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad4112560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 --2- 172.21.15.29:0/2945289871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 0x7fdad410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 --2- 172.21.15.29:0/2945289871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad4075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 -- 172.21.15.29:0/2945289871 >> 172.21.15.29:0/2945289871 conn(0x7fdad407b850 msgr2=0x7fdad407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 -- 172.21.15.29:0/2945289871 shutdown_connections 2024-09-21T01:47:19.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.195+0000 7fdadb154640 1 -- 172.21.15.29:0/2945289871 wait complete. 2024-09-21T01:47:19.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 Processor -- start 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 -- start start 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 0x7fdad419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4115c80 con 0x7fdad4110140 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4115b00 con 0x7fdad4075730 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdadb154640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4115e00 con 0x7fdad4076080 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdad96f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdad96f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33122/0 (socket says 172.21.15.29:33122) 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdad96f1640 1 -- 172.21.15.29:0/4010337641 learned_addr learned my addr 172.21.15.29:0/4010337641 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdad3fff640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 0x7fdad419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:19.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.196+0000 7fdad8ef0640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 -- 172.21.15.29:0/4010337641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 msgr2=0x7fdad419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 0x7fdad419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 -- 172.21.15.29:0/4010337641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 msgr2=0x7fdad419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 -- 172.21.15.29:0/4010337641 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41a2640 con 0x7fdad4110140 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad8ef0640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad96f1640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0098d0 tx=0x7fdacc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc003b70 con 0x7fdad4110140 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdacc003d10 con 0x7fdad4110140 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41a28d0 con 0x7fdad4110140 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.197+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc02fa20 con 0x7fdad4110140 2024-09-21T01:47:19.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.198+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a9090 con 0x7fdad4110140 2024-09-21T01:47:19.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.198+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4075b40 con 0x7fdad4110140 2024-09-21T01:47:19.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.200+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fdacc02fbc0 con 0x7fdad4110140 2024-09-21T01:47:19.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.201+0000 7fdad1ffb640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 0x7fdac40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:19.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.202+0000 7fdad8ef0640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 0x7fdac40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:19.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.202+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdacc108ae0 con 0x7fdad4110140 2024-09-21T01:47:19.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.202+0000 7fdad8ef0640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 0x7fdac40b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009c20 tx=0x7fdac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.205+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdacc0cf220 con 0x7fdad4110140 2024-09-21T01:47:19.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:19 smithi029 ceph-mon[31228]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:19.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.401+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdad419d800 con 0x7fdad4110140 2024-09-21T01:47:19.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.401+0000 7fdad1ffb640 1 -- 172.21.15.29:0/4010337641 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdad419d800 con 0x7fdad4110140 2024-09-21T01:47:19.401 INFO:teuthology.orchestra.run.smithi029.stdout:128849018942 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.403+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 msgr2=0x7fdac40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.403+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 0x7fdac40b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009c20 tx=0x7fdac8005f90 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.403+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 msgr2=0x7fdad41a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.403+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0098d0 tx=0x7fdacc0317b0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 shutdown_connections 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdac40b65a0 0x7fdac40b8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdad4110140 0x7fdad41a1f10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad4076080 0x7fdad419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 --2- 172.21.15.29:0/4010337641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdad4075730 0x7fdad419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:19.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 >> 172.21.15.29:0/4010337641 conn(0x7fdad407b850 msgr2=0x7fdad4109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:19.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 shutdown_connections 2024-09-21T01:47:19.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:19.404+0000 7fdadb154640 1 -- 172.21.15.29:0/4010337641 wait complete. 2024-09-21T01:47:19.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:19 smithi190 ceph-mon[34368]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:19 smithi066 ceph-mon[35001]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:19.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103205 for osd.9 2024-09-21T01:47:19.710 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:20.123 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822713 for osd.5 2024-09-21T01:47:20.123 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:20 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4010337641' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:47:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:20 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4010337641' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:47:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:20 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4010337641' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:47:21.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411401 for osd.2 2024-09-21T01:47:21.019 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:21 smithi029 ceph-mon[31228]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:21 smithi190 ceph-mon[34368]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:21 smithi066 ceph-mon[35001]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:21.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776152 got 292057776155 for osd.11 2024-09-21T01:47:21.948 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:21.996 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:22.483 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2024-09-21T01:47:22.483 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:23.012 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018939 got 128849018942 for osd.4 2024-09-21T01:47:23.013 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:23.291 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:23 smithi029 ceph-mon[31228]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:23 smithi190 ceph-mon[34368]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:23 smithi066 ceph-mon[35001]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 -- 172.21.15.29:0/1589700818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 msgr2=0x7f4a9c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1589700818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c102e80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a9802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 -- 172.21.15.29:0/1589700818 shutdown_connections 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1589700818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c108d00 0x7f4a9c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1589700818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c1083b0 0x7f4a9c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1589700818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c102e80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 -- 172.21.15.29:0/1589700818 >> 172.21.15.29:0/1589700818 conn(0x7f4a9c079730 msgr2=0x7f4a9c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 -- 172.21.15.29:0/1589700818 shutdown_connections 2024-09-21T01:47:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.563+0000 7f4aa3684640 1 -- 172.21.15.29:0/1589700818 wait complete. 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 Processor -- start 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 -- start start 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c1083b0 0x7f4a9c1047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 0x7f4a9c104ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1129d0 con 0x7f4a9c102a70 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112850 con 0x7f4a9c108d00 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa3684640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112b50 con 0x7f4a9c1083b0 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa1420640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa1420640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58092/0 (socket says 172.21.15.29:58092) 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa1420640 1 -- 172.21.15.29:0/1050893942 learned_addr learned my addr 172.21.15.29:0/1050893942 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.564+0000 7f4aa0c1f640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c1083b0 0x7f4a9c1047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1c21640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 0x7f4a9c104ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 -- 172.21.15.29:0/1050893942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c1083b0 msgr2=0x7f4a9c1047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c1083b0 0x7f4a9c1047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 -- 172.21.15.29:0/1050893942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 msgr2=0x7f4a9c104ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 0x7f4a9c104ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 -- 172.21.15.29:0/1050893942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c1a8790 con 0x7f4a9c102a70 2024-09-21T01:47:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1c21640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 0x7f4a9c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa1420640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005940 tx=0x7f4a980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9803f030 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a98003cb0 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1a8a20 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98035070 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.565+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a8f80 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.566+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c103cf0 con 0x7f4a9c102a70 2024-09-21T01:47:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.568+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4a98042030 con 0x7f4a9c102a70 2024-09-21T01:47:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.569+0000 7f4a927fc640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 0x7f4a880b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.569+0000 7f4aa0c1f640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 0x7f4a880b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.569+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4a98107ff0 con 0x7f4a9c102a70 2024-09-21T01:47:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.569+0000 7f4aa0c1f640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 0x7f4a880b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c079cd0 tx=0x7f4a8c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.572+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a980ce710 con 0x7f4a9c102a70 2024-09-21T01:47:23.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.772+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f4a9c105560 con 0x7f4a9c102a70 2024-09-21T01:47:23.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.772+0000 7f4a927fc640 1 -- 172.21.15.29:0/1050893942 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4a980d2af0 con 0x7f4a9c102a70 2024-09-21T01:47:23.773 INFO:teuthology.orchestra.run.smithi029.stdout:270582939681 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 msgr2=0x7f4a880b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 0x7f4a880b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c079cd0 tx=0x7f4a8c00b390 comp rx=0 tx=0).stop 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 msgr2=0x7f4a9c104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005940 tx=0x7f4a980042f0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 shutdown_connections 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a880b65a0 0x7f4a880b8ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c108d00 0x7f4a9c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.774+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a9c1083b0 0x7f4a9c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.775+0000 7f4aa3684640 1 --2- 172.21.15.29:0/1050893942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a9c102a70 0x7f4a9c104260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.775+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 >> 172.21.15.29:0/1050893942 conn(0x7f4a9c079730 msgr2=0x7f4a9c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.775+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 shutdown_connections 2024-09-21T01:47:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:23.775+0000 7f4aa3684640 1 -- 172.21.15.29:0/1050893942 wait complete. 2024-09-21T01:47:24.523 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939677 got 270582939681 for osd.10 2024-09-21T01:47:24.523 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:24.523 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T01:47:24.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:47:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1050893942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:47:25.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1050893942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:47:25.058 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1050893942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:47:25.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 -- 172.21.15.29:0/686952388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5604102ae0 msgr2=0x7f5604102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 --2- 172.21.15.29:0/686952388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5604102ae0 0x7f5604102ef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560002a410 comp rx=0 tx=0).stop 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 -- 172.21.15.29:0/686952388 shutdown_connections 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 --2- 172.21.15.29:0/686952388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 0x7f560410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 --2- 172.21.15.29:0/686952388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56041083d0 0x7f56041087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 --2- 172.21.15.29:0/686952388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5604102ae0 0x7f5604102ef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.296+0000 7f560b2eb640 1 -- 172.21.15.29:0/686952388 >> 172.21.15.29:0/686952388 conn(0x7f5604079760 msgr2=0x7f5604079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.297+0000 7f560b2eb640 1 -- 172.21.15.29:0/686952388 shutdown_connections 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.297+0000 7f560b2eb640 1 -- 172.21.15.29:0/686952388 wait complete. 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.297+0000 7f560b2eb640 1 Processor -- start 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.297+0000 7f560b2eb640 1 -- start start 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 0x7f5604199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 0x7f560419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5604112990 con 0x7f56041083d0 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5604112810 con 0x7f5604108d20 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5604112b10 con 0x7f5604102ae0 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:25.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58116/0 (socket says 172.21.15.29:58116) 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 -- 172.21.15.29:0/3520000986 learned_addr learned my addr 172.21.15.29:0/3520000986 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5609087640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 0x7f5604199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5609888640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 0x7f560419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 -- 172.21.15.29:0/3520000986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 msgr2=0x7f5604199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 0x7f5604199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 -- 172.21.15.29:0/3520000986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 msgr2=0x7f560419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 0x7f560419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 -- 172.21.15.29:0/3520000986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419f2d0 con 0x7f56041083d0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5609087640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 0x7f5604199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f5608886640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55f400d770 tx=0x7f55f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f56041083d0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f4009c30 con 0x7f56041083d0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419f5c0 con 0x7f56041083d0 2024-09-21T01:47:25.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400f620 con 0x7f56041083d0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.298+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a5d20 con 0x7f56041083d0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.299+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604103d10 con 0x7f56041083d0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.301+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f56041083d0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.302+0000 7f55fa7fc640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 0x7f55f00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.302+0000 7f5609087640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 0x7f55f00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.302+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f55f40e7010 con 0x7f56041083d0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.302+0000 7f5609087640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 0x7f55f00b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5600031440 tx=0x7f5600009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:25.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.305+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55f4009de0 con 0x7f56041083d0 2024-09-21T01:47:25.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.490+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f560410ab50 con 0x7f55f00b65a0 2024-09-21T01:47:25.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.495+0000 7f55fa7fc640 1 -- 172.21.15.29:0/3520000986 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258750 (secure 0 0 0) 0x7f560410ab50 con 0x7f55f00b65a0 2024-09-21T01:47:25.497 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:25.498 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:47:25.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 msgr2=0x7f55f00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:25.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 0x7f55f00b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5600031440 tx=0x7f5600009680 comp rx=0 tx=0).stop 2024-09-21T01:47:25.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 msgr2=0x7f5604199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:25.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55f400d770 tx=0x7f55f400dc30 comp rx=0 tx=0).stop 2024-09-21T01:47:25.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 shutdown_connections 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f55f00b65a0 0x7f55f00b8ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5604108d20 0x7f560419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56041083d0 0x7f5604199900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 --2- 172.21.15.29:0/3520000986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5604102ae0 0x7f5604199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 >> 172.21.15.29:0/3520000986 conn(0x7f5604079760 msgr2=0x7f56041015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 shutdown_connections 2024-09-21T01:47:25.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:25.498+0000 7f560b2eb640 1 -- 172.21.15.29:0/3520000986 wait complete. 2024-09-21T01:47:25.878 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:25 smithi029 ceph-mon[31228]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:25 smithi066 ceph-mon[35001]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:25 smithi190 ceph-mon[34368]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:26.238 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-21T01:47:24.109002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661317,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":725,"num_read_kb":470,"num_write":367,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402768,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539056,"statfs":{"total":1151940427776,"available":1151527993344,"internally_reserved":0,"allocated":12165120,"data_stored":6755295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003457"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225240+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:46:33.225240+0000","last_peered":"2024-09-21T01:46:33.225240+0000","last_clean":"2024-09-21T01:46:33.225240+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:46:33.225240+0000","last_undegraded":"2024-09-21T01:46:33.225240+0000","last_fullsized":"2024-09-21T01:46:33.225240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260760+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:46:33.260760+0000","last_peered":"2024-09-21T01:46:33.260760+0000","last_clean":"2024-09-21T01:46:33.260760+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:46:33.260760+0000","last_undegraded":"2024-09-21T01:46:33.260760+0000","last_fullsized":"2024-09-21T01:46:33.260760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260728+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:46:33.260728+0000","last_peered":"2024-09-21T01:46:33.260728+0000","last_clean":"2024-09-21T01:46:33.260728+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:46:33.260728+0000","last_undegraded":"2024-09-21T01:46:33.260728+0000","last_fullsized":"2024-09-21T01:46:33.260728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234921+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:46:33.234921+0000","last_peered":"2024-09-21T01:46:33.234921+0000","last_clean":"2024-09-21T01:46:33.234921+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:46:33.234921+0000","last_undegraded":"2024-09-21T01:46:33.234921+0000","last_fullsized":"2024-09-21T01:46:33.234921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228187+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:46:33.228187+0000","last_peered":"2024-09-21T01:46:33.228187+0000","last_clean":"2024-09-21T01:46:33.228187+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:46:33.228187+0000","last_undegraded":"2024-09-21T01:46:33.228187+0000","last_fullsized":"2024-09-21T01:46:33.228187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225298+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:46:33.225298+0000","last_peered":"2024-09-21T01:46:33.225298+0000","last_clean":"2024-09-21T01:46:33.225298+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:46:33.225298+0000","last_undegraded":"2024-09-21T01:46:33.225298+0000","last_fullsized":"2024-09-21T01:46:33.225298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231465+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:46:33.231465+0000","last_peered":"2024-09-21T01:46:33.231465+0000","last_clean":"2024-09-21T01:46:33.231465+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:46:33.231465+0000","last_undegraded":"2024-09-21T01:46:33.231465+0000","last_fullsized":"2024-09-21T01:46:33.231465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231501+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:46:33.231501+0000","last_peered":"2024-09-21T01:46:33.231501+0000","last_clean":"2024-09-21T01:46:33.231501+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:46:33.231501+0000","last_undegraded":"2024-09-21T01:46:33.231501+0000","last_fullsized":"2024-09-21T01:46:33.231501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228282+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:46:33.228282+0000","last_peered":"2024-09-21T01:46:33.228282+0000","last_clean":"2024-09-21T01:46:33.228282+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:46:33.228282+0000","last_undegraded":"2024-09-21T01:46:33.228282+0000","last_fullsized":"2024-09-21T01:46:33.228282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226004+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:46:33.226004+0000","last_peered":"2024-09-21T01:46:33.226004+0000","last_clean":"2024-09-21T01:46:33.226004+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:46:33.226004+0000","last_undegraded":"2024-09-21T01:46:33.226004+0000","last_fullsized":"2024-09-21T01:46:33.226004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231742+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:46:33.231742+0000","last_peered":"2024-09-21T01:46:33.231742+0000","last_clean":"2024-09-21T01:46:33.231742+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:46:33.231742+0000","last_undegraded":"2024-09-21T01:46:33.231742+0000","last_fullsized":"2024-09-21T01:46:33.231742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237387+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:46:33.237387+0000","last_peered":"2024-09-21T01:46:33.237387+0000","last_clean":"2024-09-21T01:46:33.237387+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:46:33.237387+0000","last_undegraded":"2024-09-21T01:46:33.237387+0000","last_fullsized":"2024-09-21T01:46:33.237387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226996+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:46:33.226996+0000","last_peered":"2024-09-21T01:46:33.226996+0000","last_clean":"2024-09-21T01:46:33.226996+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:46:33.226996+0000","last_undegraded":"2024-09-21T01:46:33.226996+0000","last_fullsized":"2024-09-21T01:46:33.226996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260670+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:46:33.260670+0000","last_peered":"2024-09-21T01:46:33.260670+0000","last_clean":"2024-09-21T01:46:33.260670+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:46:33.260670+0000","last_undegraded":"2024-09-21T01:46:33.260670+0000","last_fullsized":"2024-09-21T01:46:33.260670+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260568+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:46:33.260568+0000","last_peered":"2024-09-21T01:46:33.260568+0000","last_clean":"2024-09-21T01:46:33.260568+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:46:33.260568+0000","last_undegraded":"2024-09-21T01:46:33.260568+0000","last_fullsized":"2024-09-21T01:46:33.260568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227507+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:46:33.227507+0000","last_peered":"2024-09-21T01:46:33.227507+0000","last_clean":"2024-09-21T01:46:33.227507+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:46:33.227507+0000","last_undegraded":"2024-09-21T01:46:33.227507+0000","last_fullsized":"2024-09-21T01:46:33.227507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225461+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:46:33.225461+0000","last_peered":"2024-09-21T01:46:33.225461+0000","last_clean":"2024-09-21T01:46:33.225461+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:46:33.225461+0000","last_undegraded":"2024-09-21T01:46:33.225461+0000","last_fullsized":"2024-09-21T01:46:33.225461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226126+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:46:33.226126+0000","last_peered":"2024-09-21T01:46:33.226126+0000","last_clean":"2024-09-21T01:46:33.226126+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:46:33.226126+0000","last_undegraded":"2024-09-21T01:46:33.226126+0000","last_fullsized":"2024-09-21T01:46:33.226126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230980+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:46:33.230980+0000","last_peered":"2024-09-21T01:46:33.230980+0000","last_clean":"2024-09-21T01:46:33.230980+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:46:33.230980+0000","last_undegraded":"2024-09-21T01:46:33.230980+0000","last_fullsized":"2024-09-21T01:46:33.230980+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224911+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:46:33.224911+0000","last_peered":"2024-09-21T01:46:33.224911+0000","last_clean":"2024-09-21T01:46:33.224911+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:46:33.224911+0000","last_undegraded":"2024-09-21T01:46:33.224911+0000","last_fullsized":"2024-09-21T01:46:33.224911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227097+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:46:33.227097+0000","last_peered":"2024-09-21T01:46:33.227097+0000","last_clean":"2024-09-21T01:46:33.227097+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:46:33.227097+0000","last_undegraded":"2024-09-21T01:46:33.227097+0000","last_fullsized":"2024-09-21T01:46:33.227097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227796+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:46:33.227796+0000","last_peered":"2024-09-21T01:46:33.227796+0000","last_clean":"2024-09-21T01:46:33.227796+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:46:33.227796+0000","last_undegraded":"2024-09-21T01:46:33.227796+0000","last_fullsized":"2024-09-21T01:46:33.227796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223435+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:46:33.223435+0000","last_peered":"2024-09-21T01:46:33.223435+0000","last_clean":"2024-09-21T01:46:33.223435+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:46:33.223435+0000","last_undegraded":"2024-09-21T01:46:33.223435+0000","last_fullsized":"2024-09-21T01:46:33.223435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227979+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:46:33.227979+0000","last_peered":"2024-09-21T01:46:33.227979+0000","last_clean":"2024-09-21T01:46:33.227979+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:46:33.227979+0000","last_undegraded":"2024-09-21T01:46:33.227979+0000","last_fullsized":"2024-09-21T01:46:33.227979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224001+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:46:33.224001+0000","last_peered":"2024-09-21T01:46:33.224001+0000","last_clean":"2024-09-21T01:46:33.224001+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:46:33.224001+0000","last_undegraded":"2024-09-21T01:46:33.224001+0000","last_fullsized":"2024-09-21T01:46:33.224001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232919+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:46:33.232919+0000","last_peered":"2024-09-21T01:46:33.232919+0000","last_clean":"2024-09-21T01:46:33.232919+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:46:33.232919+0000","last_undegraded":"2024-09-21T01:46:33.232919+0000","last_fullsized":"2024-09-21T01:46:33.232919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224984+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:46:33.224984+0000","last_peered":"2024-09-21T01:46:33.224984+0000","last_clean":"2024-09-21T01:46:33.224984+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:46:33.224984+0000","last_undegraded":"2024-09-21T01:46:33.224984+0000","last_fullsized":"2024-09-21T01:46:33.224984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226981+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:46:33.226981+0000","last_peered":"2024-09-21T01:46:33.226981+0000","last_clean":"2024-09-21T01:46:33.226981+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:46:33.226981+0000","last_undegraded":"2024-09-21T01:46:33.226981+0000","last_fullsized":"2024-09-21T01:46:33.226981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260782+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:46:33.260782+0000","last_peered":"2024-09-21T01:46:33.260782+0000","last_clean":"2024-09-21T01:46:33.260782+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:46:33.260782+0000","last_undegraded":"2024-09-21T01:46:33.260782+0000","last_fullsized":"2024-09-21T01:46:33.260782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230552+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:46:33.230552+0000","last_peered":"2024-09-21T01:46:33.230552+0000","last_clean":"2024-09-21T01:46:33.230552+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:46:33.230552+0000","last_undegraded":"2024-09-21T01:46:33.230552+0000","last_fullsized":"2024-09-21T01:46:33.230552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230632+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:46:33.230632+0000","last_peered":"2024-09-21T01:46:33.230632+0000","last_clean":"2024-09-21T01:46:33.230632+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:46:33.230632+0000","last_undegraded":"2024-09-21T01:46:33.230632+0000","last_fullsized":"2024-09-21T01:46:33.230632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224723+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:46:33.224723+0000","last_peered":"2024-09-21T01:46:33.224723+0000","last_clean":"2024-09-21T01:46:33.224723+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:46:33.224723+0000","last_undegraded":"2024-09-21T01:46:33.224723+0000","last_fullsized":"2024-09-21T01:46:33.224723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228787+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:46:33.228787+0000","last_peered":"2024-09-21T01:46:33.228787+0000","last_clean":"2024-09-21T01:46:33.228787+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:46:33.228787+0000","last_undegraded":"2024-09-21T01:46:33.228787+0000","last_fullsized":"2024-09-21T01:46:33.228787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:21.303664+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:47:21.303664+0000","last_peered":"2024-09-21T01:47:21.303664+0000","last_clean":"2024-09-21T01:47:21.303664+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:47:21.303664+0000","last_undegraded":"2024-09-21T01:47:21.303664+0000","last_fullsized":"2024-09-21T01:47:21.303664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228873+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:46:33.228873+0000","last_peered":"2024-09-21T01:46:33.228873+0000","last_clean":"2024-09-21T01:46:33.228873+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:46:33.228873+0000","last_undegraded":"2024-09-21T01:46:33.228873+0000","last_fullsized":"2024-09-21T01:46:33.228873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223419+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:46:33.223419+0000","last_peered":"2024-09-21T01:46:33.223419+0000","last_clean":"2024-09-21T01:46:33.223419+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:46:33.223419+0000","last_undegraded":"2024-09-21T01:46:33.223419+0000","last_fullsized":"2024-09-21T01:46:33.223419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225008+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:46:33.225008+0000","last_peered":"2024-09-21T01:46:33.225008+0000","last_clean":"2024-09-21T01:46:33.225008+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:46:33.225008+0000","last_undegraded":"2024-09-21T01:46:33.225008+0000","last_fullsized":"2024-09-21T01:46:33.225008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:23.305965+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:47:23.305965+0000","last_peered":"2024-09-21T01:47:23.305965+0000","last_clean":"2024-09-21T01:47:23.305965+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:47:23.305965+0000","last_undegraded":"2024-09-21T01:47:23.305965+0000","last_fullsized":"2024-09-21T01:47:23.305965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228320+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:46:33.228320+0000","last_peered":"2024-09-21T01:46:33.228320+0000","last_clean":"2024-09-21T01:46:33.228320+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:46:33.228320+0000","last_undegraded":"2024-09-21T01:46:33.228320+0000","last_fullsized":"2024-09-21T01:46:33.228320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260933+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:46:33.260933+0000","last_peered":"2024-09-21T01:46:33.260933+0000","last_clean":"2024-09-21T01:46:33.260933+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:46:33.260933+0000","last_undegraded":"2024-09-21T01:46:33.260933+0000","last_fullsized":"2024-09-21T01:46:33.260933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227879+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:46:33.227879+0000","last_peered":"2024-09-21T01:46:33.227879+0000","last_clean":"2024-09-21T01:46:33.227879+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:46:33.227879+0000","last_undegraded":"2024-09-21T01:46:33.227879+0000","last_fullsized":"2024-09-21T01:46:33.227879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:23.306088+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:47:23.306088+0000","last_peered":"2024-09-21T01:47:23.306088+0000","last_clean":"2024-09-21T01:47:23.306088+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:47:23.306088+0000","last_undegraded":"2024-09-21T01:47:23.306088+0000","last_fullsized":"2024-09-21T01:47:23.306088+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":128,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229895+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:46:33.229895+0000","last_peered":"2024-09-21T01:46:33.229895+0000","last_clean":"2024-09-21T01:46:33.229895+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:46:33.229895+0000","last_undegraded":"2024-09-21T01:46:33.229895+0000","last_fullsized":"2024-09-21T01:46:33.229895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226066+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:46:33.226066+0000","last_peered":"2024-09-21T01:46:33.226066+0000","last_clean":"2024-09-21T01:46:33.226066+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:46:33.226066+0000","last_undegraded":"2024-09-21T01:46:33.226066+0000","last_fullsized":"2024-09-21T01:46:33.226066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232179+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:46:33.232179+0000","last_peered":"2024-09-21T01:46:33.232179+0000","last_clean":"2024-09-21T01:46:33.232179+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:46:33.232179+0000","last_undegraded":"2024-09-21T01:46:33.232179+0000","last_fullsized":"2024-09-21T01:46:33.232179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:21.303840+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:47:21.303840+0000","last_peered":"2024-09-21T01:47:21.303840+0000","last_clean":"2024-09-21T01:47:21.303840+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:47:21.303840+0000","last_undegraded":"2024-09-21T01:47:21.303840+0000","last_fullsized":"2024-09-21T01:47:21.303840+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231720+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:46:33.231720+0000","last_peered":"2024-09-21T01:46:33.231720+0000","last_clean":"2024-09-21T01:46:33.231720+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:46:33.231720+0000","last_undegraded":"2024-09-21T01:46:33.231720+0000","last_fullsized":"2024-09-21T01:46:33.231720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227856+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:46:33.227856+0000","last_peered":"2024-09-21T01:46:33.227856+0000","last_clean":"2024-09-21T01:46:33.227856+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:46:33.227856+0000","last_undegraded":"2024-09-21T01:46:33.227856+0000","last_fullsized":"2024-09-21T01:46:33.227856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234181+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:46:33.234181+0000","last_peered":"2024-09-21T01:46:33.234181+0000","last_clean":"2024-09-21T01:46:33.234181+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:46:33.234181+0000","last_undegraded":"2024-09-21T01:46:33.234181+0000","last_fullsized":"2024-09-21T01:46:33.234181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231777+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:46:33.231777+0000","last_peered":"2024-09-21T01:46:33.231777+0000","last_clean":"2024-09-21T01:46:33.231777+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:46:33.231777+0000","last_undegraded":"2024-09-21T01:46:33.231777+0000","last_fullsized":"2024-09-21T01:46:33.231777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231571+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:46:33.231571+0000","last_peered":"2024-09-21T01:46:33.231571+0000","last_clean":"2024-09-21T01:46:33.231571+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:46:33.231571+0000","last_undegraded":"2024-09-21T01:46:33.231571+0000","last_fullsized":"2024-09-21T01:46:33.231571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:19.300551+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:47:19.300551+0000","last_peered":"2024-09-21T01:47:19.300551+0000","last_clean":"2024-09-21T01:47:19.300551+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:47:19.300551+0000","last_undegraded":"2024-09-21T01:47:19.300551+0000","last_fullsized":"2024-09-21T01:47:19.300551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231672+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:46:33.231672+0000","last_peered":"2024-09-21T01:46:33.231672+0000","last_clean":"2024-09-21T01:46:33.231672+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:46:33.231672+0000","last_undegraded":"2024-09-21T01:46:33.231672+0000","last_fullsized":"2024-09-21T01:46:33.231672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231609+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231609+0000","last_peered":"2024-09-21T01:46:33.231609+0000","last_clean":"2024-09-21T01:46:33.231609+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231609+0000","last_undegraded":"2024-09-21T01:46:33.231609+0000","last_fullsized":"2024-09-21T01:46:33.231609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237072+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237072+0000","last_peered":"2024-09-21T01:46:33.237072+0000","last_clean":"2024-09-21T01:46:33.237072+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237072+0000","last_undegraded":"2024-09-21T01:46:33.237072+0000","last_fullsized":"2024-09-21T01:46:33.237072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223710+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:46:33.223710+0000","last_peered":"2024-09-21T01:46:33.223710+0000","last_clean":"2024-09-21T01:46:33.223710+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:46:33.223710+0000","last_undegraded":"2024-09-21T01:46:33.223710+0000","last_fullsized":"2024-09-21T01:46:33.223710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231445+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231445+0000","last_peered":"2024-09-21T01:46:33.231445+0000","last_clean":"2024-09-21T01:46:33.231445+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231445+0000","last_undegraded":"2024-09-21T01:46:33.231445+0000","last_fullsized":"2024-09-21T01:46:33.231445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226072+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:46:33.226072+0000","last_peered":"2024-09-21T01:46:33.226072+0000","last_clean":"2024-09-21T01:46:33.226072+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:46:33.226072+0000","last_undegraded":"2024-09-21T01:46:33.226072+0000","last_fullsized":"2024-09-21T01:46:33.226072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:22.304623+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:47:22.304623+0000","last_peered":"2024-09-21T01:47:22.304623+0000","last_clean":"2024-09-21T01:47:22.304623+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:47:22.304623+0000","last_undegraded":"2024-09-21T01:47:22.304623+0000","last_fullsized":"2024-09-21T01:47:22.304623+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237841+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:46:33.237841+0000","last_peered":"2024-09-21T01:46:33.237841+0000","last_clean":"2024-09-21T01:46:33.237841+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:46:33.237841+0000","last_undegraded":"2024-09-21T01:46:33.237841+0000","last_fullsized":"2024-09-21T01:46:33.237841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"74'6","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:18.298654+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:47:18.298654+0000","last_peered":"2024-09-21T01:47:18.298654+0000","last_clean":"2024-09-21T01:47:18.298654+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:47:18.298654+0000","last_undegraded":"2024-09-21T01:47:18.298654+0000","last_fullsized":"2024-09-21T01:47:18.298654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225616+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:46:33.225616+0000","last_peered":"2024-09-21T01:46:33.225616+0000","last_clean":"2024-09-21T01:46:33.225616+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:46:33.225616+0000","last_undegraded":"2024-09-21T01:46:33.225616+0000","last_fullsized":"2024-09-21T01:46:33.225616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225703+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225703+0000","last_peered":"2024-09-21T01:46:33.225703+0000","last_clean":"2024-09-21T01:46:33.225703+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225703+0000","last_undegraded":"2024-09-21T01:46:33.225703+0000","last_fullsized":"2024-09-21T01:46:33.225703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225654+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:46:33.225654+0000","last_peered":"2024-09-21T01:46:33.225654+0000","last_clean":"2024-09-21T01:46:33.225654+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:46:33.225654+0000","last_undegraded":"2024-09-21T01:46:33.225654+0000","last_fullsized":"2024-09-21T01:46:33.225654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230530+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230530+0000","last_peered":"2024-09-21T01:46:33.230530+0000","last_clean":"2024-09-21T01:46:33.230530+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230530+0000","last_undegraded":"2024-09-21T01:46:33.230530+0000","last_fullsized":"2024-09-21T01:46:33.230530+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227173+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:46:33.227173+0000","last_peered":"2024-09-21T01:46:33.227173+0000","last_clean":"2024-09-21T01:46:33.227173+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:46:33.227173+0000","last_undegraded":"2024-09-21T01:46:33.227173+0000","last_fullsized":"2024-09-21T01:46:33.227173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'14","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:39.858720+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:39.858720+0000","last_peered":"2024-09-21T01:46:39.858720+0000","last_clean":"2024-09-21T01:46:39.858720+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:39.858720+0000","last_undegraded":"2024-09-21T01:46:39.858720+0000","last_fullsized":"2024-09-21T01:46:39.858720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224103+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:46:33.224103+0000","last_peered":"2024-09-21T01:46:33.224103+0000","last_clean":"2024-09-21T01:46:33.224103+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:46:33.224103+0000","last_undegraded":"2024-09-21T01:46:33.224103+0000","last_fullsized":"2024-09-21T01:46:33.224103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237584+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237584+0000","last_peered":"2024-09-21T01:46:33.237584+0000","last_clean":"2024-09-21T01:46:33.237584+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237584+0000","last_undegraded":"2024-09-21T01:46:33.237584+0000","last_fullsized":"2024-09-21T01:46:33.237584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260770+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:46:33.260770+0000","last_peered":"2024-09-21T01:46:33.260770+0000","last_clean":"2024-09-21T01:46:33.260770+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:46:33.260770+0000","last_undegraded":"2024-09-21T01:46:33.260770+0000","last_fullsized":"2024-09-21T01:46:33.260770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228572+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:46:33.228572+0000","last_peered":"2024-09-21T01:46:33.228572+0000","last_clean":"2024-09-21T01:46:33.228572+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:46:33.228572+0000","last_undegraded":"2024-09-21T01:46:33.228572+0000","last_fullsized":"2024-09-21T01:46:33.228572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224910+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:46:33.224910+0000","last_peered":"2024-09-21T01:46:33.224910+0000","last_clean":"2024-09-21T01:46:33.224910+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:46:33.224910+0000","last_undegraded":"2024-09-21T01:46:33.224910+0000","last_fullsized":"2024-09-21T01:46:33.224910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223770+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:46:33.223770+0000","last_peered":"2024-09-21T01:46:33.223770+0000","last_clean":"2024-09-21T01:46:33.223770+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:46:33.223770+0000","last_undegraded":"2024-09-21T01:46:33.223770+0000","last_fullsized":"2024-09-21T01:46:33.223770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231959+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:46:33.231959+0000","last_peered":"2024-09-21T01:46:33.231959+0000","last_clean":"2024-09-21T01:46:33.231959+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:46:33.231959+0000","last_undegraded":"2024-09-21T01:46:33.231959+0000","last_fullsized":"2024-09-21T01:46:33.231959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227958+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:46:33.227958+0000","last_peered":"2024-09-21T01:46:33.227958+0000","last_clean":"2024-09-21T01:46:33.227958+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:46:33.227958+0000","last_undegraded":"2024-09-21T01:46:33.227958+0000","last_fullsized":"2024-09-21T01:46:33.227958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227724+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:46:33.227724+0000","last_peered":"2024-09-21T01:46:33.227724+0000","last_clean":"2024-09-21T01:46:33.227724+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:46:33.227724+0000","last_undegraded":"2024-09-21T01:46:33.227724+0000","last_fullsized":"2024-09-21T01:46:33.227724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260935+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:46:33.260935+0000","last_peered":"2024-09-21T01:46:33.260935+0000","last_clean":"2024-09-21T01:46:33.260935+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:46:33.260935+0000","last_undegraded":"2024-09-21T01:46:33.260935+0000","last_fullsized":"2024-09-21T01:46:33.260935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224309+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224309+0000","last_peered":"2024-09-21T01:46:33.224309+0000","last_clean":"2024-09-21T01:46:33.224309+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224309+0000","last_undegraded":"2024-09-21T01:46:33.224309+0000","last_fullsized":"2024-09-21T01:46:33.224309+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224094+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:46:33.224094+0000","last_peered":"2024-09-21T01:46:33.224094+0000","last_clean":"2024-09-21T01:46:33.224094+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:46:33.224094+0000","last_undegraded":"2024-09-21T01:46:33.224094+0000","last_fullsized":"2024-09-21T01:46:33.224094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:22.303591+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:47:22.303591+0000","last_peered":"2024-09-21T01:47:22.303591+0000","last_clean":"2024-09-21T01:47:22.303591+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:47:22.303591+0000","last_undegraded":"2024-09-21T01:47:22.303591+0000","last_fullsized":"2024-09-21T01:47:22.303591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70715,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":649,"num_read_kb":406,"num_write":242,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":212145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776157,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939682,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1053735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266732,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":991630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":398842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32616,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712536,"statfs":{"total":95995035648,"available":95961636864,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":463486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":463657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:47:26.242 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:47:26.502 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:26.906 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:26 smithi029 ceph-mon[31228]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:27.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.997+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/3556993293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f241094a0 msgr2=0x7f9f241098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:27.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.997+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/3556993293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f241094a0 0x7f9f241098b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f1802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/3556993293 shutdown_connections 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/3556993293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24109df0 0x7f9f24110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/3556993293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f241094a0 0x7f9f241098b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/3556993293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f24103b60 0x7f9f24103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/3556993293 >> 172.21.15.29:0/3556993293 conn(0x7f9f2407b850 msgr2=0x7f9f2407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/3556993293 shutdown_connections 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/3556993293 wait complete. 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 Processor -- start 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.998+0000 7f9f2b5c1640 1 -- start start 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24103b60 0x7f9f2419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 0x7f9f2419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:27.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113ab0 con 0x7f9f24109df0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113930 con 0x7f9f241094a0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2b5c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113c30 con 0x7f9f24103b60 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58126/0 (socket says 172.21.15.29:58126) 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 -- 172.21.15.29:0/1437107440 learned_addr learned my addr 172.21.15.29:0/1437107440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f28b5c640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 0x7f9f2419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f2935d640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24103b60 0x7f9f2419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 -- 172.21.15.29:0/1437107440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24103b60 msgr2=0x7f9f2419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24103b60 0x7f9f2419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 -- 172.21.15.29:0/1437107440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 msgr2=0x7f9f2419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 0x7f9f2419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 -- 172.21.15.29:0/1437107440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a0430 con 0x7f9f24109df0 2024-09-21T01:47:27.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f28b5c640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 0x7f9f2419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f29b5e640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c970 tx=0x7f9f1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c007bb0 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c007d50 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:26.999+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c018770 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.000+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a0720 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.000+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a0be0 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.001+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24103f70 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.002+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9f1c00f990 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.003+0000 7f9f227fc640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 0x7f9f0c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.003+0000 7f9f2935d640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 0x7f9f0c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.003+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9f1c0e74e0 con 0x7f9f24109df0 2024-09-21T01:47:27.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.003+0000 7f9f2935d640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 0x7f9f0c0b8b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005fd0 tx=0x7f9f14009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:27.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.007+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f1c014030 con 0x7f9f24109df0 2024-09-21T01:47:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:26 smithi066 ceph-mon[35001]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:27.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.193+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f24063800 con 0x7f9f0c0b6650 2024-09-21T01:47:27.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.198+0000 7f9f227fc640 1 -- 172.21.15.29:0/1437107440 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258750 (secure 0 0 0) 0x7f9f24063800 con 0x7f9f0c0b6650 2024-09-21T01:47:27.200 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:47:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 msgr2=0x7f9f0c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 0x7f9f0c0b8b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005fd0 tx=0x7f9f14009870 comp rx=0 tx=0).stop 2024-09-21T01:47:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 msgr2=0x7f9f2419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c970 tx=0x7f9f1c00ce30 comp rx=0 tx=0).stop 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 shutdown_connections 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f0c0b6650 0x7f9f0c0b8b60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f24109df0 0x7f9f2419fdc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f241094a0 0x7f9f2419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 --2- 172.21.15.29:0/1437107440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f24103b60 0x7f9f2419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 >> 172.21.15.29:0/1437107440 conn(0x7f9f2407b850 msgr2=0x7f9f24105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 shutdown_connections 2024-09-21T01:47:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:27.201+0000 7f9f2b5c1640 1 -- 172.21.15.29:0/1437107440 wait complete. 2024-09-21T01:47:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:26 smithi190 ceph-mon[34368]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:27 smithi066 ceph-mon[35001]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:28.083 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:27 smithi029 ceph-mon[31228]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:28.121 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-21T01:47:26.109514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661317,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":735,"num_read_kb":470,"num_write":367,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402768,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539056,"statfs":{"total":1151940427776,"available":1151527993344,"internally_reserved":0,"allocated":12165120,"data_stored":6755295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003446"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225240+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:46:33.225240+0000","last_peered":"2024-09-21T01:46:33.225240+0000","last_clean":"2024-09-21T01:46:33.225240+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:46:33.225240+0000","last_undegraded":"2024-09-21T01:46:33.225240+0000","last_fullsized":"2024-09-21T01:46:33.225240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260760+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:46:33.260760+0000","last_peered":"2024-09-21T01:46:33.260760+0000","last_clean":"2024-09-21T01:46:33.260760+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:46:33.260760+0000","last_undegraded":"2024-09-21T01:46:33.260760+0000","last_fullsized":"2024-09-21T01:46:33.260760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260728+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:46:33.260728+0000","last_peered":"2024-09-21T01:46:33.260728+0000","last_clean":"2024-09-21T01:46:33.260728+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:46:33.260728+0000","last_undegraded":"2024-09-21T01:46:33.260728+0000","last_fullsized":"2024-09-21T01:46:33.260728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234921+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:46:33.234921+0000","last_peered":"2024-09-21T01:46:33.234921+0000","last_clean":"2024-09-21T01:46:33.234921+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:46:33.234921+0000","last_undegraded":"2024-09-21T01:46:33.234921+0000","last_fullsized":"2024-09-21T01:46:33.234921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228187+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:46:33.228187+0000","last_peered":"2024-09-21T01:46:33.228187+0000","last_clean":"2024-09-21T01:46:33.228187+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:46:33.228187+0000","last_undegraded":"2024-09-21T01:46:33.228187+0000","last_fullsized":"2024-09-21T01:46:33.228187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225298+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:46:33.225298+0000","last_peered":"2024-09-21T01:46:33.225298+0000","last_clean":"2024-09-21T01:46:33.225298+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:46:33.225298+0000","last_undegraded":"2024-09-21T01:46:33.225298+0000","last_fullsized":"2024-09-21T01:46:33.225298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231465+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:46:33.231465+0000","last_peered":"2024-09-21T01:46:33.231465+0000","last_clean":"2024-09-21T01:46:33.231465+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:46:33.231465+0000","last_undegraded":"2024-09-21T01:46:33.231465+0000","last_fullsized":"2024-09-21T01:46:33.231465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231501+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:46:33.231501+0000","last_peered":"2024-09-21T01:46:33.231501+0000","last_clean":"2024-09-21T01:46:33.231501+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:46:33.231501+0000","last_undegraded":"2024-09-21T01:46:33.231501+0000","last_fullsized":"2024-09-21T01:46:33.231501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228282+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:46:33.228282+0000","last_peered":"2024-09-21T01:46:33.228282+0000","last_clean":"2024-09-21T01:46:33.228282+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:46:33.228282+0000","last_undegraded":"2024-09-21T01:46:33.228282+0000","last_fullsized":"2024-09-21T01:46:33.228282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226004+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:46:33.226004+0000","last_peered":"2024-09-21T01:46:33.226004+0000","last_clean":"2024-09-21T01:46:33.226004+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:46:33.226004+0000","last_undegraded":"2024-09-21T01:46:33.226004+0000","last_fullsized":"2024-09-21T01:46:33.226004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231742+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:46:33.231742+0000","last_peered":"2024-09-21T01:46:33.231742+0000","last_clean":"2024-09-21T01:46:33.231742+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:46:33.231742+0000","last_undegraded":"2024-09-21T01:46:33.231742+0000","last_fullsized":"2024-09-21T01:46:33.231742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237387+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:46:33.237387+0000","last_peered":"2024-09-21T01:46:33.237387+0000","last_clean":"2024-09-21T01:46:33.237387+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:46:33.237387+0000","last_undegraded":"2024-09-21T01:46:33.237387+0000","last_fullsized":"2024-09-21T01:46:33.237387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226996+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:46:33.226996+0000","last_peered":"2024-09-21T01:46:33.226996+0000","last_clean":"2024-09-21T01:46:33.226996+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:46:33.226996+0000","last_undegraded":"2024-09-21T01:46:33.226996+0000","last_fullsized":"2024-09-21T01:46:33.226996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260670+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:46:33.260670+0000","last_peered":"2024-09-21T01:46:33.260670+0000","last_clean":"2024-09-21T01:46:33.260670+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:46:33.260670+0000","last_undegraded":"2024-09-21T01:46:33.260670+0000","last_fullsized":"2024-09-21T01:46:33.260670+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260568+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:46:33.260568+0000","last_peered":"2024-09-21T01:46:33.260568+0000","last_clean":"2024-09-21T01:46:33.260568+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:46:33.260568+0000","last_undegraded":"2024-09-21T01:46:33.260568+0000","last_fullsized":"2024-09-21T01:46:33.260568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227507+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:46:33.227507+0000","last_peered":"2024-09-21T01:46:33.227507+0000","last_clean":"2024-09-21T01:46:33.227507+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:46:33.227507+0000","last_undegraded":"2024-09-21T01:46:33.227507+0000","last_fullsized":"2024-09-21T01:46:33.227507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225461+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:46:33.225461+0000","last_peered":"2024-09-21T01:46:33.225461+0000","last_clean":"2024-09-21T01:46:33.225461+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:46:33.225461+0000","last_undegraded":"2024-09-21T01:46:33.225461+0000","last_fullsized":"2024-09-21T01:46:33.225461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226126+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:46:33.226126+0000","last_peered":"2024-09-21T01:46:33.226126+0000","last_clean":"2024-09-21T01:46:33.226126+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:46:33.226126+0000","last_undegraded":"2024-09-21T01:46:33.226126+0000","last_fullsized":"2024-09-21T01:46:33.226126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230980+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:46:33.230980+0000","last_peered":"2024-09-21T01:46:33.230980+0000","last_clean":"2024-09-21T01:46:33.230980+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:46:33.230980+0000","last_undegraded":"2024-09-21T01:46:33.230980+0000","last_fullsized":"2024-09-21T01:46:33.230980+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224911+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:46:33.224911+0000","last_peered":"2024-09-21T01:46:33.224911+0000","last_clean":"2024-09-21T01:46:33.224911+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:46:33.224911+0000","last_undegraded":"2024-09-21T01:46:33.224911+0000","last_fullsized":"2024-09-21T01:46:33.224911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227097+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:46:33.227097+0000","last_peered":"2024-09-21T01:46:33.227097+0000","last_clean":"2024-09-21T01:46:33.227097+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:46:33.227097+0000","last_undegraded":"2024-09-21T01:46:33.227097+0000","last_fullsized":"2024-09-21T01:46:33.227097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227796+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:46:33.227796+0000","last_peered":"2024-09-21T01:46:33.227796+0000","last_clean":"2024-09-21T01:46:33.227796+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:46:33.227796+0000","last_undegraded":"2024-09-21T01:46:33.227796+0000","last_fullsized":"2024-09-21T01:46:33.227796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223435+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:46:33.223435+0000","last_peered":"2024-09-21T01:46:33.223435+0000","last_clean":"2024-09-21T01:46:33.223435+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:46:33.223435+0000","last_undegraded":"2024-09-21T01:46:33.223435+0000","last_fullsized":"2024-09-21T01:46:33.223435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227979+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:46:33.227979+0000","last_peered":"2024-09-21T01:46:33.227979+0000","last_clean":"2024-09-21T01:46:33.227979+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:46:33.227979+0000","last_undegraded":"2024-09-21T01:46:33.227979+0000","last_fullsized":"2024-09-21T01:46:33.227979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224001+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:46:33.224001+0000","last_peered":"2024-09-21T01:46:33.224001+0000","last_clean":"2024-09-21T01:46:33.224001+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:46:33.224001+0000","last_undegraded":"2024-09-21T01:46:33.224001+0000","last_fullsized":"2024-09-21T01:46:33.224001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232919+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:46:33.232919+0000","last_peered":"2024-09-21T01:46:33.232919+0000","last_clean":"2024-09-21T01:46:33.232919+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:46:33.232919+0000","last_undegraded":"2024-09-21T01:46:33.232919+0000","last_fullsized":"2024-09-21T01:46:33.232919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224984+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:46:33.224984+0000","last_peered":"2024-09-21T01:46:33.224984+0000","last_clean":"2024-09-21T01:46:33.224984+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:46:33.224984+0000","last_undegraded":"2024-09-21T01:46:33.224984+0000","last_fullsized":"2024-09-21T01:46:33.224984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226981+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:46:33.226981+0000","last_peered":"2024-09-21T01:46:33.226981+0000","last_clean":"2024-09-21T01:46:33.226981+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:46:33.226981+0000","last_undegraded":"2024-09-21T01:46:33.226981+0000","last_fullsized":"2024-09-21T01:46:33.226981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260782+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:46:33.260782+0000","last_peered":"2024-09-21T01:46:33.260782+0000","last_clean":"2024-09-21T01:46:33.260782+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:46:33.260782+0000","last_undegraded":"2024-09-21T01:46:33.260782+0000","last_fullsized":"2024-09-21T01:46:33.260782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230552+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:46:33.230552+0000","last_peered":"2024-09-21T01:46:33.230552+0000","last_clean":"2024-09-21T01:46:33.230552+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:46:33.230552+0000","last_undegraded":"2024-09-21T01:46:33.230552+0000","last_fullsized":"2024-09-21T01:46:33.230552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230632+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:46:33.230632+0000","last_peered":"2024-09-21T01:46:33.230632+0000","last_clean":"2024-09-21T01:46:33.230632+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:46:33.230632+0000","last_undegraded":"2024-09-21T01:46:33.230632+0000","last_fullsized":"2024-09-21T01:46:33.230632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224723+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:46:33.224723+0000","last_peered":"2024-09-21T01:46:33.224723+0000","last_clean":"2024-09-21T01:46:33.224723+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:46:33.224723+0000","last_undegraded":"2024-09-21T01:46:33.224723+0000","last_fullsized":"2024-09-21T01:46:33.224723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228787+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:46:33.228787+0000","last_peered":"2024-09-21T01:46:33.228787+0000","last_clean":"2024-09-21T01:46:33.228787+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:46:33.228787+0000","last_undegraded":"2024-09-21T01:46:33.228787+0000","last_fullsized":"2024-09-21T01:46:33.228787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:21.303664+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:47:21.303664+0000","last_peered":"2024-09-21T01:47:21.303664+0000","last_clean":"2024-09-21T01:47:21.303664+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:47:21.303664+0000","last_undegraded":"2024-09-21T01:47:21.303664+0000","last_fullsized":"2024-09-21T01:47:21.303664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228873+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:46:33.228873+0000","last_peered":"2024-09-21T01:46:33.228873+0000","last_clean":"2024-09-21T01:46:33.228873+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:46:33.228873+0000","last_undegraded":"2024-09-21T01:46:33.228873+0000","last_fullsized":"2024-09-21T01:46:33.228873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223419+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:46:33.223419+0000","last_peered":"2024-09-21T01:46:33.223419+0000","last_clean":"2024-09-21T01:46:33.223419+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:46:33.223419+0000","last_undegraded":"2024-09-21T01:46:33.223419+0000","last_fullsized":"2024-09-21T01:46:33.223419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225008+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:46:33.225008+0000","last_peered":"2024-09-21T01:46:33.225008+0000","last_clean":"2024-09-21T01:46:33.225008+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:46:33.225008+0000","last_undegraded":"2024-09-21T01:46:33.225008+0000","last_fullsized":"2024-09-21T01:46:33.225008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:23.305965+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:47:23.305965+0000","last_peered":"2024-09-21T01:47:23.305965+0000","last_clean":"2024-09-21T01:47:23.305965+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:47:23.305965+0000","last_undegraded":"2024-09-21T01:47:23.305965+0000","last_fullsized":"2024-09-21T01:47:23.305965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228320+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:46:33.228320+0000","last_peered":"2024-09-21T01:46:33.228320+0000","last_clean":"2024-09-21T01:46:33.228320+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:46:33.228320+0000","last_undegraded":"2024-09-21T01:46:33.228320+0000","last_fullsized":"2024-09-21T01:46:33.228320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260933+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:46:33.260933+0000","last_peered":"2024-09-21T01:46:33.260933+0000","last_clean":"2024-09-21T01:46:33.260933+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:46:33.260933+0000","last_undegraded":"2024-09-21T01:46:33.260933+0000","last_fullsized":"2024-09-21T01:46:33.260933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227879+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:46:33.227879+0000","last_peered":"2024-09-21T01:46:33.227879+0000","last_clean":"2024-09-21T01:46:33.227879+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:46:33.227879+0000","last_undegraded":"2024-09-21T01:46:33.227879+0000","last_fullsized":"2024-09-21T01:46:33.227879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:23.306088+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:47:23.306088+0000","last_peered":"2024-09-21T01:47:23.306088+0000","last_clean":"2024-09-21T01:47:23.306088+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:47:23.306088+0000","last_undegraded":"2024-09-21T01:47:23.306088+0000","last_fullsized":"2024-09-21T01:47:23.306088+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":128,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229895+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:46:33.229895+0000","last_peered":"2024-09-21T01:46:33.229895+0000","last_clean":"2024-09-21T01:46:33.229895+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:46:33.229895+0000","last_undegraded":"2024-09-21T01:46:33.229895+0000","last_fullsized":"2024-09-21T01:46:33.229895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226066+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:46:33.226066+0000","last_peered":"2024-09-21T01:46:33.226066+0000","last_clean":"2024-09-21T01:46:33.226066+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:46:33.226066+0000","last_undegraded":"2024-09-21T01:46:33.226066+0000","last_fullsized":"2024-09-21T01:46:33.226066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.232179+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:46:33.232179+0000","last_peered":"2024-09-21T01:46:33.232179+0000","last_clean":"2024-09-21T01:46:33.232179+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:46:33.232179+0000","last_undegraded":"2024-09-21T01:46:33.232179+0000","last_fullsized":"2024-09-21T01:46:33.232179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:21.303840+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:47:21.303840+0000","last_peered":"2024-09-21T01:47:21.303840+0000","last_clean":"2024-09-21T01:47:21.303840+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:47:21.303840+0000","last_undegraded":"2024-09-21T01:47:21.303840+0000","last_fullsized":"2024-09-21T01:47:21.303840+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231720+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:46:33.231720+0000","last_peered":"2024-09-21T01:46:33.231720+0000","last_clean":"2024-09-21T01:46:33.231720+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:46:33.231720+0000","last_undegraded":"2024-09-21T01:46:33.231720+0000","last_fullsized":"2024-09-21T01:46:33.231720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227856+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:46:33.227856+0000","last_peered":"2024-09-21T01:46:33.227856+0000","last_clean":"2024-09-21T01:46:33.227856+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:46:33.227856+0000","last_undegraded":"2024-09-21T01:46:33.227856+0000","last_fullsized":"2024-09-21T01:46:33.227856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.234181+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:46:33.234181+0000","last_peered":"2024-09-21T01:46:33.234181+0000","last_clean":"2024-09-21T01:46:33.234181+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:46:33.234181+0000","last_undegraded":"2024-09-21T01:46:33.234181+0000","last_fullsized":"2024-09-21T01:46:33.234181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231777+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:46:33.231777+0000","last_peered":"2024-09-21T01:46:33.231777+0000","last_clean":"2024-09-21T01:46:33.231777+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:46:33.231777+0000","last_undegraded":"2024-09-21T01:46:33.231777+0000","last_fullsized":"2024-09-21T01:46:33.231777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231571+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:46:33.231571+0000","last_peered":"2024-09-21T01:46:33.231571+0000","last_clean":"2024-09-21T01:46:33.231571+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:46:33.231571+0000","last_undegraded":"2024-09-21T01:46:33.231571+0000","last_fullsized":"2024-09-21T01:46:33.231571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:24.308454+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:47:24.308454+0000","last_peered":"2024-09-21T01:47:24.308454+0000","last_clean":"2024-09-21T01:47:24.308454+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:47:24.308454+0000","last_undegraded":"2024-09-21T01:47:24.308454+0000","last_fullsized":"2024-09-21T01:47:24.308454+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231672+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:46:33.231672+0000","last_peered":"2024-09-21T01:46:33.231672+0000","last_clean":"2024-09-21T01:46:33.231672+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:46:33.231672+0000","last_undegraded":"2024-09-21T01:46:33.231672+0000","last_fullsized":"2024-09-21T01:46:33.231672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231609+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231609+0000","last_peered":"2024-09-21T01:46:33.231609+0000","last_clean":"2024-09-21T01:46:33.231609+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231609+0000","last_undegraded":"2024-09-21T01:46:33.231609+0000","last_fullsized":"2024-09-21T01:46:33.231609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237072+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237072+0000","last_peered":"2024-09-21T01:46:33.237072+0000","last_clean":"2024-09-21T01:46:33.237072+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237072+0000","last_undegraded":"2024-09-21T01:46:33.237072+0000","last_fullsized":"2024-09-21T01:46:33.237072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223710+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:46:33.223710+0000","last_peered":"2024-09-21T01:46:33.223710+0000","last_clean":"2024-09-21T01:46:33.223710+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:46:33.223710+0000","last_undegraded":"2024-09-21T01:46:33.223710+0000","last_fullsized":"2024-09-21T01:46:33.223710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231445+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231445+0000","last_peered":"2024-09-21T01:46:33.231445+0000","last_clean":"2024-09-21T01:46:33.231445+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231445+0000","last_undegraded":"2024-09-21T01:46:33.231445+0000","last_fullsized":"2024-09-21T01:46:33.231445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226072+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:46:33.226072+0000","last_peered":"2024-09-21T01:46:33.226072+0000","last_clean":"2024-09-21T01:46:33.226072+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:46:33.226072+0000","last_undegraded":"2024-09-21T01:46:33.226072+0000","last_fullsized":"2024-09-21T01:46:33.226072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:22.304623+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:47:22.304623+0000","last_peered":"2024-09-21T01:47:22.304623+0000","last_clean":"2024-09-21T01:47:22.304623+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:47:22.304623+0000","last_undegraded":"2024-09-21T01:47:22.304623+0000","last_fullsized":"2024-09-21T01:47:22.304623+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237841+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:46:33.237841+0000","last_peered":"2024-09-21T01:46:33.237841+0000","last_clean":"2024-09-21T01:46:33.237841+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:46:33.237841+0000","last_undegraded":"2024-09-21T01:46:33.237841+0000","last_fullsized":"2024-09-21T01:46:33.237841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"74'6","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:23.305859+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:47:23.305859+0000","last_peered":"2024-09-21T01:47:23.305859+0000","last_clean":"2024-09-21T01:47:23.305859+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:47:23.305859+0000","last_undegraded":"2024-09-21T01:47:23.305859+0000","last_fullsized":"2024-09-21T01:47:23.305859+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225616+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:46:33.225616+0000","last_peered":"2024-09-21T01:46:33.225616+0000","last_clean":"2024-09-21T01:46:33.225616+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:46:33.225616+0000","last_undegraded":"2024-09-21T01:46:33.225616+0000","last_fullsized":"2024-09-21T01:46:33.225616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225703+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225703+0000","last_peered":"2024-09-21T01:46:33.225703+0000","last_clean":"2024-09-21T01:46:33.225703+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225703+0000","last_undegraded":"2024-09-21T01:46:33.225703+0000","last_fullsized":"2024-09-21T01:46:33.225703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225654+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:46:33.225654+0000","last_peered":"2024-09-21T01:46:33.225654+0000","last_clean":"2024-09-21T01:46:33.225654+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:46:33.225654+0000","last_undegraded":"2024-09-21T01:46:33.225654+0000","last_fullsized":"2024-09-21T01:46:33.225654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230530+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230530+0000","last_peered":"2024-09-21T01:46:33.230530+0000","last_clean":"2024-09-21T01:46:33.230530+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230530+0000","last_undegraded":"2024-09-21T01:46:33.230530+0000","last_fullsized":"2024-09-21T01:46:33.230530+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227173+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:46:33.227173+0000","last_peered":"2024-09-21T01:46:33.227173+0000","last_clean":"2024-09-21T01:46:33.227173+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:46:33.227173+0000","last_undegraded":"2024-09-21T01:46:33.227173+0000","last_fullsized":"2024-09-21T01:46:33.227173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'14","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:39.858720+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:39.858720+0000","last_peered":"2024-09-21T01:46:39.858720+0000","last_clean":"2024-09-21T01:46:39.858720+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:39.858720+0000","last_undegraded":"2024-09-21T01:46:39.858720+0000","last_fullsized":"2024-09-21T01:46:39.858720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224103+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:46:33.224103+0000","last_peered":"2024-09-21T01:46:33.224103+0000","last_clean":"2024-09-21T01:46:33.224103+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:46:33.224103+0000","last_undegraded":"2024-09-21T01:46:33.224103+0000","last_fullsized":"2024-09-21T01:46:33.224103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237584+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237584+0000","last_peered":"2024-09-21T01:46:33.237584+0000","last_clean":"2024-09-21T01:46:33.237584+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237584+0000","last_undegraded":"2024-09-21T01:46:33.237584+0000","last_fullsized":"2024-09-21T01:46:33.237584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260770+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:46:33.260770+0000","last_peered":"2024-09-21T01:46:33.260770+0000","last_clean":"2024-09-21T01:46:33.260770+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:46:33.260770+0000","last_undegraded":"2024-09-21T01:46:33.260770+0000","last_fullsized":"2024-09-21T01:46:33.260770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228572+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:46:33.228572+0000","last_peered":"2024-09-21T01:46:33.228572+0000","last_clean":"2024-09-21T01:46:33.228572+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:46:33.228572+0000","last_undegraded":"2024-09-21T01:46:33.228572+0000","last_fullsized":"2024-09-21T01:46:33.228572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224910+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:46:33.224910+0000","last_peered":"2024-09-21T01:46:33.224910+0000","last_clean":"2024-09-21T01:46:33.224910+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:46:33.224910+0000","last_undegraded":"2024-09-21T01:46:33.224910+0000","last_fullsized":"2024-09-21T01:46:33.224910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223770+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:46:33.223770+0000","last_peered":"2024-09-21T01:46:33.223770+0000","last_clean":"2024-09-21T01:46:33.223770+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:46:33.223770+0000","last_undegraded":"2024-09-21T01:46:33.223770+0000","last_fullsized":"2024-09-21T01:46:33.223770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231959+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:46:33.231959+0000","last_peered":"2024-09-21T01:46:33.231959+0000","last_clean":"2024-09-21T01:46:33.231959+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:46:33.231959+0000","last_undegraded":"2024-09-21T01:46:33.231959+0000","last_fullsized":"2024-09-21T01:46:33.231959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227958+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:46:33.227958+0000","last_peered":"2024-09-21T01:46:33.227958+0000","last_clean":"2024-09-21T01:46:33.227958+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:46:33.227958+0000","last_undegraded":"2024-09-21T01:46:33.227958+0000","last_fullsized":"2024-09-21T01:46:33.227958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227724+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:46:33.227724+0000","last_peered":"2024-09-21T01:46:33.227724+0000","last_clean":"2024-09-21T01:46:33.227724+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:46:33.227724+0000","last_undegraded":"2024-09-21T01:46:33.227724+0000","last_fullsized":"2024-09-21T01:46:33.227724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.260935+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:46:33.260935+0000","last_peered":"2024-09-21T01:46:33.260935+0000","last_clean":"2024-09-21T01:46:33.260935+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:46:33.260935+0000","last_undegraded":"2024-09-21T01:46:33.260935+0000","last_fullsized":"2024-09-21T01:46:33.260935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224309+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224309+0000","last_peered":"2024-09-21T01:46:33.224309+0000","last_clean":"2024-09-21T01:46:33.224309+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224309+0000","last_undegraded":"2024-09-21T01:46:33.224309+0000","last_fullsized":"2024-09-21T01:46:33.224309+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224094+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:46:33.224094+0000","last_peered":"2024-09-21T01:46:33.224094+0000","last_clean":"2024-09-21T01:46:33.224094+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:46:33.224094+0000","last_undegraded":"2024-09-21T01:46:33.224094+0000","last_fullsized":"2024-09-21T01:46:33.224094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:47:22.303591+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:47:22.303591+0000","last_peered":"2024-09-21T01:47:22.303591+0000","last_clean":"2024-09-21T01:47:22.303591+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:47:22.303591+0000","last_undegraded":"2024-09-21T01:47:22.303591+0000","last_fullsized":"2024-09-21T01:47:22.303591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70715,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":659,"num_read_kb":406,"num_write":242,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":212145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776158,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939682,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1053735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":991630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":398842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32616,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712536,"statfs":{"total":95995035648,"available":95961636864,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":463486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":463657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:47:28.124 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T01:47:28.124 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T01:47:28.124 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T01:47:28.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T01:47:28.125 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:27 smithi190 ceph-mon[34368]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 -- 172.21.15.29:0/4062306270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18103bb0 msgr2=0x7f2e18103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/4062306270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18103bb0 0x7f2e18103fc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009900 tx=0x7f2e0802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 -- 172.21.15.29:0/4062306270 shutdown_connections 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/4062306270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18109e40 0x7f2e18110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/4062306270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e18109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:28.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/4062306270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18103bb0 0x7f2e18103fc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 -- 172.21.15.29:0/4062306270 >> 172.21.15.29:0/4062306270 conn(0x7f2e180ff600 msgr2=0x7f2e18101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 -- 172.21.15.29:0/4062306270 shutdown_connections 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.863+0000 7f2e1d59c640 1 -- 172.21.15.29:0/4062306270 wait complete. 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 Processor -- start 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 -- start start 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18103bb0 0x7f2e1819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e1819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18113b00 con 0x7f2e18109e40 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18113980 con 0x7f2e181094f0 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e1d59c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18113c80 con 0x7f2e18103bb0 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58154/0 (socket says 172.21.15.29:58154) 2024-09-21T01:47:28.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e177fe640 1 -- 172.21.15.29:0/1426302408 learned_addr learned my addr 172.21.15.29:0/1426302408 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e16ffd640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18103bb0 0x7f2e1819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.864+0000 7f2e167fc640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e1819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 -- 172.21.15.29:0/1426302408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18103bb0 msgr2=0x7f2e1819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18103bb0 0x7f2e1819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 -- 172.21.15.29:0/1426302408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 msgr2=0x7f2e1819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e1819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 -- 172.21.15.29:0/1426302408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e181a04e0 con 0x7f2e18109e40 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e167fc640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e1819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e177fe640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000eaa0 tx=0x7f2e1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000cdb0 con 0x7f2e18109e40 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e10005020 con 0x7f2e18109e40 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100116d0 con 0x7f2e18109e40 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e181a07d0 con 0x7f2e18109e40 2024-09-21T01:47:28.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.865+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181a6f30 con 0x7f2e18109e40 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.866+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18103fc0 con 0x7f2e18109e40 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.868+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2e100051d0 con 0x7f2e18109e40 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.869+0000 7f2dfffff640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 0x7f2e000b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.869+0000 7f2e16ffd640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 0x7f2e000b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.869+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2e100e7950 con 0x7f2e18109e40 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.869+0000 7f2e16ffd640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 0x7f2e000b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e08031440 tx=0x7f2e08009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:28.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:28.872+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2e10016030 con 0x7f2e18109e40 2024-09-21T01:47:28.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:28 smithi190 ceph-mon[34368]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:29.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:28 smithi066 ceph-mon[35001]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:29.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:28 smithi029 ceph-mon[31228]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:47:29.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.099+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2e1819b6a0 con 0x7f2e18109e40 2024-09-21T01:47:29.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.100+0000 7f2dfffff640 1 -- 172.21.15.29:0/1426302408 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2e1819b6a0 con 0x7f2e18109e40 2024-09-21T01:47:29.101 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:29.101 INFO:teuthology.orchestra.run.smithi029.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T01:47:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 msgr2=0x7f2e000b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 0x7f2e000b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e08031440 tx=0x7f2e08009560 comp rx=0 tx=0).stop 2024-09-21T01:47:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 msgr2=0x7f2e1819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000eaa0 tx=0x7f2e1000ef60 comp rx=0 tx=0).stop 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 shutdown_connections 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2e000b6620 0x7f2e000b8b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e18109e40 0x7f2e1819fdb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e181094f0 0x7f2e1819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 --2- 172.21.15.29:0/1426302408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e18103bb0 0x7f2e1819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 >> 172.21.15.29:0/1426302408 conn(0x7f2e180ff600 msgr2=0x7f2e18101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 shutdown_connections 2024-09-21T01:47:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:29.102+0000 7f2e1d59c640 1 -- 172.21.15.29:0/1426302408 wait complete. 2024-09-21T01:47:29.910 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T01:47:29.911 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:47:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:29 smithi066 ceph-mon[35001]: pgmap v334: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:29 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1426302408' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:47:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:29 smithi029 ceph-mon[31228]: pgmap v334: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:29 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1426302408' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:47:30.175 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:29 smithi190 ceph-mon[34368]: pgmap v334: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:29 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1426302408' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:47:30.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 -- 172.21.15.29:0/709892308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8041094a0 msgr2=0x7fe8041098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:30.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 --2- 172.21.15.29:0/709892308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8041094a0 0x7fe8041098b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009900 tx=0x7fe7f402a410 comp rx=0 tx=0).stop 2024-09-21T01:47:30.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 -- 172.21.15.29:0/709892308 shutdown_connections 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 --2- 172.21.15.29:0/709892308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe804109df0 0x7fe804110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 --2- 172.21.15.29:0/709892308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8041094a0 0x7fe8041098b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 --2- 172.21.15.29:0/709892308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804103b60 0x7fe804103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 -- 172.21.15.29:0/709892308 >> 172.21.15.29:0/709892308 conn(0x7fe80407b850 msgr2=0x7fe80407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 -- 172.21.15.29:0/709892308 shutdown_connections 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.689+0000 7fe80b798640 1 -- 172.21.15.29:0/709892308 wait complete. 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.690+0000 7fe80b798640 1 Processor -- start 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.690+0000 7fe80b798640 1 -- start start 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.690+0000 7fe80b798640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.690+0000 7fe80b798640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8041094a0 0x7fe80419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.690+0000 7fe80b798640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804109df0 0x7fe80419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe80b798640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe804113ab0 con 0x7fe804103b60 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe80b798640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe804113930 con 0x7fe804109df0 2024-09-21T01:47:30.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe80b798640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe804113c30 con 0x7fe8041094a0 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58162/0 (socket says 172.21.15.29:58162) 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 -- 172.21.15.29:0/1286138956 learned_addr learned my addr 172.21.15.29:0/1286138956 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809d35640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804109df0 0x7fe80419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe808d33640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8041094a0 0x7fe80419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 -- 172.21.15.29:0/1286138956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8041094a0 msgr2=0x7fe80419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8041094a0 0x7fe80419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 -- 172.21.15.29:0/1286138956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804109df0 msgr2=0x7fe80419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804109df0 0x7fe80419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 -- 172.21.15.29:0/1286138956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8041a0370 con 0x7fe804103b60 2024-09-21T01:47:30.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe809534640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c920 tx=0x7fe80000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800007c70 con 0x7fe804103b60 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.691+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe800007e10 con 0x7fe804103b60 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.692+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8041a0660 con 0x7fe804103b60 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.692+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800018820 con 0x7fe804103b60 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.692+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a6e80 con 0x7fe804103b60 2024-09-21T01:47:30.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.692+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe804103f70 con 0x7fe804103b60 2024-09-21T01:47:30.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.694+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe80000f450 con 0x7fe804103b60 2024-09-21T01:47:30.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.695+0000 7fe7fa7fc640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 0x7fe7f00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:30.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.695+0000 7fe808d33640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 0x7fe7f00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:30.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.695+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe8000e7800 con 0x7fe804103b60 2024-09-21T01:47:30.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.695+0000 7fe808d33640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 0x7fe7f00b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009230 tx=0x7fe7f4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:30.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.698+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe800014030 con 0x7fe804103b60 2024-09-21T01:47:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.926+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe80410bc10 con 0x7fe804103b60 2024-09-21T01:47:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.927+0000 7fe7fa7fc640 1 -- 172.21.15.29:0/1286138956 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7fe8000adf70 con 0x7fe804103b60 2024-09-21T01:47:30.928 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:30.928 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:47:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:47:30.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 msgr2=0x7fe7f00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:30.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 0x7fe7f00b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009230 tx=0x7fe7f4031400 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 msgr2=0x7fe80419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c920 tx=0x7fe80000cde0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 shutdown_connections 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe7f00b65d0 0x7fe7f00b8ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe804109df0 0x7fe80419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8041094a0 0x7fe80419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 --2- 172.21.15.29:0/1286138956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe804103b60 0x7fe80419a4c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 >> 172.21.15.29:0/1286138956 conn(0x7fe80407b850 msgr2=0x7fe804107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 shutdown_connections 2024-09-21T01:47:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:30.929+0000 7fe80b798640 1 -- 172.21.15.29:0/1286138956 wait complete. 2024-09-21T01:47:31.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:47:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:31 smithi066 ceph-mon[35001]: pgmap v335: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:31 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1286138956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:32.065 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:31 smithi029 ceph-mon[31228]: pgmap v335: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:32.065 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:31 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1286138956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:31 smithi190 ceph-mon[34368]: pgmap v335: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:31 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1286138956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 -- 172.21.15.29:0/317075742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda801062e0 msgr2=0x7fda801066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 --2- 172.21.15.29:0/317075742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda801062e0 0x7fda801066f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 -- 172.21.15.29:0/317075742 shutdown_connections 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 --2- 172.21.15.29:0/317075742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80106c30 0x7fda80110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 --2- 172.21.15.29:0/317075742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda801062e0 0x7fda801066f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 --2- 172.21.15.29:0/317075742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80102000 0x7fda80102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 -- 172.21.15.29:0/317075742 >> 172.21.15.29:0/317075742 conn(0x7fda80079e40 msgr2=0x7fda8007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 -- 172.21.15.29:0/317075742 shutdown_connections 2024-09-21T01:47:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.441+0000 7fda86272640 1 -- 172.21.15.29:0/317075742 wait complete. 2024-09-21T01:47:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.442+0000 7fda86272640 1 Processor -- start 2024-09-21T01:47:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.442+0000 7fda86272640 1 -- start start 2024-09-21T01:47:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.442+0000 7fda86272640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80102000 0x7fda8019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.442+0000 7fda86272640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 0x7fda8019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.442+0000 7fda86272640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda86272640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda80113a90 con 0x7fda80106c30 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda86272640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda80113910 con 0x7fda801062e0 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda86272640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda80113c10 con 0x7fda80102000 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57882/0 (socket says 172.21.15.29:57882) 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 -- 172.21.15.29:0/893595049 learned_addr learned my addr 172.21.15.29:0/893595049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7f7fe640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 0x7fda8019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7ffff640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80102000 0x7fda8019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 -- 172.21.15.29:0/893595049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80102000 msgr2=0x7fda8019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80102000 0x7fda8019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 -- 172.21.15.29:0/893595049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 msgr2=0x7fda8019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 0x7fda8019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 -- 172.21.15.29:0/893595049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801a25d0 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7f7fe640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 0x7fda8019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda8480f640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda7800cce0 tx=0x7fda7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800f450 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda801a28c0 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda7800faa0 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78018b70 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.443+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801a90e0 con 0x7fda80106c30 2024-09-21T01:47:32.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.445+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda80102410 con 0x7fda80106c30 2024-09-21T01:47:32.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.447+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fda78018d10 con 0x7fda80106c30 2024-09-21T01:47:32.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.448+0000 7fda7d7fa640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 0x7fda680b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:32.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.448+0000 7fda7ffff640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 0x7fda680b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:32.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.448+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fda780e78c0 con 0x7fda80106c30 2024-09-21T01:47:32.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.449+0000 7fda7ffff640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 0x7fda680b8b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda700058c0 tx=0x7fda7000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.453+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda78014030 con 0x7fda80106c30 2024-09-21T01:47:32.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.687+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda801a92b0 con 0x7fda80106c30 2024-09-21T01:47:32.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.688+0000 7fda7d7fa640 1 -- 172.21.15.29:0/893595049 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7fda780ae000 con 0x7fda80106c30 2024-09-21T01:47:32.689 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:32.689 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:47:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.690+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 msgr2=0x7fda680b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.690+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 0x7fda680b8b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda700058c0 tx=0x7fda7000a4c0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.690+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 msgr2=0x7fda801a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.690+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda7800cce0 tx=0x7fda7800a620 comp rx=0 tx=0).stop 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 shutdown_connections 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fda680b6650 0x7fda680b8b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda80106c30 0x7fda801a1f60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda801062e0 0x7fda8019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 --2- 172.21.15.29:0/893595049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda80102000 0x7fda8019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 >> 172.21.15.29:0/893595049 conn(0x7fda80079e40 msgr2=0x7fda801074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 shutdown_connections 2024-09-21T01:47:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:32.691+0000 7fda86272640 1 -- 172.21.15.29:0/893595049 wait complete. 2024-09-21T01:47:33.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/893595049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/893595049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/893595049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:33.348 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:47:33.609 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:34.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:33 smithi029 ceph-mon[31228]: pgmap v336: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:33 smithi066 ceph-mon[35001]: pgmap v336: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/1878460886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e781083d0 msgr2=0x7f4e781087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/1878460886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e781083d0 0x7f4e781087e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009900 tx=0x7f4e6c02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/1878460886 shutdown_connections 2024-09-21T01:47:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/1878460886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 0x7f4e7810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/1878460886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e781083d0 0x7f4e781087e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/1878460886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e78102a90 0x7f4e78102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/1878460886 >> 172.21.15.29:0/1878460886 conn(0x7f4e78079750 msgr2=0x7f4e78079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/1878460886 shutdown_connections 2024-09-21T01:47:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.107+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/1878460886 wait complete. 2024-09-21T01:47:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.108+0000 7f4e7cd6d640 1 Processor -- start 2024-09-21T01:47:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.108+0000 7f4e7cd6d640 1 -- start start 2024-09-21T01:47:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 0x7f4e7819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 0x7f4e7819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78113ad0 con 0x7f4e78102a90 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78113950 con 0x7f4e781083d0 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7cd6d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78113c50 con 0x7f4e78108d20 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57904/0 (socket says 172.21.15.29:57904) 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 -- 172.21.15.29:0/3045825614 learned_addr learned my addr 172.21.15.29:0/3045825614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e76d9d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 0x7f4e7819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e75d9b640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 0x7f4e7819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 -- 172.21.15.29:0/3045825614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 msgr2=0x7f4e7819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 0x7f4e7819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 -- 172.21.15.29:0/3045825614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 msgr2=0x7f4e7819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 0x7f4e7819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 -- 172.21.15.29:0/3045825614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e781a0410 con 0x7f4e78102a90 2024-09-21T01:47:34.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e75d9b640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 0x7f4e7819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e7659c640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b4a0 tx=0x7f4e6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.109+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68014070 con 0x7f4e78102a90 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.110+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e781a0700 con 0x7f4e78102a90 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.110+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e68010ac0 con 0x7f4e78102a90 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.110+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017c30 con 0x7f4e78102a90 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.110+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e781a6ee0 con 0x7f4e78102a90 2024-09-21T01:47:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.111+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e78103d10 con 0x7f4e78102a90 2024-09-21T01:47:34.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.112+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4e68017070 con 0x7f4e78102a90 2024-09-21T01:47:34.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.113+0000 7f4e677fe640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 0x7f4e600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:34.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.113+0000 7f4e75d9b640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 0x7f4e600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:34.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.113+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4e680e6ab0 con 0x7f4e78102a90 2024-09-21T01:47:34.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.114+0000 7f4e75d9b640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 0x7f4e600b8ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0096c0 tx=0x7f4e6c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:34.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.118+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e680ad1f0 con 0x7f4e78102a90 2024-09-21T01:47:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:33 smithi190 ceph-mon[34368]: pgmap v336: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.305+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e78102ea0 con 0x7f4e78102a90 2024-09-21T01:47:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.307+0000 7f4e677fe640 1 -- 172.21.15.29:0/3045825614 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7f4e680b15d0 con 0x7f4e78102a90 2024-09-21T01:47:34.309 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:34.309 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:47:34.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 msgr2=0x7f4e600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:34.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 0x7f4e600b8ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0096c0 tx=0x7f4e6c031400 comp rx=0 tx=0).stop 2024-09-21T01:47:34.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 msgr2=0x7f4e7819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:34.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b4a0 tx=0x7f4e6800b960 comp rx=0 tx=0).stop 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 shutdown_connections 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e600b65a0 0x7f4e600b8ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e78108d20 0x7f4e7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e781083d0 0x7f4e7819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 --2- 172.21.15.29:0/3045825614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e78102a90 0x7f4e7819a4a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 >> 172.21.15.29:0/3045825614 conn(0x7f4e78079750 msgr2=0x7f4e78105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 shutdown_connections 2024-09-21T01:47:34.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:34.311+0000 7f4e7cd6d640 1 -- 172.21.15.29:0/3045825614 wait complete. 2024-09-21T01:47:35.036 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:34 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3045825614' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:47:35.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:47:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:34 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3045825614' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:47:35.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:34 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3045825614' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:47:35.302 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 -- 172.21.15.29:0/1663485613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 msgr2=0x7f1e0c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 --2- 172.21.15.29:0/1663485613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c102e80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009900 tx=0x7f1dfc02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 -- 172.21.15.29:0/1663485613 shutdown_connections 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 --2- 172.21.15.29:0/1663485613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 0x7f1e0c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 --2- 172.21.15.29:0/1663485613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 0x7f1e0c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 --2- 172.21.15.29:0/1663485613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c102e80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 -- 172.21.15.29:0/1663485613 >> 172.21.15.29:0/1663485613 conn(0x7f1e0c079730 msgr2=0x7f1e0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 -- 172.21.15.29:0/1663485613 shutdown_connections 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.800+0000 7f1e115d0640 1 -- 172.21.15.29:0/1663485613 wait complete. 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 Processor -- start 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 -- start start 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 0x7f1e0c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 0x7f1e0c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1129c0 con 0x7f1e0c102a70 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c112840 con 0x7f1e0c108d00 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e115d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c112b40 con 0x7f1e0c1083b0 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e0affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e0affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57916/0 (socket says 172.21.15.29:57916) 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e0affd640 1 -- 172.21.15.29:0/3108191329 learned_addr learned my addr 172.21.15.29:0/3108191329 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e0b7fe640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 0x7f1e0c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.801+0000 7f1e0a7fc640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 0x7f1e0c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 -- 172.21.15.29:0/3108191329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 msgr2=0x7f1e0c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 0x7f1e0c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 -- 172.21.15.29:0/3108191329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 msgr2=0x7f1e0c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 0x7f1e0c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 -- 172.21.15.29:0/3108191329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c19f2e0 con 0x7f1e0c102a70 2024-09-21T01:47:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e0affd640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc005ab0 tx=0x7f1dfc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc03f070 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dfc031b60 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c19f570 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc035030 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.802+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c1a5d30 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.803+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0c103cf0 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.805+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1dfc042070 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.806+0000 7f1df3fff640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 0x7f1df40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.806+0000 7f1e0a7fc640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 0x7f1df40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.806+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1dfc108310 con 0x7f1e0c102a70 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.806+0000 7f1e0a7fc640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 0x7f1df40b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c100b60 tx=0x7f1e00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:35.809+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1dfc0d2dd0 con 0x7f1e0c102a70 2024-09-21T01:47:36.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.039+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e0c102e80 con 0x7f1e0c102a70 2024-09-21T01:47:36.040 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:35 smithi029 ceph-mon[31228]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:36.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.039+0000 7f1df3fff640 1 -- 172.21.15.29:0/3108191329 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7f1dfc0d7d10 con 0x7f1e0c102a70 2024-09-21T01:47:36.041 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:36.041 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:47:36.041 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:47:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 msgr2=0x7f1df40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 0x7f1df40b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c100b60 tx=0x7f1e00005f90 comp rx=0 tx=0).stop 2024-09-21T01:47:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 msgr2=0x7f1e0c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc005ab0 tx=0x7f1dfc003c10 comp rx=0 tx=0).stop 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 shutdown_connections 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1df40b6620 0x7f1df40b8b30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c108d00 0x7f1e0c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c1083b0 0x7f1e0c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 --2- 172.21.15.29:0/3108191329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e0c102a70 0x7f1e0c199370 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 >> 172.21.15.29:0/3108191329 conn(0x7f1e0c079730 msgr2=0x7f1e0c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 shutdown_connections 2024-09-21T01:47:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:36.042+0000 7f1e115d0640 1 -- 172.21.15.29:0/3108191329 wait complete. 2024-09-21T01:47:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:35 smithi066 ceph-mon[35001]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:35 smithi190 ceph-mon[34368]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:36.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T01:46:10.616066+0000', 'modified': '2024-09-21T01:46:22.201278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24671, 'mds_1': 24644, 'mds_2': 24640, 'mds_3': 24625, 'mds_4': 24620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14778': {'gid': 14778, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6839/3540512645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 3540512645}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 3540512645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14784': {'gid': 14784, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6841/2160178023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6840', 'nonce': 2160178023}, {'type': 'v1', 'addr': '172.21.15.29:6841', 'nonce': 2160178023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/3991511259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 3991511259}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 3991511259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/3428222627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 3428222627}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 3428222627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/4140562793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 4140562793}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 4140562793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.66:6835/2455557806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2455557806}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2455557806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24625': {'gid': 24625, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/2047991561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 2047991561}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 2047991561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.190:6839/636701059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 636701059}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 636701059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.66:6837/1538999358', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 1538999358}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1538999358}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6839/2350524023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2350524023}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2350524023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671, 24644, 24640, 24625, 24620]} 2024-09-21T01:47:36.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T01:47:36.820 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T01:47:37.091 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:37.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:36 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3108191329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:37.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:36 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3108191329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:37.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:36 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3108191329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 -- 172.21.15.29:0/1661570906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 msgr2=0x7f132810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 --2- 172.21.15.29:0/1661570906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132810f530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1320009ca0 tx=0x7f132002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 -- 172.21.15.29:0/1661570906 shutdown_connections 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 --2- 172.21.15.29:0/1661570906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132810f530 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 --2- 172.21.15.29:0/1661570906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 0x7f13281087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 --2- 172.21.15.29:0/1661570906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 0x7f1328102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 -- 172.21.15.29:0/1661570906 >> 172.21.15.29:0/1661570906 conn(0x7f1328079710 msgr2=0x7f1328079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 -- 172.21.15.29:0/1661570906 shutdown_connections 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.595+0000 7f13307a9640 1 -- 172.21.15.29:0/1661570906 wait complete. 2024-09-21T01:47:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 Processor -- start 2024-09-21T01:47:37.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 -- start start 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 0x7f1328199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 0x7f13281998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13281129c0 con 0x7f1328108d00 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1328112840 con 0x7f1328102a70 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f13307a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1328112b40 con 0x7f13281083b0 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f132ed46640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f132ed46640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57944/0 (socket says 172.21.15.29:57944) 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f132dd44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 0x7f13281998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f132ed46640 1 -- 172.21.15.29:0/524826396 learned_addr learned my addr 172.21.15.29:0/524826396 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.596+0000 7f132e545640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 0x7f1328199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 -- 172.21.15.29:0/524826396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 msgr2=0x7f13281998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 0x7f13281998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 -- 172.21.15.29:0/524826396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 msgr2=0x7f1328199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 0x7f1328199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 -- 172.21.15.29:0/524826396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132819f2c0 con 0x7f1328108d00 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f132ed46640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1320005cc0 tx=0x7f132002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f131f7fe640 1 -- 172.21.15.29:0/524826396 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320003bf0 con 0x7f1328108d00 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f131f7fe640 1 -- 172.21.15.29:0/524826396 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1320003d90 con 0x7f1328108d00 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132819f550 con 0x7f1328108d00 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f131f7fe640 1 -- 172.21.15.29:0/524826396 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132002fbc0 con 0x7f1328108d00 2024-09-21T01:47:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.597+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281a5d10 con 0x7f1328108d00 2024-09-21T01:47:37.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.600+0000 7f131f7fe640 1 -- 172.21.15.29:0/524826396 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f132002fd60 con 0x7f1328108d00 2024-09-21T01:47:37.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.600+0000 7f1336d1d740 1 Processor -- start 2024-09-21T01:47:37.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.600+0000 7f1336d1d740 1 -- start start 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ecfef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ecff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ec76c10 con 0x55bd0e938280 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ec66040 con 0x55bd0e958f60 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f1336d1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ec69e50 con 0x55bd0e9300a0 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57952/0 (socket says 172.21.15.29:57952) 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 -- 172.21.15.29:0/3269460703 learned_addr learned my addr 172.21.15.29:0/3269460703 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132e545640 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ecff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f131f7fe640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 0x7f13140b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ecfef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 -- 172.21.15.29:0/3269460703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 msgr2=0x55bd0ecff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ecff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 -- 172.21.15.29:0/3269460703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 msgr2=0x55bd0ecfef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ecfef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132ed46640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 0x7f13140b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 -- 172.21.15.29:0/3269460703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0e9c4540 con 0x55bd0e938280 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f131f7fe640 1 -- 172.21.15.29:0/524826396 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1320108910 con 0x7f1328108d00 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.601+0000 7f132dd44640 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1318012540 tx=0x7f1318005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef621a4f65d6112a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f131cff9640 1 -- 172.21.15.29:0/3269460703 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318004300 con 0x55bd0e938280 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f131cff9640 1 -- 172.21.15.29:0/3269460703 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13180044a0 con 0x55bd0e938280 2024-09-21T01:47:37.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f131cff9640 1 -- 172.21.15.29:0/3269460703 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13180035f0 con 0x55bd0e938280 2024-09-21T01:47:37.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f1336d1d740 1 -- 172.21.15.29:0/3269460703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 msgr2=0x55bd0ecff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1318012540 tx=0x7f1318005c90 comp rx=0 tx=0).stop 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.602+0000 7f132ed46640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 0x7f13140b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce5f0 tx=0x7f13200f2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 -- 172.21.15.29:0/3269460703 shutdown_connections 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ecff980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ecff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3269460703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ecfef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 -- 172.21.15.29:0/3269460703 >> 172.21.15.29:0/3269460703 conn(0x55bd0e94d350 msgr2=0x55bd0ecfcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 -- 172.21.15.29:0/3269460703 shutdown_connections 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 -- 172.21.15.29:0/3269460703 wait complete. 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 -- start start 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 Processor -- start 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.603+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd0eac75f0 con 0x55bd0e958f60 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ec01010 con 0x55bd0e938280 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed0f8f0 con 0x55bd0e9300a0 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 10 client.0 _ll_register_callbacks cb 0x55bd0ec60340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57964/0 (socket says 172.21.15.29:57964) 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 -- 172.21.15.29:0/3350579162 learned_addr learned my addr 172.21.15.29:0/3350579162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132e545640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 -- 172.21.15.29:0/3350579162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 msgr2=0x55bd0ed0a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 -- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 msgr2=0x55bd0ed0aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed0aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 -- 172.21.15.29:0/3350579162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0ec2ea00 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f132dd44640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55bd0ed0cfe0 tx=0x7f1318004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdeaca2a8b8430 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13180048d0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1318004fa0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.604+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13180044a0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13180051a0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 10 client.15150 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd0ed1b7f0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13180044a0 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f131801c020 con 0x55bd0e958f60 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:37.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 10 client.15150 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 20 client.15150 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e9178e0 0x55bd0ed1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 4 client.15150 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.605+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd0ec78b60 con 0x55bd0e9178e0 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.606+0000 7f132e545640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e9178e0 0x55bd0ed1efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.606+0000 7f132e545640 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e9178e0 0x55bd0ed1efc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=204e62ba732e5c33 server_cookie=8bb6149d0f30a73c in_seq=0 out_seq=0 2024-09-21T01:47:37.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.606+0000 7f1311ffb640 10 client.15150 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.607+0000 7f1311ffb640 1 -- 172.21.15.29:0/3350579162 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd0ec78b60 con 0x55bd0e9178e0 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.607+0000 7f1311ffb640 10 client.15150 handle_command_reply: tid=0 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.607+0000 7f1336d1d740 1 client.15150 shutdown 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.607+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 msgr2=0x55bd0ed18c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.607+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55bd0ed0cfe0 tx=0x7f1318004bc0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 shutdown_connections 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e9178e0 0x55bd0ed1efc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e958f60 0x55bd0ed18c10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1311ffb640 0 client.15150 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed0aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 --2- 172.21.15.29:0/3350579162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 >> 172.21.15.29:0/3350579162 conn(0x55bd0e94d350 msgr2=0x55bd0ed02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 shutdown_connections 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 1 -- 172.21.15.29:0/3350579162 wait complete. 2024-09-21T01:47:37.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.608+0000 7f1336d1d740 20 client.15150 trim_cache size 0 max 16384 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 Processor -- start 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 -- start start 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e958f60 0x55bd0ed190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1d300 con 0x55bd0e9300a0 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1d9d0 con 0x55bd0e938280 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f1336d1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1e0a0 con 0x55bd0e958f60 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.626+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57978/0 (socket says 172.21.15.29:57978) 2024-09-21T01:47:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 -- 172.21.15.29:0/4243767997 learned_addr learned my addr 172.21.15.29:0/4243767997 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e958f60 0x55bd0ed190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132e545640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 -- 172.21.15.29:0/4243767997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e958f60 msgr2=0x55bd0ed190c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e958f60 0x55bd0ed190c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 -- 172.21.15.29:0/4243767997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 msgr2=0x55bd0ed18b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed18b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 -- 172.21.15.29:0/4243767997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0ed011f0 con 0x55bd0e9300a0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132e545640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f132dd44640 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f131800b8a0 tx=0x7f131801e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105ebcb4b5f33c1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f1313fff640 1 -- 172.21.15.29:0/4243767997 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318004100 con 0x55bd0e9300a0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f1313fff640 1 -- 172.21.15.29:0/4243767997 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f131800b390 con 0x55bd0e9300a0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.627+0000 7f1313fff640 1 -- 172.21.15.29:0/4243767997 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318004c30 con 0x55bd0e9300a0 2024-09-21T01:47:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- 172.21.15.29:0/4243767997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 msgr2=0x55bd0ed05020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f131800b8a0 tx=0x7f131801e450 comp rx=0 tx=0).stop 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- 172.21.15.29:0/4243767997 shutdown_connections 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e958f60 0x55bd0ed190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e938280 0x55bd0ed18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 --2- 172.21.15.29:0/4243767997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e9300a0 0x55bd0ed05020 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- 172.21.15.29:0/4243767997 >> 172.21.15.29:0/4243767997 conn(0x55bd0e94d350 msgr2=0x55bd0ed0a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- 172.21.15.29:0/4243767997 shutdown_connections 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- 172.21.15.29:0/4243767997 wait complete. 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 -- start start 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.628+0000 7f1336d1d740 1 Processor -- start 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ed27750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1ee40 con 0x55bd0e938280 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1e770 con 0x55bd0e958f60 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ed1f510 con 0x55bd0e9300a0 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1336d1d740 10 client.0 _ll_register_callbacks cb 0x7f132400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57992/0 (socket says 172.21.15.29:57992) 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 -- 172.21.15.29:0/821406608 learned_addr learned my addr 172.21.15.29:0/821406608 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132dd44640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132dd44640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ed27750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 -- 172.21.15.29:0/821406608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 msgr2=0x55bd0ed0ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 -- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 msgr2=0x55bd0ed27750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ed27750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 -- 172.21.15.29:0/821406608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0ed27c90 con 0x55bd0e938280 2024-09-21T01:47:37.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132dd44640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ed27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f132e545640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55bd0ed1f6c0 tx=0x7f132400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c54092f91134bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.629+0000 7f1311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324011640 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f13240117e0 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131800b390 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13240119e0 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 10 client.15156 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd0ed08360 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f131800b390 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f132402fb30 con 0x55bd0e938280 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 10 client.15156 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 20 client.15156 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e8ecc90 0x55bd0ed8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:37.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 4 client.15156 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:47:37.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.630+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "status"}) -- 0x55bd0ec74610 con 0x55bd0e8ecc90 2024-09-21T01:47:37.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f132dd44640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e8ecc90 0x55bd0ed8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f132dd44640 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e8ecc90 0x55bd0ed8fe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32892919cbaee6a7 server_cookie=9b7fdc7be0cb9ae4 in_seq=0 out_seq=0 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1311ffb640 10 client.15156 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1311ffb640 1 -- 172.21.15.29:0/821406608 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bd0ec74610 con 0x55bd0e8ecc90 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1311ffb640 10 client.15156 handle_command_reply: tid=0 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1336d1d740 1 client.15156 shutdown 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 msgr2=0x55bd0ed27210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55bd0ed1f6c0 tx=0x7f132400bef0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 shutdown_connections 2024-09-21T01:47:37.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.631+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55bd0e8ecc90 0x55bd0ed8fe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd0e958f60 0x55bd0ed27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd0e938280 0x55bd0ed27210 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1311ffb640 0 client.15156 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 --2- 172.21.15.29:0/821406608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd0e9300a0 0x55bd0ed0ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 >> 172.21.15.29:0/821406608 conn(0x55bd0e94d350 msgr2=0x55bd0ecfc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 shutdown_connections 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 1 -- 172.21.15.29:0/821406608 wait complete. 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f1336d1d740 20 client.15156 trim_cache size 0 max 16384 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:47:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 4, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24620, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 76.439499546999997, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 116.27591633199999 2024-09-21T01:47:37.640 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 msgr2=0x7f13140b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 0x7f13140b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce5f0 tx=0x7f13200f2040 comp rx=0 tx=0).stop 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 msgr2=0x7f132819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1320005cc0 tx=0x7f132002f380 comp rx=0 tx=0).stop 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 shutdown_connections 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f13140b6620 0x7f13140b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1328108d00 0x7f132819eb90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13281083b0 0x7f13281998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 --2- 172.21.15.29:0/524826396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1328102a70 0x7f1328199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.632+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 >> 172.21.15.29:0/524826396 conn(0x7f1328079710 msgr2=0x7f1328106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.633+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 shutdown_connections 2024-09-21T01:47:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:37.633+0000 7f13307a9640 1 -- 172.21.15.29:0/524826396 wait complete. 2024-09-21T01:47:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:37 smithi029 ceph-mon[31228]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:37 smithi190 ceph-mon[34368]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:37 smithi066 ceph-mon[35001]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:38.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T01:47:38.653 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 -- 172.21.15.29:0/813898009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81028b0 msgr2=0x7f9fa8102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 --2- 172.21.15.29:0/813898009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81028b0 0x7f9fa8102cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009900 tx=0x7f9f9802a410 comp rx=0 tx=0).stop 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 -- 172.21.15.29:0/813898009 shutdown_connections 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 --2- 172.21.15.29:0/813898009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 --2- 172.21.15.29:0/813898009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81081f0 0x7f9fa8108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 --2- 172.21.15.29:0/813898009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81028b0 0x7f9fa8102cc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 -- 172.21.15.29:0/813898009 >> 172.21.15.29:0/813898009 conn(0x7f9fa80fe340 msgr2=0x7f9fa8100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 -- 172.21.15.29:0/813898009 shutdown_connections 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.128+0000 7f9fa759e640 1 -- 172.21.15.29:0/813898009 wait complete. 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.129+0000 7f9fa759e640 1 Processor -- start 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.129+0000 7f9fa759e640 1 -- start start 2024-09-21T01:47:39.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.129+0000 7f9fa759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81028b0 0x7f9fa819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.129+0000 7f9fa759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa8113920 con 0x7f9fa81081f0 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81137a0 con 0x7f9fa8108b40 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa8113aa0 con 0x7f9fa81028b0 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58006/0 (socket says 172.21.15.29:58006) 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3893667907 learned_addr learned my addr 172.21.15.29:0/3893667907 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa659c640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81028b0 0x7f9fa819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3893667907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81028b0 msgr2=0x7f9fa819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81028b0 0x7f9fa819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3893667907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 msgr2=0x7f9fa819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3893667907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa81a01a0 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c8c0 tx=0x7f9f9000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9f7fe640 1 -- 172.21.15.29:0/3893667907 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90007bb0 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9f7fe640 1 -- 172.21.15.29:0/3893667907 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f90007d50 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81a0490 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9f9f7fe640 1 -- 172.21.15.29:0/3893667907 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900187c0 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.130+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81a6cb0 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.133+0000 7f9f9f7fe640 1 -- 172.21.15.29:0/3893667907 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9f9000fa20 con 0x7f9fa81081f0 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.133+0000 7f9fb2f78740 1 Processor -- start 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.133+0000 7f9fb2f78740 1 -- start start 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a83460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8346630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5a82bdc10 con 0x55f5a7f7f280 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5a82ad040 con 0x55f5a7f770a0 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a82b0e50 con 0x55f5a7f9ff60 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58010/0 (socket says 172.21.15.29:58010) 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 -- 172.21.15.29:0/1984921293 learned_addr learned my addr 172.21.15.29:0/1984921293 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a83460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9fa659c640 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8346630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9f7fe640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 0x7f9f940b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 -- 172.21.15.29:0/1984921293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 msgr2=0x55f5a83460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a83460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 -- 172.21.15.29:0/1984921293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 msgr2=0x55f5a8346630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8346630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.134+0000 7f9f9ffff640 1 -- 172.21.15.29:0/1984921293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a800b540 con 0x55f5a7f7f280 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 0x7f9f940b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9f9f7fe640 1 -- 172.21.15.29:0/3893667907 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9f900e7580 con 0x7f9fa81081f0 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f900e4e80 tx=0x7f9f9007a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a768672cd12203a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9f9cff9640 1 -- 172.21.15.29:0/1984921293 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900ad390 con 0x55f5a7f7f280 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9f9cff9640 1 -- 172.21.15.29:0/1984921293 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9007a330 con 0x55f5a7f7f280 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.135+0000 7f9f9cff9640 1 -- 172.21.15.29:0/1984921293 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9007a960 con 0x55f5a7f7f280 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1984921293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 msgr2=0x55f5a8346b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 0x7f9f940b8b60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005fd0 tx=0x7f9fa0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f900e4e80 tx=0x7f9f9007a2c0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1984921293 shutdown_connections 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a8346b70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8346630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1984921293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a83460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1984921293 >> 172.21.15.29:0/1984921293 conn(0x55f5a7f94350 msgr2=0x55f5a8343fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1984921293 shutdown_connections 2024-09-21T01:47:39.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.136+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1984921293 wait complete. 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 -- start start 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 Processor -- start 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 0x55f5a8351cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f7f280 0x55f5a8352200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5a810e5f0 con 0x55f5a7f9ff60 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8248010 con 0x55f5a7f7f280 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8358b00 con 0x55f5a7f770a0 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fb2f78740 10 client.0 _ll_register_callbacks cb 0x55f5a82a7340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fa6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.137+0000 7f9fa6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58014/0 (socket says 172.21.15.29:58014) 2024-09-21T01:47:39.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/1278644026 learned_addr learned my addr 172.21.15.29:0/1278644026 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f7f280 0x55f5a8352200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa659c640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 0x55f5a8351cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 msgr2=0x55f5a8351cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 0x55f5a8351cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/1278644026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f7f280 msgr2=0x55f5a8352200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f7f280 0x55f5a8352200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/1278644026 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a8275a00 con 0x55f5a7f9ff60 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa659c640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 0x55f5a8351cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000f960 tx=0x7f9fa0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8860d8e4fb24f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa001ed60 con 0x55f5a7f9ff60 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9fa001b410 con 0x55f5a7f9ff60 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5a82bfb60 con 0x55f5a7f9ff60 2024-09-21T01:47:39.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9fa001b610 con 0x55f5a7f9ff60 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fb2f78740 10 client.15168 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.138+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5a8364c40 con 0x55f5a7f9ff60 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f5a82bfb60 con 0x55f5a7f9ff60 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9fa0025070 con 0x55f5a7f9ff60 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 20 client.15168 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f5e8e0 0x55f5a83686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 4 client.15168 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a82bb610 con 0x55f5a7f5e8e0 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.139+0000 7f9fa659c640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f5e8e0 0x55f5a83686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.140+0000 7f9fa659c640 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f5e8e0 0x55f5a83686d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0a4e15dceffb617 server_cookie=8c4a220ffcde6412 in_seq=0 out_seq=0 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.140+0000 7f9f8dffb640 10 client.15168 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:47:39.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9f8dffb640 1 -- 172.21.15.29:0/1278644026 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5a82bb610 con 0x55f5a7f5e8e0 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9f8dffb640 10 client.15168 handle_command_reply: tid=0 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 client.15168 shutdown 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 msgr2=0x55f5a8361eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000f960 tx=0x7f9fa0004ab0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 shutdown_connections 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a8361eb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f7f280 0x55f5a8352200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f5e8e0 0x55f5a83686d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/1278644026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f770a0 0x55f5a8351cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9f8dffb640 0 client.15168 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 >> 172.21.15.29:0/1278644026 conn(0x55f5a7f94350 msgr2=0x55f5a8342760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 shutdown_connections 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 1 -- 172.21.15.29:0/1278644026 wait complete. 2024-09-21T01:47:39.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.141+0000 7f9fb2f78740 20 client.15168 trim_cache size 0 max 16384 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 Processor -- start 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 -- start start 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8351ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a834cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8366a10 con 0x55f5a7f7f280 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5a83670e0 con 0x55f5a7f770a0 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.159+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a83677b0 con 0x55f5a7f9ff60 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58020/0 (socket says 172.21.15.29:58020) 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 -- 172.21.15.29:0/4137827112 learned_addr learned my addr 172.21.15.29:0/4137827112 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9fa659c640 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8351ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a834cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 -- 172.21.15.29:0/4137827112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 msgr2=0x55f5a834cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a834cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 -- 172.21.15.29:0/4137827112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 msgr2=0x55f5a8351ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8351ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 -- 172.21.15.29:0/4137827112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a8352b60 con 0x55f5a7f7f280 2024-09-21T01:47:39.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f900a9b90 tx=0x7f9f9000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e21d65666778f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f8ffff640 1 -- 172.21.15.29:0/4137827112 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900b3230 con 0x55f5a7f7f280 2024-09-21T01:47:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f8ffff640 1 -- 172.21.15.29:0/4137827112 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f900ad0d0 con 0x55f5a7f7f280 2024-09-21T01:47:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.160+0000 7f9f8ffff640 1 -- 172.21.15.29:0/4137827112 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900acd30 con 0x55f5a7f7f280 2024-09-21T01:47:39.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 -- 172.21.15.29:0/4137827112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 msgr2=0x55f5a83521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f900a9b90 tx=0x7f9f9000fd10 comp rx=0 tx=0).stop 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 -- 172.21.15.29:0/4137827112 shutdown_connections 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f9ff60 0x55f5a834cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f7f280 0x55f5a83521e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/4137827112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8351ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 -- 172.21.15.29:0/4137827112 >> 172.21.15.29:0/4137827112 conn(0x55f5a7f94350 msgr2=0x55f5a83620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 -- 172.21.15.29:0/4137827112 shutdown_connections 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.161+0000 7f9fb2f78740 1 -- 172.21.15.29:0/4137827112 wait complete. 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 -- start start 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 Processor -- start 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8370ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f7f280 0x55f5a836a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8368550 con 0x55f5a7f9ff60 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8367e80 con 0x55f5a7f770a0 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a8368c20 con 0x55f5a7f7f280 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fb2f78740 10 client.0 _ll_register_callbacks cb 0x7f9fa000f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58028/0 (socket says 172.21.15.29:58028) 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/274196199 learned_addr learned my addr 172.21.15.29:0/274196199 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa659c640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8370ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9f9ffff640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f7f280 0x55f5a836a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f7f280 msgr2=0x55f5a836a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.162+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f7f280 0x55f5a836a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/274196199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 msgr2=0x55f5a8370ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8370ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fa6d9d640 1 -- 172.21.15.29:0/274196199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a836ae00 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fa6d9d640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000f930 tx=0x7f9fa001b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a399098c9b79bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa001eb60 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9fa001ed00 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f900ad0d0 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9fa0042050 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fb2f78740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5a83707f0 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.163+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f900ad0d0 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9fa0025040 con 0x55f5a7f9ff60 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 10 client.15174 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 20 client.15174 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:39.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f33c90 0x55f5a83d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 4 client.15174 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "status"}) -- 0x55f5a82b8a40 con 0x55f5a7f33c90 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fa659c640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f33c90 0x55f5a83d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9f8dffb640 10 client.15174 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.164+0000 7f9fa659c640 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f33c90 0x55f5a83d7230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc1abd47ce9a0113 server_cookie=8fad658043127791 in_seq=0 out_seq=0 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9f8dffb640 1 -- 172.21.15.29:0/274196199 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f5a82b8a40 con 0x55f5a7f33c90 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9f8dffb640 10 client.15174 handle_command_reply: tid=0 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 client.15174 shutdown 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 msgr2=0x55f5a836a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000f930 tx=0x7f9fa001b410 comp rx=0 tx=0).stop 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 shutdown_connections 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5a7f9ff60 0x55f5a836a8c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f5a7f33c90 0x55f5a83d7230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a7f7f280 0x55f5a836a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 --2- 172.21.15.29:0/274196199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5a7f770a0 0x55f5a8370ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 >> 172.21.15.29:0/274196199 conn(0x55f5a7f94350 msgr2=0x55f5a83620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9f8dffb640 0 client.15174 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 shutdown_connections 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 1 -- 172.21.15.29:0/274196199 wait complete. 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.165+0000 7f9fb2f78740 20 client.15174 trim_cache size 0 max 16384 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 3, 2024-09-21T01:47:39.171 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24625, 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 78.981278875000001, 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 113.00194136899999 2024-09-21T01:47:39.172 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 msgr2=0x7f9f940b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 0x7f9f940b8b60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005fd0 tx=0x7f9fa0004a40 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 msgr2=0x7f9fa819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c8c0 tx=0x7f9f9000cd80 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 shutdown_connections 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9f940b6650 0x7f9f940b8b60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fa8108b40 0x7f9fa819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fa81081f0 0x7f9fa819a890 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 --2- 172.21.15.29:0/3893667907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fa81028b0 0x7f9fa819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:39.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 >> 172.21.15.29:0/3893667907 conn(0x7f9fa80fe340 msgr2=0x7f9fa8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:39.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 shutdown_connections 2024-09-21T01:47:39.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:39.166+0000 7f9fa759e640 1 -- 172.21.15.29:0/3893667907 wait complete. 2024-09-21T01:47:39.887 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T01:47:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:39 smithi029 ceph-mon[31228]: pgmap v339: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:40.147 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:39 smithi190 ceph-mon[34368]: pgmap v339: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:39 smithi066 ceph-mon[35001]: pgmap v339: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.662+0000 7fc69fe87640 1 -- 172.21.15.29:0/89579888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 msgr2=0x7fc698110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.662+0000 7fc69fe87640 1 --2- 172.21.15.29:0/89579888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698110620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc690009ca0 tx=0x7fc69002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 -- 172.21.15.29:0/89579888 shutdown_connections 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 --2- 172.21.15.29:0/89579888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698110620 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 --2- 172.21.15.29:0/89579888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 0x7fc6981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 --2- 172.21.15.29:0/89579888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 0x7fc698103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 -- 172.21.15.29:0/89579888 >> 172.21.15.29:0/89579888 conn(0x7fc69807b850 msgr2=0x7fc69807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 -- 172.21.15.29:0/89579888 shutdown_connections 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 -- 172.21.15.29:0/89579888 wait complete. 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 Processor -- start 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.663+0000 7fc69fe87640 1 -- start start 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 0x7fc698075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 0x7fc698075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc698113ab0 con 0x7fc698109df0 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc698113930 con 0x7fc698103b60 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69fe87640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc698113c30 con 0x7fc6981094a0 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58040/0 (socket says 172.21.15.29:58040) 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 -- 172.21.15.29:0/2912864828 learned_addr learned my addr 172.21.15.29:0/2912864828 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69d422640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 0x7fc698075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69dc23640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 0x7fc698075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 -- 172.21.15.29:0/2912864828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 msgr2=0x7fc698075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 0x7fc698075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 -- 172.21.15.29:0/2912864828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 msgr2=0x7fc698075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 0x7fc698075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 -- 172.21.15.29:0/2912864828 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6981a8760 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc69e424640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc690005cc0 tx=0x7fc69002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.664+0000 7fc68effd640 1 -- 172.21.15.29:0/2912864828 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690003bf0 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.665+0000 7fc68effd640 1 -- 172.21.15.29:0/2912864828 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc690003d90 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.665+0000 7fc68effd640 1 -- 172.21.15.29:0/2912864828 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69002fbc0 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.665+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6981a89f0 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.665+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a8f80 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.667+0000 7fc6a63fb740 1 Processor -- start 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.667+0000 7fc68effd640 1 -- 172.21.15.29:0/2912864828 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc69002fd60 con 0x7fc698109df0 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 -- start start 2024-09-21T01:47:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d05280 0x5633c80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5633c8043c10 con 0x5633c7d25f60 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5633c8033040 con 0x5633c7d05280 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633c8036e50 con 0x5633c7cfd0a0 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58052/0 (socket says 172.21.15.29:58052) 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 -- 172.21.15.29:0/1917380189 learned_addr learned my addr 172.21.15.29:0/1917380189 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69dc23640 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 -- 172.21.15.29:0/1917380189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 msgr2=0x5633c80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 -- 172.21.15.29:0/1917380189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d05280 msgr2=0x5633c80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d05280 0x5633c80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 -- 172.21.15.29:0/1917380189 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c7d91540 con 0x5633c7d25f60 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.668+0000 7fc69d422640 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b6c0 tx=0x7fc68800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2a4d357dbd649e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc68effd640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 0x7fc6840b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc69dc23640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 0x7fc6840b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc683fff640 1 -- 172.21.15.29:0/1917380189 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688003bd0 con 0x5633c7d25f60 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc683fff640 1 -- 172.21.15.29:0/1917380189 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc688016030 con 0x5633c7d25f60 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc683fff640 1 -- 172.21.15.29:0/1917380189 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688004bb0 con 0x5633c7d25f60 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.669+0000 7fc68effd640 1 -- 172.21.15.29:0/2912864828 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc690108860 con 0x7fc698109df0 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1917380189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 msgr2=0x5633c80cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc69dc23640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 0x7fc6840b8b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5633c80cab80 tx=0x7fc69400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b6c0 tx=0x7fc68800bb80 comp rx=0 tx=0).stop 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1917380189 shutdown_connections 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d05280 0x5633c80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1917380189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d25f60 0x5633c80cbf00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1917380189 >> 172.21.15.29:0/1917380189 conn(0x5633c7d1a350 msgr2=0x5633c80c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1917380189 shutdown_connections 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1917380189 wait complete. 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 -- start start 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.670+0000 7fc6a63fb740 1 Processor -- start 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5633c7e945f0 con 0x5633c7cfd0a0 2024-09-21T01:47:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5633c7fce010 con 0x5633c7d25f60 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633c80da680 con 0x5633c7d05280 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc6a63fb740 10 client.0 _ll_register_callbacks cb 0x5633c802d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58056/0 (socket says 172.21.15.29:58056) 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 -- 172.21.15.29:0/1935407182 learned_addr learned my addr 172.21.15.29:0/1935407182 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69d422640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69e424640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 -- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 msgr2=0x5633c80d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 -- 172.21.15.29:0/1935407182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 msgr2=0x5633c80e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80e39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69dc23640 1 -- 172.21.15.29:0/1935407182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c7ffba00 con 0x5633c7cfd0a0 2024-09-21T01:47:40.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69d422640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.671+0000 7fc69e424640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc69dc23640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6940107a0 tx=0x7fc6940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a76604d399d17a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6817fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69400fa80 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc69400fc20 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc688016030 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc69400fe20 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6a63fb740 10 client.15186 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c80e65f0 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.672+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc688016030 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc69401f4a0 con 0x5633c7cfd0a0 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:47:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 20 client.15186 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7ce48e0 0x5633c80ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 4 client.15186 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:47:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c8045b60 con 0x5633c7ce48e0 2024-09-21T01:47:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.673+0000 7fc69d422640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7ce48e0 0x5633c80ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.674+0000 7fc6817fa640 10 client.15186 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.674+0000 7fc69d422640 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7ce48e0 0x5633c80ea0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=129389ec5e3a3892 server_cookie=674b956b0950a421 in_seq=0 out_seq=0 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.674+0000 7fc6817fa640 1 -- 172.21.15.29:0/1935407182 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633c8045b60 con 0x5633c7ce48e0 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.674+0000 7fc6817fa640 10 client.15186 handle_command_reply: tid=0 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 client.15186 shutdown 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 msgr2=0x5633c80d7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6940107a0 tx=0x7fc6940046c0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 shutdown_connections 2024-09-21T01:47:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7ce48e0 0x5633c80ea0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6817fa640 0 client.15186 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/1935407182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80d7780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 >> 172.21.15.29:0/1935407182 conn(0x5633c7d1a350 msgr2=0x5633c80cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 shutdown_connections 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 1 -- 172.21.15.29:0/1935407182 wait complete. 2024-09-21T01:47:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.675+0000 7fc6a63fb740 20 client.15186 trim_cache size 0 max 16384 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.693+0000 7fc6a63fb740 1 Processor -- start 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.693+0000 7fc6a63fb740 1 -- start start 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5633c80e83f0 con 0x5633c7d05280 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5633c80e8ac0 con 0x5633c7d25f60 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633c80e9190 con 0x5633c7cfd0a0 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58066/0 (socket says 172.21.15.29:58066) 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 -- 172.21.15.29:0/3501048705 learned_addr learned my addr 172.21.15.29:0/3501048705 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69dc23640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69e424640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 -- 172.21.15.29:0/3501048705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 msgr2=0x5633c80e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 -- 172.21.15.29:0/3501048705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 msgr2=0x5633c80d6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80d6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 -- 172.21.15.29:0/3501048705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c80ced60 con 0x5633c7d05280 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69dc23640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc69d422640 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b6c0 tx=0x7fc688003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1256d7fdbda1bde3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.694+0000 7fc6837fe640 1 -- 172.21.15.29:0/3501048705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688005030 con 0x5633c7d05280 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6837fe640 1 -- 172.21.15.29:0/3501048705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc688016070 con 0x5633c7d05280 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6837fe640 1 -- 172.21.15.29:0/3501048705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688004de0 con 0x5633c7d05280 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3501048705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 msgr2=0x5633c80e3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b6c0 tx=0x7fc688003e60 comp rx=0 tx=0).stop 2024-09-21T01:47:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3501048705 shutdown_connections 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80d6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7d05280 0x5633c80e3e50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3501048705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7cfd0a0 0x5633c80e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3501048705 >> 172.21.15.29:0/3501048705 conn(0x5633c7d1a350 msgr2=0x5633c80d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3501048705 shutdown_connections 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.695+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3501048705 wait complete. 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 -- start start 2024-09-21T01:47:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 Processor -- start 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80ec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5633c80e9f30 con 0x5633c7cfd0a0 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5633c80e9860 con 0x5633c7d25f60 2024-09-21T01:47:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633c80ea600 con 0x5633c7d05280 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc6a63fb740 10 client.0 _ll_register_callbacks cb 0x7fc69400ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc69dc23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc69dc23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58074/0 (socket says 172.21.15.29:58074) 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc69dc23640 1 -- 172.21.15.29:0/3114646497 learned_addr learned my addr 172.21.15.29:0/3114646497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.696+0000 7fc69e424640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69d422640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 -- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 msgr2=0x5633c80ebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80ebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 -- 172.21.15.29:0/3114646497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 msgr2=0x5633c80ec420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80ec420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 -- 172.21.15.29:0/3114646497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c80ecac0 con 0x5633c7cfd0a0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc69dc23640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc694006010 tx=0x7fc69400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f92672bab5c19c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6817fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69400f880 con 0x5633c7cfd0a0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc69400fa20 con 0x5633c7cfd0a0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc688016070 con 0x5633c7cfd0a0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc694041050 con 0x5633c7cfd0a0 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6a63fb740 10 client.15192 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.697+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c8155580 con 0x5633c7cfd0a0 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc688016070 con 0x5633c7cfd0a0 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc69403b400 con 0x5633c7cfd0a0 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 20 client.15192 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7cb9c90 0x5633c8158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 4 client.15192 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "status"}) -- 0x5633c8041610 con 0x5633c7cb9c90 2024-09-21T01:47:40.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.698+0000 7fc69d422640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7cb9c90 0x5633c8158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6817fa640 10 client.15192 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc69d422640 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7cb9c90 0x5633c8158d60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c985079fd35e31cd server_cookie=c6caa56a293c6b0a in_seq=0 out_seq=0 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6817fa640 1 -- 172.21.15.29:0/3114646497 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5633c8041610 con 0x5633c7cb9c90 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6817fa640 10 client.15192 handle_command_reply: tid=0 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 client.15192 shutdown 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 msgr2=0x5633c80f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc694006010 tx=0x7fc69400b4a0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 shutdown_connections 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5633c7cb9c90 0x5633c8158d60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633c7d25f60 0x5633c80ec420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633c7d05280 0x5633c80ebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 --2- 172.21.15.29:0/3114646497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5633c7cfd0a0 0x5633c80f2a30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6817fa640 0 client.15192 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 >> 172.21.15.29:0/3114646497 conn(0x5633c7d1a350 msgr2=0x5633c80d5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 shutdown_connections 2024-09-21T01:47:40.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 1 -- 172.21.15.29:0/3114646497 wait complete. 2024-09-21T01:47:40.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.699+0000 7fc6a63fb740 20 client.15192 trim_cache size 0 max 16384 2024-09-21T01:47:40.710 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:47:40.710 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:47:40.710 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 2, 2024-09-21T01:47:40.710 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24640, 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 81.528584777999995, 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 109.460139864 2024-09-21T01:47:40.711 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:47:40.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 msgr2=0x7fc6840b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 0x7fc6840b8b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5633c80cab80 tx=0x7fc69400b410 comp rx=0 tx=0).stop 2024-09-21T01:47:40.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 msgr2=0x7fc698076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:40.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc690005cc0 tx=0x7fc69002f380 comp rx=0 tx=0).stop 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 shutdown_connections 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc6840b6620 0x7fc6840b8b30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc698109df0 0x7fc698076190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6981094a0 0x7fc698075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 --2- 172.21.15.29:0/2912864828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc698103b60 0x7fc698075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 >> 172.21.15.29:0/2912864828 conn(0x7fc69807b850 msgr2=0x7fc698107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 shutdown_connections 2024-09-21T01:47:40.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:40.700+0000 7fc69fe87640 1 -- 172.21.15.29:0/2912864828 wait complete. 2024-09-21T01:47:41.394 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T01:47:41.650 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:42.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:41 smithi029 ceph-mon[31228]: pgmap v340: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:42.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 -- 172.21.15.29:0/2181725972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 msgr2=0x7f19b810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 --2- 172.21.15.29:0/2181725972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b810d8d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009900 tx=0x7f19ac02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:42.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 -- 172.21.15.29:0/2181725972 shutdown_connections 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 --2- 172.21.15.29:0/2181725972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b810de60 0x7f19b8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 --2- 172.21.15.29:0/2181725972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b810d8d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 --2- 172.21.15.29:0/2181725972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8104a30 0x7f19b8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 -- 172.21.15.29:0/2181725972 >> 172.21.15.29:0/2181725972 conn(0x7f19b8079760 msgr2=0x7f19b8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 -- 172.21.15.29:0/2181725972 shutdown_connections 2024-09-21T01:47:42.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.139+0000 7f19be643640 1 -- 172.21.15.29:0/2181725972 wait complete. 2024-09-21T01:47:42.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 Processor -- start 2024-09-21T01:47:42.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 -- start start 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b8104a30 0x7f19b819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b810de60 0x7f19b819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8113aa0 con 0x7f19b8105380 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8113920 con 0x7f19b8104a30 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19be643640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8113c20 con 0x7f19b810de60 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60748/0 (socket says 172.21.15.29:60748) 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19b77fe640 1 -- 172.21.15.29:0/1495112984 learned_addr learned my addr 172.21.15.29:0/1495112984 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19b7fff640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b8104a30 0x7f19b819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.140+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b810de60 0x7f19b819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 -- 172.21.15.29:0/1495112984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b810de60 msgr2=0x7f19b819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b810de60 0x7f19b819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 -- 172.21.15.29:0/1495112984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b8104a30 msgr2=0x7f19b819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b8104a30 0x7f19b819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 -- 172.21.15.29:0/1495112984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b81a0440 con 0x7f19b8105380 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b77fe640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f19ac02a930 tx=0x7f19ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b57fa640 1 -- 172.21.15.29:0/1495112984 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac03f030 con 0x7f19b8105380 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b57fa640 1 -- 172.21.15.29:0/1495112984 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ac003aa0 con 0x7f19b8105380 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b81a06d0 con 0x7f19b8105380 2024-09-21T01:47:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19b57fa640 1 -- 172.21.15.29:0/1495112984 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac035070 con 0x7f19b8105380 2024-09-21T01:47:42.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.141+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b81a6e90 con 0x7f19b8105380 2024-09-21T01:47:42.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 Processor -- start 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 -- start start 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fedbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19b57fa640 1 -- 172.21.15.29:0/1495112984 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f19ac042030 con 0x7f19b8105380 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724fedc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55724fe53c10 con 0x55724fb0d0a0 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55724fe43040 con 0x55724fb15280 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55724fe46e50 con 0x55724fb35f60 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60756/0 (socket says 172.21.15.29:60756) 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19b7fff640 1 -- 172.21.15.29:0/1645215496 learned_addr learned my addr 172.21.15.29:0/1645215496 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fedbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.144+0000 7f19b77fe640 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724fedc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 -- 172.21.15.29:0/1645215496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 msgr2=0x55724fedbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fedbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 -- 172.21.15.29:0/1645215496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 msgr2=0x55724fedc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724fedc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 -- 172.21.15.29:0/1645215496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724fba1540 con 0x55724fb0d0a0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b7fff640 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bcb0 tx=0x7f19a80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb7399f6573f6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f199e7fc640 1 -- 172.21.15.29:0/1645215496 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8005020 con 0x55724fb0d0a0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f199e7fc640 1 -- 172.21.15.29:0/1645215496 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a80047a0 con 0x55724fb0d0a0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b57fa640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 0x7f19a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f199e7fc640 1 -- 172.21.15.29:0/1645215496 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8011640 con 0x55724fb0d0a0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 0x7f19a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.145+0000 7f19b57fa640 1 -- 172.21.15.29:0/1495112984 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f19ac107b70 con 0x7f19b8105380 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 -- 172.21.15.29:0/1645215496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 msgr2=0x55724fedc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 0x7f19a00b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f19b0008e40 tx=0x7f19b00085f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bcb0 tx=0x7f19a80107a0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 -- 172.21.15.29:0/1645215496 shutdown_connections 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724fedc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fedc440 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/1645215496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fedbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 -- 172.21.15.29:0/1645215496 >> 172.21.15.29:0/1645215496 conn(0x55724fb2a350 msgr2=0x55724fed9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 -- 172.21.15.29:0/1645215496 shutdown_connections 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.146+0000 7f19c4bb7740 1 -- 172.21.15.29:0/1645215496 wait complete. 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 -- start start 2024-09-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 Processor -- start 2024-09-21T01:47:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fee7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55724fca45f0 con 0x55724fb0d0a0 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55724fdde010 con 0x55724fb35f60 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55724feea6e0 con 0x55724fb15280 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60770/0 (socket says 172.21.15.29:60770) 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19b7fff640 1 -- 172.21.15.29:0/862664515 learned_addr learned my addr 172.21.15.29:0/862664515 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.147+0000 7f19c4bb7740 10 client.0 _ll_register_callbacks cb 0x55724fe3d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b77fe640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fee7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 -- 172.21.15.29:0/862664515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 msgr2=0x55724fee7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fee7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 -- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 msgr2=0x55724fef3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 -- 172.21.15.29:0/862664515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724fe0ba00 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b77fe640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fee7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19b7fff640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bc80 tx=0x7f19a800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8d1e47554cccf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a802b060 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f19a8004b60 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a80047a0 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19a8005000 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19c4bb7740 10 client.15204 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.148+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55724fef65d0 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a80047a0 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19a8011ba0 con 0x55724fb0d0a0 2024-09-21T01:47:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 20 client.15204 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724faf48e0 0x55724fef9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 4 client.15204 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724fe55b60 con 0x55724faf48e0 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.149+0000 7f19b77fe640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724faf48e0 0x55724fef9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.150+0000 7f19b77fe640 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724faf48e0 0x55724fef9c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb1546edd7760e12 server_cookie=751aebc2a06e6e16 in_seq=0 out_seq=0 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.150+0000 7f19977fe640 10 client.15204 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19977fe640 1 -- 172.21.15.29:0/862664515 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55724fe55b60 con 0x55724faf48e0 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19977fe640 10 client.15204 handle_command_reply: tid=0 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 client.15204 shutdown 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 msgr2=0x55724fee77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bc80 tx=0x7f19a800bfd0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 shutdown_connections 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724faf48e0 0x55724fef9c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fee7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/862664515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee77d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 >> 172.21.15.29:0/862664515 conn(0x55724fb2a350 msgr2=0x55724fedf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19977fe640 0 client.15204 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 shutdown_connections 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 1 -- 172.21.15.29:0/862664515 wait complete. 2024-09-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.151+0000 7f19c4bb7740 20 client.15204 trim_cache size 0 max 16384 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.169+0000 7f19c4bb7740 1 Processor -- start 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.169+0000 7f19c4bb7740 1 -- start start 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fef3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55724fef7f80 con 0x55724fb0d0a0 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55724fef8650 con 0x55724fb35f60 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55724fef8d20 con 0x55724fb15280 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60780/0 (socket says 172.21.15.29:60780) 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 -- 172.21.15.29:0/3878833561 learned_addr learned my addr 172.21.15.29:0/3878833561 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b77fe640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fef3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 -- 172.21.15.29:0/3878833561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 msgr2=0x55724fef3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fef3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 -- 172.21.15.29:0/3878833561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 msgr2=0x55724fef3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 -- 172.21.15.29:0/3878833561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724fede790 con 0x55724fb0d0a0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b77fe640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fef3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f19b7fff640 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19a8010ca0 tx=0x7f19a8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694614ac4ad7b1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.170+0000 7f199dffb640 1 -- 172.21.15.29:0/3878833561 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8036070 con 0x55724fb0d0a0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f199dffb640 1 -- 172.21.15.29:0/3878833561 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8004780 con 0x55724fb0d0a0 2024-09-21T01:47:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f199dffb640 1 -- 172.21.15.29:0/3878833561 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a802b040 con 0x55724fb0d0a0 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 -- 172.21.15.29:0/3878833561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 msgr2=0x55724fee2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19a8010ca0 tx=0x7f19a8005c70 comp rx=0 tx=0).stop 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 -- 172.21.15.29:0/3878833561 shutdown_connections 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb35f60 0x55724fef3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb15280 0x55724fef3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/3878833561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724fee2020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 -- 172.21.15.29:0/3878833561 >> 172.21.15.29:0/3878833561 conn(0x55724fb2a350 msgr2=0x55724fee7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 -- 172.21.15.29:0/3878833561 shutdown_connections 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.171+0000 7f19c4bb7740 1 -- 172.21.15.29:0/3878833561 wait complete. 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 -- start start 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 Processor -- start 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724ff01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fefb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55724fef9ac0 con 0x55724fb0d0a0 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55724fef93f0 con 0x55724fb15280 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55724fefa190 con 0x55724fb35f60 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.172+0000 7f19c4bb7740 10 client.0 _ll_register_callbacks cb 0x7f19ac10b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60792/0 (socket says 172.21.15.29:60792) 2024-09-21T01:47:42.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 -- 172.21.15.29:0/2863663232 learned_addr learned my addr 172.21.15.29:0/2863663232 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b77fe640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724ff01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 -- 172.21.15.29:0/2863663232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 msgr2=0x55724fefb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fefb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fefb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 -- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 msgr2=0x55724ff01580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724ff01580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 -- 172.21.15.29:0/2863663232 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724fefbc00 con 0x55724fb0d0a0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19bcbe0640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fefb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19b7fff640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19a8010e80 tx=0x7f19a8004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad14226d5597e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8036050 con 0x55724fb0d0a0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f19a801ba40 con 0x55724fb0d0a0 2024-09-21T01:47:42.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.173+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a8004780 con 0x55724fb0d0a0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19a802b040 con 0x55724fb0d0a0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 10 client.15210 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55724ff004b0 con 0x55724fb0d0a0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8004780 con 0x55724fb0d0a0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19a8011650 con 0x55724fb0d0a0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 20 client.15210 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724fac9c90 0x55724ff67e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 4 client.15210 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "status"}) -- 0x55724fe51610 con 0x55724fac9c90 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.174+0000 7f19b77fe640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724fac9c90 0x55724ff67e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:42.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19977fe640 10 client.15210 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19b77fe640 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724fac9c90 0x55724ff67e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4639078dfb5b27ad server_cookie=5c89716e14c7baf6 in_seq=0 out_seq=0 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19977fe640 1 -- 172.21.15.29:0/2863663232 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55724fe51610 con 0x55724fac9c90 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19977fe640 10 client.15210 handle_command_reply: tid=0 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 client.15210 shutdown 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 msgr2=0x55724ff00f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19a8010e80 tx=0x7f19a8004e80 comp rx=0 tx=0).stop 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 shutdown_connections 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55724fac9c90 0x55724ff67e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55724fb35f60 0x55724fefb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55724fb15280 0x55724ff01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 --2- 172.21.15.29:0/2863663232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55724fb0d0a0 0x55724ff00f50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 >> 172.21.15.29:0/2863663232 conn(0x55724fb2a350 msgr2=0x55724fee2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19977fe640 0 client.15210 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:47:42.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 shutdown_connections 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.175+0000 7f19c4bb7740 1 -- 172.21.15.29:0/2863663232 wait complete. 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19c4bb7740 20 client.15210 trim_cache size 0 max 16384 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 1, 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24644, 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:47:42.182 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 84.013097918, 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 113.33160191100001 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:47:42.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 msgr2=0x7f19a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 0x7f19a00b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f19b0008e40 tx=0x7f19b00085f0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 msgr2=0x7f19b819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f19ac02a930 tx=0x7f19ac004270 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 shutdown_connections 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f19a00b65a0 0x7f19a00b8ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b810de60 0x7f19b819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19b8105380 0x7f19b819aa70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 --2- 172.21.15.29:0/1495112984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b8104a30 0x7f19b819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 >> 172.21.15.29:0/1495112984 conn(0x7f19b8079760 msgr2=0x7f19b81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 shutdown_connections 2024-09-21T01:47:42.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:42.176+0000 7f19be643640 1 -- 172.21.15.29:0/1495112984 wait complete. 2024-09-21T01:47:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:41 smithi190 ceph-mon[34368]: pgmap v340: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:41 smithi066 ceph-mon[35001]: pgmap v340: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:43.014 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:47:43.274 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 -- 172.21.15.29:0/1525010354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 msgr2=0x7f8480110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 --2- 172.21.15.29:0/1525010354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f8480110620 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8478009ca0 tx=0x7f847802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 -- 172.21.15.29:0/1525010354 shutdown_connections 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 --2- 172.21.15.29:0/1525010354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f8480110620 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 --2- 172.21.15.29:0/1525010354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f84801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 --2- 172.21.15.29:0/1525010354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f8480103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 -- 172.21.15.29:0/1525010354 >> 172.21.15.29:0/1525010354 conn(0x7f848007b850 msgr2=0x7f848007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 -- 172.21.15.29:0/1525010354 shutdown_connections 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.757+0000 7f8486561640 1 -- 172.21.15.29:0/1525010354 wait complete. 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.758+0000 7f8486561640 1 Processor -- start 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.758+0000 7f8486561640 1 -- start start 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.758+0000 7f8486561640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f848019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8486561640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f848019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8486561640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8486561640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8480113ab0 con 0x7f8480109df0 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8486561640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8480113930 con 0x7f8480103b60 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8486561640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8480113c30 con 0x7f84801094a0 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f847ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f848019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60798/0 (socket says 172.21.15.29:60798) 2024-09-21T01:47:43.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f847ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f848019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44376/0 (socket says 172.21.15.29:44376) 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 -- 172.21.15.29:0/1861423588 learned_addr learned my addr 172.21.15.29:0/1861423588 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f848019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 -- 172.21.15.29:0/1861423588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 msgr2=0x7f848019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f848019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 -- 172.21.15.29:0/1861423588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 msgr2=0x7f848019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f848019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 -- 172.21.15.29:0/1861423588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84801a02f0 con 0x7f8480109df0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f848019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f8484afe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8478005cc0 tx=0x7f847802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.759+0000 7f847d7fa640 1 -- 172.21.15.29:0/1861423588 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478003bf0 con 0x7f8480109df0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.760+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84801a0580 con 0x7f8480109df0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.760+0000 7f847d7fa640 1 -- 172.21.15.29:0/1861423588 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8478003d90 con 0x7f8480109df0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.760+0000 7f847d7fa640 1 -- 172.21.15.29:0/1861423588 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847802fbc0 con 0x7f8480109df0 2024-09-21T01:47:43.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.760+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a0ac0 con 0x7f8480109df0 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.762+0000 7f848cad5740 1 Processor -- start 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.762+0000 7f847d7fa640 1 -- 172.21.15.29:0/1861423588 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f847802fd60 con 0x7f8480109df0 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 -- start start 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5e3280 0x560c3a9aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560c3a921c10 con 0x560c3a5db0a0 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560c3a911040 con 0x560c3a603f60 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f848cad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c3a914e50 con 0x560c3a5e3280 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60800/0 (socket says 172.21.15.29:60800) 2024-09-21T01:47:43.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 -- 172.21.15.29:0/2576792819 learned_addr learned my addr 172.21.15.29:0/2576792819 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847f7fe640 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847f7fe640 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5e3280 0x560c3a9aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847d7fa640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 0x7f84680b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 -- 172.21.15.29:0/2576792819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5e3280 msgr2=0x560c3a9aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5e3280 0x560c3a9aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 -- 172.21.15.29:0/2576792819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 msgr2=0x560c3a9a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.763+0000 7f847ffff640 1 -- 172.21.15.29:0/2576792819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c3a66f540 con 0x560c3a5db0a0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f847d7fa640 1 -- 172.21.15.29:0/1861423588 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8478108810 con 0x7f8480109df0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f8484afe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 0x7f84680b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f847ffff640 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f847000bd80 tx=0x7f84700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28526cead59027f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f84667fc640 1 -- 172.21.15.29:0/2576792819 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470005220 con 0x560c3a5db0a0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f84667fc640 1 -- 172.21.15.29:0/2576792819 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8470004c50 con 0x560c3a5db0a0 2024-09-21T01:47:43.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.764+0000 7f84667fc640 1 -- 172.21.15.29:0/2576792819 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700117c0 con 0x560c3a5db0a0 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 -- 172.21.15.29:0/2576792819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 msgr2=0x560c3a9aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f847000bd80 tx=0x7f84700107a0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f8484afe640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 0x7f84680b8ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84780049d0 tx=0x7f847802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 -- 172.21.15.29:0/2576792819 shutdown_connections 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5e3280 0x560c3a9aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9aa440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 --2- 172.21.15.29:0/2576792819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.765+0000 7f848cad5740 1 -- 172.21.15.29:0/2576792819 >> 172.21.15.29:0/2576792819 conn(0x560c3a5f8350 msgr2=0x560c3a9a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.766+0000 7f848cad5740 1 -- 172.21.15.29:0/2576792819 shutdown_connections 2024-09-21T01:47:43.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.766+0000 7f848cad5740 1 -- 172.21.15.29:0/2576792819 wait complete. 2024-09-21T01:47:43.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.766+0000 7f848cad5740 1 -- start start 2024-09-21T01:47:43.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.766+0000 7f848cad5740 1 Processor -- start 2024-09-21T01:47:43.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5db0a0 0x560c3a9b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560c3a7725f0 con 0x560c3a5e3280 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560c3a8ac010 con 0x560c3a603f60 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9bca10 con 0x560c3a5db0a0 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f848cad5740 10 client.0 _ll_register_callbacks cb 0x560c3a90b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60814/0 (socket says 172.21.15.29:60814) 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 -- 172.21.15.29:0/1924710362 learned_addr learned my addr 172.21.15.29:0/1924710362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5db0a0 0x560c3a9b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 -- 172.21.15.29:0/1924710362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5db0a0 msgr2=0x560c3a9b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5db0a0 0x560c3a9b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 -- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 msgr2=0x560c3a9c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 -- 172.21.15.29:0/1924710362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c3a8d9a00 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f847ffff640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8470005690 tx=0x7f84700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ed393a4effbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.767+0000 7f845f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000be70 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8470005010 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8470004c50 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8470005210 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c3a9c8940 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8470004c50 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f84700039d0 con 0x560c3a5e3280 2024-09-21T01:47:43.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 20 client.15222 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a5c28e0 0x560c3a9cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 4 client.15222 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.768+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c3a923b60 con 0x560c3a5c28e0 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.769+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a5c28e0 0x560c3a9cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.769+0000 7f845f7fe640 10 client.15222 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.769+0000 7f847f7fe640 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a5c28e0 0x560c3a9cbf90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7914cbe3a45a605a server_cookie=57197eb2b93598af in_seq=0 out_seq=0 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f845f7fe640 1 -- 172.21.15.29:0/1924710362 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c3a923b60 con 0x560c3a5c28e0 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f845f7fe640 10 client.15222 handle_command_reply: tid=0 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 client.15222 shutdown 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 msgr2=0x560c3a9ba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8470005690 tx=0x7f84700048f0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 shutdown_connections 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a5c28e0 0x560c3a9cbf90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a603f60 0x560c3a9c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9ba2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 --2- 172.21.15.29:0/1924710362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5db0a0 0x560c3a9b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f845f7fe640 0 client.15222 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 >> 172.21.15.29:0/1924710362 conn(0x560c3a5f8350 msgr2=0x560c3a9ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 shutdown_connections 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 1 -- 172.21.15.29:0/1924710362 wait complete. 2024-09-21T01:47:43.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.770+0000 7f848cad5740 20 client.15222 trim_cache size 0 max 16384 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.788+0000 7f848cad5740 1 Processor -- start 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.788+0000 7f848cad5740 1 -- start start 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5e3280 0x560c3a9b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9ca2d0 con 0x560c3a5db0a0 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9ca9a0 con 0x560c3a5e3280 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f848cad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9cb070 con 0x560c3a5c28e0 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60830/0 (socket says 172.21.15.29:60830) 2024-09-21T01:47:43.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 -- 172.21.15.29:0/3650546966 learned_addr learned my addr 172.21.15.29:0/3650546966 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847f7fe640 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5e3280 0x560c3a9b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 -- 172.21.15.29:0/3650546966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 msgr2=0x560c3a9b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847f7fe640 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 -- 172.21.15.29:0/3650546966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5e3280 msgr2=0x560c3a9b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5e3280 0x560c3a9b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 -- 172.21.15.29:0/3650546966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c3a9b80b0 con 0x560c3a5db0a0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.789+0000 7f847ffff640 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8470006010 tx=0x7f84700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e8ff3191f26aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f8465ffb640 1 -- 172.21.15.29:0/3650546966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000bba0 con 0x560c3a5db0a0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f8465ffb640 1 -- 172.21.15.29:0/3650546966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f847000be70 con 0x560c3a5db0a0 2024-09-21T01:47:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f8465ffb640 1 -- 172.21.15.29:0/3650546966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470004ba0 con 0x560c3a5db0a0 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 -- 172.21.15.29:0/3650546966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 msgr2=0x560c3a9b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8470006010 tx=0x7f84700107a0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 -- 172.21.15.29:0/3650546966 shutdown_connections 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5e3280 0x560c3a9b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5db0a0 0x560c3a9b4d60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 --2- 172.21.15.29:0/3650546966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.790+0000 7f848cad5740 1 -- 172.21.15.29:0/3650546966 >> 172.21.15.29:0/3650546966 conn(0x560c3a5f8350 msgr2=0x560c3a9b9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.791+0000 7f848cad5740 1 -- 172.21.15.29:0/3650546966 shutdown_connections 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.791+0000 7f848cad5740 1 -- 172.21.15.29:0/3650546966 wait complete. 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.791+0000 7f848cad5740 1 -- start start 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.791+0000 7f848cad5740 1 Processor -- start 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 0x560c3a9d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9cbe10 con 0x560c3a5e3280 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9cb740 con 0x560c3a5db0a0 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c3a9cc4e0 con 0x560c3a5c28e0 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f848cad5740 10 client.0 _ll_register_callbacks cb 0x7f847400d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:47:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60844/0 (socket says 172.21.15.29:60844) 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 -- 172.21.15.29:0/736255872 learned_addr learned my addr 172.21.15.29:0/736255872 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847ffff640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 0x560c3a9d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 -- 172.21.15.29:0/736255872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 msgr2=0x560c3a9d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 -- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 msgr2=0x560c3a9d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 0x560c3a9d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 -- 172.21.15.29:0/736255872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c3a9d1760 con 0x560c3a5e3280 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847ffff640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 0x560c3a9d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f847f7fe640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f84740148e0 tx=0x7f8474014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec3202adde0d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.792+0000 7f845f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:47:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474005020 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f84740051c0 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847000be70 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84740053c0 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 10 client.15228 fetch_fsmap learned FSMap version 27 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c3aa37680 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f847000be70 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8474037450 con 0x560c3a5e3280 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 20 client.15228 populate_metadata read hostname 'smithi029' 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a603f60 0x560c3aa3af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 4 client.15228 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:47:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.793+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "status"}) -- 0x560c3a91f610 con 0x560c3a603f60 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f847ffff640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a603f60 0x560c3aa3af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f845f7fe640 10 client.15228 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f847ffff640 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a603f60 0x560c3aa3af60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56dde18ca6c1795 server_cookie=2af11e854c5190ca in_seq=0 out_seq=0 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f845f7fe640 1 -- 172.21.15.29:0/736255872 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560c3a91f610 con 0x560c3a603f60 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f845f7fe640 10 client.15228 handle_command_reply: tid=0 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f848cad5740 1 client.15228 shutdown 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 msgr2=0x560c3a9d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.794+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f84740148e0 tx=0x7f8474014040 comp rx=0 tx=0).stop 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 shutdown_connections 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560c3a603f60 0x560c3aa3af60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560c3a5e3280 0x560c3a9d3230 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c3a5db0a0 0x560c3a9d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 --2- 172.21.15.29:0/736255872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c3a5c28e0 0x560c3a9d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 >> 172.21.15.29:0/736255872 conn(0x560c3a5f8350 msgr2=0x560c3a9b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f845f7fe640 0 client.15228 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 shutdown_connections 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 1 -- 172.21.15.29:0/736255872 wait complete. 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f848cad5740 20 client.15228 trim_cache size 0 max 16384 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24671, 2024-09-21T01:47:43.801 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 92.947678323000005, 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 103.573542476 2024-09-21T01:47:43.802 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 msgr2=0x7f84680b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 0x7f84680b8ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84780049d0 tx=0x7f847802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 msgr2=0x7f848019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8478005cc0 tx=0x7f847802f380 comp rx=0 tx=0).stop 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 shutdown_connections 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84680b65d0 0x7f84680b8ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8480109df0 0x7f848019fc80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84801094a0 0x7f848019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 --2- 172.21.15.29:0/1861423588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8480103b60 0x7f848019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:43.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.795+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 >> 172.21.15.29:0/1861423588 conn(0x7f848007b850 msgr2=0x7f84801074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:43.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.796+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 shutdown_connections 2024-09-21T01:47:43.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:43.796+0000 7f8486561640 1 -- 172.21.15.29:0/1861423588 wait complete. 2024-09-21T01:47:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:43 smithi029 ceph-mon[31228]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:43 smithi190 ceph-mon[34368]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:43 smithi066 ceph-mon[35001]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:44.710 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T01:47:44.710 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T01:47:44.726 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:47:44.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T01:47:44.989 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:45.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 -- 172.21.15.29:0/824001748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c1047e0 msgr2=0x7fe36c104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 --2- 172.21.15.29:0/824001748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c1047e0 0x7fe36c104bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe360009900 tx=0x7fe36002a480 comp rx=0 tx=0).stop 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 -- 172.21.15.29:0/824001748 shutdown_connections 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 --2- 172.21.15.29:0/824001748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 0x7fe36c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 --2- 172.21.15.29:0/824001748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c1047e0 0x7fe36c104bf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 --2- 172.21.15.29:0/824001748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c103750 0x7fe36c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 -- 172.21.15.29:0/824001748 >> 172.21.15.29:0/824001748 conn(0x7fe36c0ff0c0 msgr2=0x7fe36c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 -- 172.21.15.29:0/824001748 shutdown_connections 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 -- 172.21.15.29:0/824001748 wait complete. 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.460+0000 7fe374074640 1 Processor -- start 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 -- start start 2024-09-21T01:47:45.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c1047e0 0x7fe36c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 0x7fe36c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c10f400 con 0x7fe36c103750 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c10f280 con 0x7fe36c1047e0 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe374074640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c10f580 con 0x7fe36c105180 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60860/0 (socket says 172.21.15.29:60860) 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 -- 172.21.15.29:0/798845014 learned_addr learned my addr 172.21.15.29:0/798845014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe37160f640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c1047e0 0x7fe36c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:45.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe372611640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 0x7fe36c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 -- 172.21.15.29:0/798845014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 msgr2=0x7fe36c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 0x7fe36c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 -- 172.21.15.29:0/798845014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c1047e0 msgr2=0x7fe36c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c1047e0 0x7fe36c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 -- 172.21.15.29:0/798845014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c19ba10 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.461+0000 7fe371e10640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00eab0 tx=0x7fe35c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.462+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00ce40 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.462+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c19bd00 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.462+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35c011920 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.462+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c010a30 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.462+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a2400 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.463+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c10a910 con 0x7fe36c103750 2024-09-21T01:47:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.464+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe35c010bd0 con 0x7fe36c103750 2024-09-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.465+0000 7fe36affd640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 0x7fe3540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.465+0000 7fe37160f640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 0x7fe3540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.465+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe35c0e7b00 con 0x7fe36c103750 2024-09-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.466+0000 7fe37160f640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 0x7fe3540b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe360000c00 tx=0x7fe360031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.468+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe35c016030 con 0x7fe36c103750 2024-09-21T01:47:45.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.677+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe36c063800 con 0x7fe3540b65a0 2024-09-21T01:47:45.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.678+0000 7fe36affd640 1 -- 172.21.15.29:0/798845014 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe36c063800 con 0x7fe3540b65a0 2024-09-21T01:47:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:Backend: cephadm 2024-09-21T01:47:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:Available: Yes 2024-09-21T01:47:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:Paused: No 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 msgr2=0x7fe3540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 0x7fe3540b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe360000c00 tx=0x7fe360031400 comp rx=0 tx=0).stop 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 msgr2=0x7fe36c19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00eab0 tx=0x7fe35c00ef70 comp rx=0 tx=0).stop 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 shutdown_connections 2024-09-21T01:47:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3540b65a0 0x7fe3540b8ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36c105180 0x7fe36c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c1047e0 0x7fe36c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 --2- 172.21.15.29:0/798845014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c103750 0x7fe36c19a050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 >> 172.21.15.29:0/798845014 conn(0x7fe36c0ff0c0 msgr2=0x7fe36c107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 shutdown_connections 2024-09-21T01:47:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:45.681+0000 7fe374074640 1 -- 172.21.15.29:0/798845014 wait complete. 2024-09-21T01:47:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:45 smithi029 ceph-mon[31228]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:46.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:45 smithi066 ceph-mon[35001]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:45 smithi190 ceph-mon[34368]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:46.540 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T01:47:46.803 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:47.071 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:46 smithi029 ceph-mon[31228]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:47.128 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:46 smithi190 ceph-mon[34368]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:47.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:46 smithi066 ceph-mon[35001]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 -- 172.21.15.29:0/982677843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c104070 msgr2=0x7f9a3c10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 --2- 172.21.15.29:0/982677843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c104070 0x7f9a3c10aa10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009ca0 tx=0x7f9a3402a950 comp rx=0 tx=0).stop 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 -- 172.21.15.29:0/982677843 shutdown_connections 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 --2- 172.21.15.29:0/982677843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c104070 0x7f9a3c10aa10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 --2- 172.21.15.29:0/982677843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 --2- 172.21.15.29:0/982677843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a3c102640 0x7f9a3c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 -- 172.21.15.29:0/982677843 >> 172.21.15.29:0/982677843 conn(0x7f9a3c0fdfb0 msgr2=0x7f9a3c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 -- 172.21.15.29:0/982677843 shutdown_connections 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.296+0000 7f9a439f2640 1 -- 172.21.15.29:0/982677843 wait complete. 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 Processor -- start 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 -- start start 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c102640 0x7f9a3c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a3c104070 0x7f9a3c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10e270 con 0x7f9a3c1036d0 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10e0f0 con 0x7f9a3c104070 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a439f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10e3f0 con 0x7f9a3c102640 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60890/0 (socket says 172.21.15.29:60890) 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 -- 172.21.15.29:0/3474375272 learned_addr learned my addr 172.21.15.29:0/3474375272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a4178e640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c102640 0x7f9a3c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 -- 172.21.15.29:0/3474375272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c102640 msgr2=0x7f9a3c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c102640 0x7f9a3c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 -- 172.21.15.29:0/3474375272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a3c104070 msgr2=0x7f9a3c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a3c104070 0x7f9a3c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.297+0000 7f9a40f8d640 1 -- 172.21.15.29:0/3474375272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3c19b930 con 0x7f9a3c1036d0 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a40f8d640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c920 tx=0x7f9a3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30007c70 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3c19bc20 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a30007e10 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30018820 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.298+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3c1a23b0 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.299+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3c1a2580 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.301+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9a3000f450 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.301+0000 7f9a2e7fc640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 0x7f9a280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.302+0000 7f9a4178e640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 0x7f9a280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.302+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9a300e78b0 con 0x7f9a3c1036d0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.302+0000 7f9a4178e640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 0x7f9a280b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a380058c0 tx=0x7f9a3800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.305+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a30014030 con 0x7f9a3c1036d0 2024-09-21T01:47:47.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.502+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9a3c0fff70 con 0x7f9a280b65a0 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.515+0000 7f9a2e7fc640 1 -- 172.21.15.29:0/3474375272 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f9a3c0fff70 con 0x7f9a280b65a0 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stdout:mds.a smithi029 running (110s) 23s ago 109s 84.7M - 19.3.0-5120-g18432218 ddce307e1f32 b3c3b027cc23 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stdout:mds.b smithi066 running (107s) 39s ago 107s 94.2M - 19.3.0-5120-g18432218 ddce307e1f32 d706560110ff 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stdout:mds.c smithi190 running (2m) 39s ago 2m 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 b258dc1a7157 2024-09-21T01:47:47.516 INFO:teuthology.orchestra.run.smithi029.stdout:mds.d smithi029 running (2m) 23s ago 2m 81.5M - 19.3.0-5120-g18432218 ddce307e1f32 b60110428a8d 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.e smithi066 running (2m) 39s ago 2m 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 202cd76b07a0 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.f smithi190 running (2m) 39s ago 2m 87.9M - 19.3.0-5120-g18432218 ddce307e1f32 80ca97b51ab7 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.g smithi029 running (113s) 23s ago 113s 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 e76e03888d3c 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.h smithi066 running (119s) 39s ago 118s 88.4M - 19.3.0-5120-g18432218 ddce307e1f32 7d8ba37782dd 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.i smithi190 running (116s) 39s ago 116s 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 2c1753eadbb5 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.j smithi029 running (2m) 23s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 5a3bebcbc17c 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.k smithi066 running (2m) 39s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 e2af42667462 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mds.l smithi190 running (2m) 39s ago 2m 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 d95b20e31e9d 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mgr.x smithi029 *:9283,8765 running (9m) 23s ago 9m 591M - 19.3.0-5120-g18432218 ddce307e1f32 c936566d9865 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mgr.y smithi066 *:8443,8765 running (8m) 39s ago 8m 526M - 19.3.0-5120-g18432218 ddce307e1f32 214909ed47c4 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mgr.z smithi190 *:8443,8765 running (8m) 39s ago 8m 526M - 19.3.0-5120-g18432218 ddce307e1f32 38bdce38cac8 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mon.a smithi029 running (10m) 23s ago 10m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 68e5ba2ba7bf 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mon.b smithi066 running (8m) 39s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 14d13366ac40 2024-09-21T01:47:47.517 INFO:teuthology.orchestra.run.smithi029.stdout:mon.c smithi190 running (8m) 39s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 1a326e76f88c 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0 smithi029 running (7m) 23s ago 7m 119M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3f345437ca97 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1 smithi066 running (6m) 39s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 75a6f39d9974 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2 smithi190 running (6m) 39s ago 6m 125M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5a8955b00379 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3 smithi029 running (6m) 23s ago 6m 124M 4273M 19.3.0-5120-g18432218 ddce307e1f32 cef33e83099c 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4 smithi066 running (5m) 39s ago 5m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f2626f2c1d51 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5 smithi190 running (5m) 39s ago 5m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2dc86988c297 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6 smithi029 running (4m) 23s ago 4m 119M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3167c39b81dc 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7 smithi066 running (4m) 39s ago 4m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 e8acaea2482e 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8 smithi190 running (3m) 39s ago 3m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b0fb9bac760a 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9 smithi029 running (3m) 23s ago 3m 112M 4273M 19.3.0-5120-g18432218 ddce307e1f32 7401c93c6717 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10 smithi066 running (3m) 39s ago 3m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 67ae749cee58 2024-09-21T01:47:47.518 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11 smithi190 running (2m) 39s ago 2m 118M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5ad1a28a723a 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 msgr2=0x7f9a280b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 0x7f9a280b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a380058c0 tx=0x7f9a3800a2d0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 msgr2=0x7f9a3c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c920 tx=0x7f9a3000cde0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 shutdown_connections 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9a280b65a0 0x7f9a280b8ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a3c104070 0x7f9a3c19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a3c1036d0 0x7f9a3c19a600 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 --2- 172.21.15.29:0/3474375272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a3c102640 0x7f9a3c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:47.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 >> 172.21.15.29:0/3474375272 conn(0x7f9a3c0fdfb0 msgr2=0x7f9a3c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:47.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 shutdown_connections 2024-09-21T01:47:47.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:47.517+0000 7f9a439f2640 1 -- 172.21.15.29:0/3474375272 wait complete. 2024-09-21T01:47:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:47 smithi029 ceph-mon[31228]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:48.186 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T01:47:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:47 smithi190 ceph-mon[34368]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:47 smithi066 ceph-mon[35001]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:48.444 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:48.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 -- 172.21.15.29:0/3493302803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f470806b920 msgr2=0x7f47081063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 --2- 172.21.15.29:0/3493302803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f470806b920 0x7f47081063f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4700009900 tx=0x7f470002a410 comp rx=0 tx=0).stop 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 -- 172.21.15.29:0/3493302803 shutdown_connections 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 --2- 172.21.15.29:0/3493302803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4708109470 0x7f470810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 --2- 172.21.15.29:0/3493302803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708106930 0x7f4708108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 --2- 172.21.15.29:0/3493302803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f470806b920 0x7f47081063f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 -- 172.21.15.29:0/3493302803 >> 172.21.15.29:0/3493302803 conn(0x7f47080fdfb0 msgr2=0x7f4708100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 -- 172.21.15.29:0/3493302803 shutdown_connections 2024-09-21T01:47:48.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.964+0000 7f470f28f640 1 -- 172.21.15.29:0/3493302803 wait complete. 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 Processor -- start 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 -- start start 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470806b920 0x7f470819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 0x7f470819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:48.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f470810f3d0 con 0x7f4708106930 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f470810f250 con 0x7f4708109470 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470f28f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f470810f550 con 0x7f470806b920 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.965+0000 7f470c82a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470c82a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60906/0 (socket says 172.21.15.29:60906) 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470c82a640 1 -- 172.21.15.29:0/3594821360 learned_addr learned my addr 172.21.15.29:0/3594821360 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 0x7f470819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d02b640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470806b920 0x7f470819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 -- 172.21.15.29:0/3594821360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470806b920 msgr2=0x7f470819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470806b920 0x7f470819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 -- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 msgr2=0x7f470819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 -- 172.21.15.29:0/3594821360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470819ba30 con 0x7f4708109470 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470c82a640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:47:48.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470d82c640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 0x7f470819b210 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46f800be20 tx=0x7f46f800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8019070 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.966+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470819bd20 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.967+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80148f0 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.967+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8012840 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.967+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081a2420 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.967+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47080ffad0 con 0x7f4708109470 2024-09-21T01:47:48.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.969+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f46f801c070 con 0x7f4708109470 2024-09-21T01:47:48.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.970+0000 7f47067fc640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 0x7f46f00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:48.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.970+0000 7f470d02b640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 0x7f46f00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:48.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.970+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f46f80e7010 con 0x7f4708109470 2024-09-21T01:47:48.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.971+0000 7f470d02b640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 0x7f46f00b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4700000c00 tx=0x7f4700031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:48.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:48.973+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46f80b1b30 con 0x7f4708109470 2024-09-21T01:47:49.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:48 smithi029 ceph-mon[31228]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:49.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.170+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4708063800 con 0x7f46f00b65a0 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.176+0000 7f47067fc640 1 -- 172.21.15.29:0/3594821360 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4708063800 con 0x7f46f00b65a0 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:mds.all 12/12 41s ago 2m smithi029=a;smithi029=d;smithi029=g;smithi029=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:mgr 3/3 41s ago 8m smithi029=x;smithi066=y;smithi190=z;count:3 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:mon 3/3 41s ago 8m smithi029:172.21.15.29=a;smithi066:172.21.15.66=b;smithi190:172.21.15.190=c;count:3 2024-09-21T01:47:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:osd 12 41s ago - 2024-09-21T01:47:49.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 msgr2=0x7f46f00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:49.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 0x7f46f00b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4700000c00 tx=0x7f4700031400 comp rx=0 tx=0).stop 2024-09-21T01:47:49.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 msgr2=0x7f470819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:49.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 0x7f470819b210 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46f800be20 tx=0x7f46f800bf10 comp rx=0 tx=0).stop 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 shutdown_connections 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b65a0 0x7f46f00b8ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708109470 0x7f470819b210 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708106930 0x7f470819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 --2- 172.21.15.29:0/3594821360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470806b920 0x7f470819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 >> 172.21.15.29:0/3594821360 conn(0x7f47080fdfb0 msgr2=0x7f4708104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 shutdown_connections 2024-09-21T01:47:49.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:49.178+0000 7f470f28f640 1 -- 172.21.15.29:0/3594821360 wait complete. 2024-09-21T01:47:49.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:48 smithi190 ceph-mon[34368]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:49.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:48 smithi066 ceph-mon[35001]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:49.963 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T01:47:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:49 smithi029 ceph-mon[31228]: pgmap v344: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:50.223 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:49 smithi190 ceph-mon[34368]: pgmap v344: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:49 smithi066 ceph-mon[35001]: pgmap v344: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:50.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 -- 172.21.15.29:0/1357741680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 msgr2=0x7efe1c10b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:50.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 --2- 172.21.15.29:0/1357741680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c10b990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efe14009900 tx=0x7efe1402a410 comp rx=0 tx=0).stop 2024-09-21T01:47:50.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 -- 172.21.15.29:0/1357741680 shutdown_connections 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 --2- 172.21.15.29:0/1357741680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c10b990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 --2- 172.21.15.29:0/1357741680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 0x7efe1c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 --2- 172.21.15.29:0/1357741680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c1063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 -- 172.21.15.29:0/1357741680 >> 172.21.15.29:0/1357741680 conn(0x7efe1c0fdfa0 msgr2=0x7efe1c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 -- 172.21.15.29:0/1357741680 shutdown_connections 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.750+0000 7efe2409e640 1 -- 172.21.15.29:0/1357741680 wait complete. 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 Processor -- start 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 -- start start 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 0x7efe1c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f3d0 con 0x7efe1c109480 2024-09-21T01:47:50.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f250 con 0x7efe1c1068e0 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.751+0000 7efe2409e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f550 con 0x7efe1c06c600 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60920/0 (socket says 172.21.15.29:60920) 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 -- 172.21.15.29:0/38595619 learned_addr learned my addr 172.21.15.29:0/38595619 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe21639640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 0x7efe1c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe21e3a640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 -- 172.21.15.29:0/38595619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 msgr2=0x7efe1c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 -- 172.21.15.29:0/38595619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 msgr2=0x7efe1c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 0x7efe1c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 -- 172.21.15.29:0/38595619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c19b960 con 0x7efe1c109480 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe21e3a640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2263b640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efe14005ec0 tx=0x7efe14003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:50.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1403f070 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe14031c50 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c19bbf0 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14035030 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.752+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1a2180 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.754+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c0ffac0 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.755+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efe14042070 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.756+0000 7efe1affd640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 0x7efe040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.756+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7efe141084b0 con 0x7efe1c109480 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.756+0000 7efe21e3a640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 0x7efe040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.757+0000 7efe21e3a640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 0x7efe040b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005c40 tx=0x7efe0c0082f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:50.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.759+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe140ceb90 con 0x7efe1c109480 2024-09-21T01:47:50.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.956+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efe1c063800 con 0x7efe040b65a0 2024-09-21T01:47:50.959 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:50 smithi029 ceph-mon[31228]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:50.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.958+0000 7efe1affd640 1 -- 172.21.15.29:0/38595619 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efe1c063800 con 0x7efe040b65a0 2024-09-21T01:47:50.959 INFO:teuthology.orchestra.run.smithi029.stdout:HOST ADDR LABELS STATUS 2024-09-21T01:47:50.959 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029 172.21.15.29 2024-09-21T01:47:50.960 INFO:teuthology.orchestra.run.smithi029.stdout:smithi066 172.21.15.66 2024-09-21T01:47:50.960 INFO:teuthology.orchestra.run.smithi029.stdout:smithi190 172.21.15.190 2024-09-21T01:47:50.960 INFO:teuthology.orchestra.run.smithi029.stdout:3 hosts in cluster 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 msgr2=0x7efe040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 0x7efe040b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005c40 tx=0x7efe0c0082f0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 msgr2=0x7efe1c19b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efe14005ec0 tx=0x7efe14003c00 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 shutdown_connections 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efe040b65a0 0x7efe040b8ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe1c109480 0x7efe1c19b140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe1c1068e0 0x7efe1c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 --2- 172.21.15.29:0/38595619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe1c06c600 0x7efe1c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 >> 172.21.15.29:0/38595619 conn(0x7efe1c0fdfa0 msgr2=0x7efe1c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 shutdown_connections 2024-09-21T01:47:50.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:50.960+0000 7efe2409e640 1 -- 172.21.15.29:0/38595619 wait complete. 2024-09-21T01:47:51.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:50 smithi190 ceph-mon[34368]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:51.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:50 smithi066 ceph-mon[35001]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:51.692 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T01:47:51.956 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:52.077 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:51 smithi029 ceph-mon[31228]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:51 smithi190 ceph-mon[34368]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:52.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:51 smithi066 ceph-mon[35001]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 -- 172.21.15.29:0/1351197512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0102640 msgr2=0x7f83b0102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 --2- 172.21.15.29:0/1351197512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0102640 0x7f83b0102ab0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009900 tx=0x7f83a002a480 comp rx=0 tx=0).stop 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 -- 172.21.15.29:0/1351197512 shutdown_connections 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 --2- 172.21.15.29:0/1351197512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b0104070 0x7f83b010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 --2- 172.21.15.29:0/1351197512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b01036d0 0x7f83b0103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 --2- 172.21.15.29:0/1351197512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0102640 0x7f83b0102ab0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 -- 172.21.15.29:0/1351197512 >> 172.21.15.29:0/1351197512 conn(0x7f83b00fdfb0 msgr2=0x7f83b0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 -- 172.21.15.29:0/1351197512 shutdown_connections 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.475+0000 7f83b642f640 1 -- 172.21.15.29:0/1351197512 wait complete. 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 Processor -- start 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 -- start start 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b0102640 0x7f83b006b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b01036d0 0x7f83b019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:52.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e260 con 0x7f83b0104070 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e0e0 con 0x7f83b01036d0 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b642f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e3e0 con 0x7f83b0102640 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.476+0000 7f83b49cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45180/0 (socket says 172.21.15.29:45180) 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 -- 172.21.15.29:0/2441842052 learned_addr learned my addr 172.21.15.29:0/2441842052 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83a7fff640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b01036d0 0x7f83b019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83affff640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b0102640 0x7f83b006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 -- 172.21.15.29:0/2441842052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b0102640 msgr2=0x7f83b006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b0102640 0x7f83b006b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 -- 172.21.15.29:0/2441842052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b01036d0 msgr2=0x7f83b019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b01036d0 0x7f83b019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 -- 172.21.15.29:0/2441842052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b01a1a50 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b49cc640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f83a800eaa0 tx=0x7f83a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a800cdb0 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b01a1d40 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a8005020 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a80116d0 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.477+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b01a2170 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.478+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b0109820 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.480+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f83a80051d0 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.481+0000 7f83adffb640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 0x7f839c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.481+0000 7f83affff640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 0x7f839c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.481+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f83a80e7aa0 con 0x7f83b0104070 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.481+0000 7f83affff640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 0x7f839c0b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83a0031440 tx=0x7f83a00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:52.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.484+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83a8016030 con 0x7f83b0104070 2024-09-21T01:47:52.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.679+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f83b0063800 con 0x7f839c0b65a0 2024-09-21T01:47:52.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.681+0000 7f83adffb640 1 -- 172.21.15.29:0/2441842052 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f83b0063800 con 0x7f839c0b65a0 2024-09-21T01:47:52.682 INFO:teuthology.orchestra.run.smithi029.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T01:47:52.682 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009B480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-21T01:47:52.682 INFO:teuthology.orchestra.run.smithi029.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T01:47:52.682 INFO:teuthology.orchestra.run.smithi029.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 msgr2=0x7f839c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 0x7f839c0b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83a0031440 tx=0x7f83a00314a0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 msgr2=0x7f83b01a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f83a800eaa0 tx=0x7f83a800ef60 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 shutdown_connections 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f839c0b65a0 0x7f839c0b8ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0104070 0x7f83b01a1230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83b01036d0 0x7f83b019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 --2- 172.21.15.29:0/2441842052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b0102640 0x7f83b006b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 >> 172.21.15.29:0/2441842052 conn(0x7f83b00fdfb0 msgr2=0x7f83b01069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 shutdown_connections 2024-09-21T01:47:52.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:52.684+0000 7f83b642f640 1 -- 172.21.15.29:0/2441842052 wait complete. 2024-09-21T01:47:53.047 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:52 smithi029 ceph-mon[31228]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:52 smithi190 ceph-mon[34368]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:52 smithi066 ceph-mon[35001]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:53.551 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T01:47:53.560 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:47:53.560 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T01:47:53.822 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:54.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:53 smithi029 ceph-mon[31228]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:54.111 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:53 smithi029 ceph-mon[31228]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:53 smithi190 ceph-mon[34368]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:53 smithi190 ceph-mon[34368]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:53 smithi066 ceph-mon[35001]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:47:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:53 smithi066 ceph-mon[35001]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 -- 172.21.15.29:0/2469225709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 msgr2=0x7f4e58102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/2469225709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e58102ac0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009900 tx=0x7f4e5402a480 comp rx=0 tx=0).stop 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 -- 172.21.15.29:0/2469225709 shutdown_connections 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/2469225709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 0x7f4e5810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/2469225709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 0x7f4e58103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/2469225709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e58102ac0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 -- 172.21.15.29:0/2469225709 >> 172.21.15.29:0/2469225709 conn(0x7f4e580fdfe0 msgr2=0x7f4e58100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 -- 172.21.15.29:0/2469225709 shutdown_connections 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.326+0000 7f4e5f70a640 1 -- 172.21.15.29:0/2469225709 wait complete. 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 Processor -- start 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 -- start start 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 0x7f4e5819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 0x7f4e5819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f3f0 con 0x7f4e58102650 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f270 con 0x7f4e58104080 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5f70a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f570 con 0x7f4e581036e0 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45200/0 (socket says 172.21.15.29:45200) 2024-09-21T01:47:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 -- 172.21.15.29:0/760081418 learned_addr learned my addr 172.21.15.29:0/760081418 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5dca7640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 0x7f4e5819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5cca5640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 0x7f4e5819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 -- 172.21.15.29:0/760081418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 msgr2=0x7f4e5819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 0x7f4e5819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 -- 172.21.15.29:0/760081418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 msgr2=0x7f4e5819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 0x7f4e5819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.327+0000 7f4e5d4a6640 1 -- 172.21.15.29:0/760081418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819ba60 con 0x7f4e58102650 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e5d4a6640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e54004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5403f030 con 0x7f4e58102650 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e54003cb0 con 0x7f4e58102650 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819bcf0 con 0x7f4e58102650 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54035070 con 0x7f4e58102650 2024-09-21T01:47:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.328+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a2450 con 0x7f4e58102650 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.329+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58109830 con 0x7f4e58102650 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.330+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4e54042030 con 0x7f4e58102650 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.332+0000 7f4e4e7fc640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 0x7f4e440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.332+0000 7f4e5cca5640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 0x7f4e440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.332+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4e54108180 con 0x7f4e58102650 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.332+0000 7f4e5cca5640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 0x7f4e440b8b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009890 tx=0x7f4e48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:54.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.335+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e540ce8c0 con 0x7f4e58102650 2024-09-21T01:47:54.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.566+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4e5802e640 con 0x7f4e58102650 2024-09-21T01:47:54.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.566+0000 7f4e4e7fc640 1 -- 172.21.15.29:0/760081418 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3126 (secure 0 0 0) 0x7f4e54003e60 con 0x7f4e58102650 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:e27 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:btime 2024-09-21T01:46:22:201303+0000 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:legacy client fscid: 1 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:Filesystem 'cephfs' (1) 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:fs_name cephfs 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 27 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-21T01:46:10.616066+0000 2024-09-21T01:47:54.567 INFO:teuthology.orchestra.run.smithi029.stdout:modified 2024-09-21T01:46:22.201278+0000 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:tableserver 0 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:root 0 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:session_timeout 300 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:session_autoclose 300 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:max_file_size 1099511627776 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:max_xattr_size 65536 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:required_client_features {} 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:last_failure 0 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:last_failure_osd_epoch 0 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:max_mds 5 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:in 0,1,2,3,4 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:up {0=24671,1=24644,2=24640,3=24625,4=24620} 2024-09-21T01:47:54.568 INFO:teuthology.orchestra.run.smithi029.stdout:failed 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:damaged 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:stopped 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:data_pools [3] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:metadata_pool 2 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:inline_data disabled 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:balancer 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:bal_rank_mask -1 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:standby_count_wanted 0 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:qdb_cluster leader: 24671 members: 24671,24644,24640,24625,24620 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b{0:24671} state up:active seq 4 addr [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.f{0:24613} state up:standby-replay seq 1 addr [v2:172.21.15.190:6834/4140562793,v1:172.21.15.190:6835/4140562793] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.h{1:24644} state up:active seq 9 addr [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.k{1:24596} state up:standby-replay seq 1 addr [v2:172.21.15.66:6832/3428222627,v1:172.21.15.66:6833/3428222627] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.i{2:24640} state up:active seq 8 addr [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.569 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.l{2:24586} state up:standby-replay seq 1 addr [v2:172.21.15.190:6832/3991511259,v1:172.21.15.190:6833/3991511259] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c{3:24625} state up:active seq 10 addr [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a{3:14784} state up:standby-replay seq 1 addr [v2:172.21.15.29:6840/2160178023,v1:172.21.15.29:6841/2160178023] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.e{4:24620} state up:active seq 11 addr [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.g{4:14778} state up:standby-replay seq 1 addr [v2:172.21.15.29:6838/3540512645,v1:172.21.15.29:6839/3540512645] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:Standby daemons: 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.j{-1:14712} state up:standby seq 1 addr [v2:172.21.15.29:6834/3427004495,v1:172.21.15.29:6835/3427004495] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d{-1:14724} state up:standby seq 1 addr [v2:172.21.15.29:6836/333752891,v1:172.21.15.29:6837/333752891] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.568+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 msgr2=0x7f4e440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.568+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 0x7f4e440b8b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009890 tx=0x7f4e48008040 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.568+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 msgr2=0x7f4e5819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.568+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e54004340 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 shutdown_connections 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e440b6620 0x7f4e440b8b30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e58104080 0x7f4e5819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e581036e0 0x7f4e5819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 --2- 172.21.15.29:0/760081418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58102650 0x7f4e5819a0a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 >> 172.21.15.29:0/760081418 conn(0x7f4e580fdfe0 msgr2=0x7f4e58106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 shutdown_connections 2024-09-21T01:47:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:54.569+0000 7f4e5f70a640 1 -- 172.21.15.29:0/760081418 wait complete. 2024-09-21T01:47:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:54 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/760081418' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:47:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:54 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/760081418' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:47:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:54 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/760081418' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:47:55.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T01:47:55.662 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:56.060 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:55 smithi029 ceph-mon[31228]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:56.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.154+0000 7f88bdd4c640 1 -- 172.21.15.29:0/534655555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 msgr2=0x7f88b8103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:56.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.154+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/534655555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b8103ae0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac02a480 comp rx=0 tx=0).stop 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 -- 172.21.15.29:0/534655555 shutdown_connections 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/534655555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 0x7f88b810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/534655555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b8103ae0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/534655555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 0x7f88b8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 -- 172.21.15.29:0/534655555 >> 172.21.15.29:0/534655555 conn(0x7f88b80fdfd0 msgr2=0x7f88b8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 -- 172.21.15.29:0/534655555 shutdown_connections 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 -- 172.21.15.29:0/534655555 wait complete. 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.155+0000 7f88bdd4c640 1 Processor -- start 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 -- start start 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 0x7f88b806c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 0x7f88b81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e270 con 0x7f88b81036d0 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e0f0 con 0x7f88b8104070 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88bdd4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e3f0 con 0x7f88b8102640 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45232/0 (socket says 172.21.15.29:45232) 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 -- 172.21.15.29:0/1453304686 learned_addr learned my addr 172.21.15.29:0/1453304686 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b77fe640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 0x7f88b806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b7fff640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 0x7f88b81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 -- 172.21.15.29:0/1453304686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 msgr2=0x7f88b806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 0x7f88b806c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 -- 172.21.15.29:0/1453304686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 msgr2=0x7f88b81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 0x7f88b81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.156+0000 7f88b6ffd640 1 -- 172.21.15.29:0/1453304686 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a1aa0 con 0x7f88b81036d0 2024-09-21T01:47:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88b6ffd640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006010 tx=0x7f88ac004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac03f030 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac003cb0 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a1d30 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac035070 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.157+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a21c0 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.158+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b8109800 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.160+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f88ac042030 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.160+0000 7f88b4ff9640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 0x7f88a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.160+0000 7f88b77fe640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 0x7f88a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.160+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f88ac1082e0 con 0x7f88b81036d0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.161+0000 7f88b77fe640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 0x7f88a00b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:56.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.163+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88ac0cea20 con 0x7f88b81036d0 2024-09-21T01:47:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:55 smithi190 ceph-mon[34368]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:55 smithi066 ceph-mon[35001]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:47:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.357+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f88b802e640 con 0x7f88b81036d0 2024-09-21T01:47:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.358+0000 7f88b4ff9640 1 -- 172.21.15.29:0/1453304686 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v76) ==== 56+0+5265 (secure 0 0 0) 0x7f88ac003e60 con 0x7f88b81036d0 2024-09-21T01:47:56.358 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 76 2024-09-21T01:47:56.358 INFO:teuthology.orchestra.run.smithi029.stdout:fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-21T01:37:42.108556+0000 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:modified 2024-09-21T01:46:33.220142+0000 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:crush_version 26 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:full_ratio 0.95 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:backfillfull_ratio 0.9 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:nearfull_ratio 0.85 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:require_min_compat_client luminous 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:min_compat_client luminous 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:require_osd_release squid 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:stretch_mode_enabled false 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T01:47:56.359 INFO:teuthology.orchestra.run.smithi029.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:max_osd 12 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0 up in weight 1 up_from 8 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6802/271350255,v1:172.21.15.29:6803/271350255] [v2:172.21.15.29:6804/271350255,v1:172.21.15.29:6805/271350255] exists,up 3c63b551-55db-4486-b93e-1f5773c9cdaf 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1 up in weight 1 up_from 13 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/2183970412,v1:172.21.15.66:6801/2183970412] [v2:172.21.15.66:6802/2183970412,v1:172.21.15.66:6803/2183970412] exists,up 9f29e6aa-4705-4810-9e3a-0118bec1dd83 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2 up in weight 1 up_from 18 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/2887309374,v1:172.21.15.190:6801/2887309374] [v2:172.21.15.190:6802/2887309374,v1:172.21.15.190:6803/2887309374] exists,up ebe1d952-803c-4222-88cd-cff3887aaecc 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3 up in weight 1 up_from 24 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6810/1440460638,v1:172.21.15.29:6811/1440460638] [v2:172.21.15.29:6812/1440460638,v1:172.21.15.29:6813/1440460638] exists,up 0f101639-e43c-4377-8de2-63ffbe3ce76d 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4 up in weight 1 up_from 30 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/2683216793,v1:172.21.15.66:6809/2683216793] [v2:172.21.15.66:6810/2683216793,v1:172.21.15.66:6811/2683216793] exists,up 4ffbf866-c852-4333-ac24-19c595975987 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5 up in weight 1 up_from 36 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/4247249906,v1:172.21.15.190:6809/4247249906] [v2:172.21.15.190:6810/4247249906,v1:172.21.15.190:6811/4247249906] exists,up 429eb433-7213-450d-a624-bb75cff27b4d 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6 up in weight 1 up_from 41 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6818/797110744,v1:172.21.15.29:6819/797110744] [v2:172.21.15.29:6820/797110744,v1:172.21.15.29:6821/797110744] exists,up b3d0fb70-1bc7-4a2d-9a80-eff59ed00770 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7 up in weight 1 up_from 47 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/3795777782,v1:172.21.15.66:6817/3795777782] [v2:172.21.15.66:6818/3795777782,v1:172.21.15.66:6819/3795777782] exists,up 24bf68e6-9205-4bee-9d74-38c475daeada 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/1425209392,v1:172.21.15.190:6817/1425209392] [v2:172.21.15.190:6818/1425209392,v1:172.21.15.190:6819/1425209392] exists,up fafc4cd9-401d-402c-8248-1a219ef74397 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9 up in weight 1 up_from 58 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6826/246841085,v1:172.21.15.29:6827/246841085] [v2:172.21.15.29:6828/246841085,v1:172.21.15.29:6829/246841085] exists,up d46d9b84-5641-4633-9077-c086f62e0617 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10 up in weight 1 up_from 63 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/1772263510,v1:172.21.15.66:6825/1772263510] [v2:172.21.15.66:6826/1772263510,v1:172.21.15.66:6827/1772263510] exists,up cd901192-e9bc-46ef-835f-a4dd14265604 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11 up in weight 1 up_from 68 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/3508320462,v1:172.21.15.190:6825/3508320462] [v2:172.21.15.190:6826/3508320462,v1:172.21.15.190:6827/3508320462] exists,up 962f6e91-94b7-4a35-9aec-7011884d4b92 2024-09-21T01:47:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 2.6 [8,11] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 2.b [2,1] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 2.e [2,4] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 2.1c [8,11] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 3.0 [5,9] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 3.e [4,8] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 3.38 [3,9] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:pg_upmap_items 3.3c [4,9] 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/1068014806 expires 2024-09-22T01:38:31.990579+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/2194300330 expires 2024-09-22T01:38:31.990579+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:6801/3233923662 expires 2024-09-22T01:38:31.990579+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/363656689 expires 2024-09-22T01:38:04.746763+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/1115869563 expires 2024-09-22T01:38:31.990579+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/280632558 expires 2024-09-22T01:38:04.746763+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:0/1445197802 expires 2024-09-22T01:38:04.746763+0000 2024-09-21T01:47:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:6800/3233923662 expires 2024-09-22T01:38:31.990579+0000 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:6801/2164591336 expires 2024-09-22T01:38:04.746763+0000 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stdout:blocklist 172.21.15.29:6800/2164591336 expires 2024-09-22T01:38:04.746763+0000 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 msgr2=0x7f88a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 0x7f88a00b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a8005850 comp rx=0 tx=0).stop 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 msgr2=0x7f88b819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006010 tx=0x7f88ac004340 comp rx=0 tx=0).stop 2024-09-21T01:47:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 shutdown_connections 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f88a00b65a0 0x7f88a00b8ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88b8104070 0x7f88b81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88b81036d0 0x7f88b819ed50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 --2- 172.21.15.29:0/1453304686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b8102640 0x7f88b806c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.360+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 >> 172.21.15.29:0/1453304686 conn(0x7f88b80fdfd0 msgr2=0x7f88b81069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.361+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 shutdown_connections 2024-09-21T01:47:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:56.361+0000 7f88bdd4c640 1 -- 172.21.15.29:0/1453304686 wait complete. 2024-09-21T01:47:57.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:56 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1453304686' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:47:57.211 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T01:47:57.224 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:47:57.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:56 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1453304686' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:47:57.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:56 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1453304686' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:47:57.486 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:58.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 -- 172.21.15.29:0/479185575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89201083d0 msgr2=0x7f89201087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:58.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 --2- 172.21.15.29:0/479185575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89201083d0 0x7f89201087e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8914009900 tx=0x7f891402a410 comp rx=0 tx=0).stop 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 -- 172.21.15.29:0/479185575 shutdown_connections 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 --2- 172.21.15.29:0/479185575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 0x7f892010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 --2- 172.21.15.29:0/479185575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89201083d0 0x7f89201087e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 --2- 172.21.15.29:0/479185575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8920102ae0 0x7f8920102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 -- 172.21.15.29:0/479185575 >> 172.21.15.29:0/479185575 conn(0x7f8920079760 msgr2=0x7f8920079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 -- 172.21.15.29:0/479185575 shutdown_connections 2024-09-21T01:47:58.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.013+0000 7f89263b0640 1 -- 172.21.15.29:0/479185575 wait complete. 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 Processor -- start 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 -- start start 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 0x7f892019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 0x7f892019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8920113ad0 con 0x7f8920102ae0 2024-09-21T01:47:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8920113950 con 0x7f89201083d0 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f89263b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8920113c50 con 0x7f8920108d20 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f891ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f891ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45260/0 (socket says 172.21.15.29:45260) 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f891ffff640 1 -- 172.21.15.29:0/3124604661 learned_addr learned my addr 172.21.15.29:0/3124604661 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f892494d640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 0x7f892019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.014+0000 7f891f7fe640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 0x7f892019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 -- 172.21.15.29:0/3124604661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 msgr2=0x7f892019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 0x7f892019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 -- 172.21.15.29:0/3124604661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 msgr2=0x7f892019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 0x7f892019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 -- 172.21.15.29:0/3124604661 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89201a0390 con 0x7f8920102ae0 2024-09-21T01:47:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891f7fe640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 0x7f892019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891ffff640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f891000b4a0 tx=0x7f891000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910014070 con 0x7f8920102ae0 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89201a0680 con 0x7f8920102ae0 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8910010ac0 con 0x7f8920102ae0 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910017c30 con 0x7f8920102ae0 2024-09-21T01:47:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.015+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892010fec0 con 0x7f8920102ae0 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.016+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8920103d10 con 0x7f8920102ae0 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.018+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8910017070 con 0x7f8920102ae0 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.018+0000 7f891d7fa640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 0x7f89080b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.019+0000 7f891f7fe640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 0x7f89080b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.019+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f89100e6ab0 con 0x7f8920102ae0 2024-09-21T01:47:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.019+0000 7f891f7fe640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 0x7f89080b8ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f8914031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:58.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.022+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89100ad1f0 con 0x7f8920102ae0 2024-09-21T01:47:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:57 smithi190 ceph-mon[34368]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:58.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.249+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f892019b610 con 0x7f8920102ae0 2024-09-21T01:47:58.249 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:57 smithi029 ceph-mon[31228]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:58.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.249+0000 7f891d7fa640 1 -- 172.21.15.29:0/3124604661 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7f892019b610 con 0x7f8920102ae0 2024-09-21T01:47:58.250 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:47:58.250 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:47:58.250 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 msgr2=0x7f89080b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 0x7f89080b8ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f8914031400 comp rx=0 tx=0).stop 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 msgr2=0x7f892019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f891000b4a0 tx=0x7f891000b960 comp rx=0 tx=0).stop 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 shutdown_connections 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89080b65d0 0x7f89080b8ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8920108d20 0x7f892019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89201083d0 0x7f892019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 --2- 172.21.15.29:0/3124604661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8920102ae0 0x7f892019a4e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:58.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 >> 172.21.15.29:0/3124604661 conn(0x7f8920079760 msgr2=0x7f89201052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:58.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 shutdown_connections 2024-09-21T01:47:58.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:58.253+0000 7f89263b0640 1 -- 172.21.15.29:0/3124604661 wait complete. 2024-09-21T01:47:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:57 smithi066 ceph-mon[35001]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T01:47:58.998 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:58 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3124604661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:59.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:47:59.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:58 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3124604661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:59.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:58 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3124604661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:47:59.302 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:47:59.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 -- 172.21.15.29:0/850048419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c102a80 msgr2=0x7f144c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:59.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 --2- 172.21.15.29:0/850048419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c102a80 0x7f144c102e90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f143c009900 tx=0x7f143c02a410 comp rx=0 tx=0).stop 2024-09-21T01:47:59.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 -- 172.21.15.29:0/850048419 shutdown_connections 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 --2- 172.21.15.29:0/850048419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c108d10 0x7f144c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 --2- 172.21.15.29:0/850048419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 0x7f144c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 --2- 172.21.15.29:0/850048419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c102a80 0x7f144c102e90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 -- 172.21.15.29:0/850048419 >> 172.21.15.29:0/850048419 conn(0x7f144c079740 msgr2=0x7f144c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 -- 172.21.15.29:0/850048419 shutdown_connections 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.794+0000 7f1452a14640 1 -- 172.21.15.29:0/850048419 wait complete. 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.795+0000 7f1452a14640 1 Processor -- start 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.795+0000 7f1452a14640 1 -- start start 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c102a80 0x7f144c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 0x7f144c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f144c112a00 con 0x7f144c108d10 2024-09-21T01:47:59.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f144c112880 con 0x7f144c102a80 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1452a14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f144c112b80 con 0x7f144c1083c0 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45288/0 (socket says 172.21.15.29:45288) 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 -- 172.21.15.29:0/1496805599 learned_addr learned my addr 172.21.15.29:0/1496805599 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f144bfff640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c102a80 0x7f144c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f144b7fe640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 0x7f144c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 -- 172.21.15.29:0/1496805599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 msgr2=0x7f144c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 0x7f144c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 -- 172.21.15.29:0/1496805599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c102a80 msgr2=0x7f144c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c102a80 0x7f144c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.796+0000 7f1450fb1640 1 -- 172.21.15.29:0/1496805599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19f360 con 0x7f144c108d10 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f1450fb1640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f144400b970 tx=0x7f144400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444019070 con 0x7f144c108d10 2024-09-21T01:47:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c19f650 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1444014440 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440075d0 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.797+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a5db0 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.798+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c103d00 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.799+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f144401c070 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.800+0000 7f14497fa640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 0x7f14340b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.800+0000 7f144bfff640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 0x7f14340b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.800+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f14440e69f0 con 0x7f144c108d10 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.801+0000 7f144bfff640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 0x7f14340b8ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f143c031440 tx=0x7f143c009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:47:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:47:59.804+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14440b14b0 con 0x7f144c108d10 2024-09-21T01:47:59.960 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:47:59 smithi029 ceph-mon[31228]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.034+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f144c19a520 con 0x7f144c108d10 2024-09-21T01:48:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.034+0000 7f14497fa640 1 -- 172.21.15.29:0/1496805599 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11970 (secure 0 0 0) 0x7f14440b63f0 con 0x7f144c108d10 2024-09-21T01:48:00.035 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:00.035 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"btime":"2024-09-21T01:46:22:201303+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:46:22.201278+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:48:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.036+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 msgr2=0x7f14340b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.036+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 0x7f14340b8ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f143c031440 tx=0x7f143c009560 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.036+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 msgr2=0x7f144c19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.036+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f144400b970 tx=0x7f144400be30 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 shutdown_connections 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14340b65a0 0x7f14340b8ab0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f144c108d10 0x7f144c19ec30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144c1083c0 0x7f144c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 --2- 172.21.15.29:0/1496805599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f144c102a80 0x7f144c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 >> 172.21.15.29:0/1496805599 conn(0x7f144c079740 msgr2=0x7f144c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 shutdown_connections 2024-09-21T01:48:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:00.037+0000 7f1452a14640 1 -- 172.21.15.29:0/1496805599 wait complete. 2024-09-21T01:48:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:47:59 smithi190 ceph-mon[34368]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:47:59 smithi066 ceph-mon[35001]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:00.828 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:48:01.085 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:01.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1496805599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:01.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:00 smithi029 ceph-mon[31228]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:48:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1496805599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:00 smithi190 ceph-mon[34368]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:48:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1496805599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:00 smithi066 ceph-mon[35001]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:48:01.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 -- 172.21.15.29:0/537024922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 msgr2=0x7fad2c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:01.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 --2- 172.21.15.29:0/537024922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c10f520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fad24009ca0 tx=0x7fad2402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 -- 172.21.15.29:0/537024922 shutdown_connections 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 --2- 172.21.15.29:0/537024922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c10f520 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 --2- 172.21.15.29:0/537024922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c1083a0 0x7fad2c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 --2- 172.21.15.29:0/537024922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c102a60 0x7fad2c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 -- 172.21.15.29:0/537024922 >> 172.21.15.29:0/537024922 conn(0x7fad2c079730 msgr2=0x7fad2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 -- 172.21.15.29:0/537024922 shutdown_connections 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.589+0000 7fad30f19640 1 -- 172.21.15.29:0/537024922 wait complete. 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 Processor -- start 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 -- start start 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c102a60 0x7fad2c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c1083a0 0x7fad2c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c113a80 con 0x7fad2c108cf0 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c113900 con 0x7fad2c102a60 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad30f19640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c113c00 con 0x7fad2c1083a0 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad2ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad2ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45306/0 (socket says 172.21.15.29:45306) 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.590+0000 7fad2ad9d640 1 -- 172.21.15.29:0/1510458286 learned_addr learned my addr 172.21.15.29:0/1510458286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2a59c640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c102a60 0x7fad2c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad29d9b640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c1083a0 0x7fad2c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 -- 172.21.15.29:0/1510458286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c1083a0 msgr2=0x7fad2c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c1083a0 0x7fad2c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 -- 172.21.15.29:0/1510458286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c102a60 msgr2=0x7fad2c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c102a60 0x7fad2c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 -- 172.21.15.29:0/1510458286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c1a0340 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad2ad9d640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fad24005cc0 tx=0x7fad2402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24003cf0 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c1a05d0 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad240044f0 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2402fc30 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.591+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c1a0b10 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.592+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2c102e70 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.595+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fad240046a0 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.596+0000 7fad1b7fe640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 0x7fad140b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.596+0000 7fad2a59c640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 0x7fad140b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.596+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fad24108b50 con 0x7fad2c108cf0 2024-09-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.596+0000 7fad2a59c640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 0x7fad140b8ae0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00abe0 tx=0x7fad1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:01.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.599+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad240cf230 con 0x7fad2c108cf0 2024-09-21T01:48:01.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.786+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad2c02e640 con 0x7fad2c108cf0 2024-09-21T01:48:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.788+0000 7fad1b7fe640 1 -- 172.21.15.29:0/1510458286 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7fad240d3610 con 0x7fad2c108cf0 2024-09-21T01:48:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:01.789 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:48:01.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.790+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 msgr2=0x7fad140b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:01.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.790+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 0x7fad140b8ae0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00abe0 tx=0x7fad1c00a2d0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.790+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 msgr2=0x7fad2c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:01.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.790+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fad24005cc0 tx=0x7fad2402f470 comp rx=0 tx=0).stop 2024-09-21T01:48:01.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 shutdown_connections 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fad140b65d0 0x7fad140b8ae0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad2c108cf0 0x7fad2c19fcd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad2c1083a0 0x7fad2c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 --2- 172.21.15.29:0/1510458286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad2c102a60 0x7fad2c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 >> 172.21.15.29:0/1510458286 conn(0x7fad2c079730 msgr2=0x7fad2c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 shutdown_connections 2024-09-21T01:48:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:01.791+0000 7fad30f19640 1 -- 172.21.15.29:0/1510458286 wait complete. 2024-09-21T01:48:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:01 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1510458286' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:01 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1510458286' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:01 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1510458286' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:02.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T01:46:10.616066+0000', 'modified': '2024-09-21T01:46:22.201278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24671, 'mds_1': 24644, 'mds_2': 24640, 'mds_3': 24625, 'mds_4': 24620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14778': {'gid': 14778, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6839/3540512645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 3540512645}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 3540512645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14784': {'gid': 14784, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6841/2160178023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6840', 'nonce': 2160178023}, {'type': 'v1', 'addr': '172.21.15.29:6841', 'nonce': 2160178023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/3991511259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 3991511259}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 3991511259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/3428222627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 3428222627}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 3428222627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/4140562793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 4140562793}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 4140562793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.66:6835/2455557806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2455557806}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2455557806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24625': {'gid': 24625, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/2047991561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 2047991561}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 2047991561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.190:6839/636701059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 636701059}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 636701059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.66:6837/1538999358', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 1538999358}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1538999358}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6839/2350524023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2350524023}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2350524023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671, 24644, 24640, 24625, 24620]} 2024-09-21T01:48:02.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T01:48:02.607 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T01:48:02.864 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:03.199 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:02 smithi029 ceph-mon[31228]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:03.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:02 smithi190 ceph-mon[34368]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2556677325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109ed0 msgr2=0x7f12e0110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2556677325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109ed0 0x7f12e0110650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009ca0 tx=0x7f12d802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2556677325 shutdown_connections 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2556677325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109ed0 0x7f12e0110650 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2556677325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109580 0x7f12e0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2556677325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2556677325 >> 172.21.15.29:0/2556677325 conn(0x7f12e0077800 msgr2=0x7f12e0077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2556677325 shutdown_connections 2024-09-21T01:48:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.293+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2556677325 wait complete. 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.294+0000 7f12e7cf1640 1 Processor -- start 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.294+0000 7f12e7cf1640 1 -- start start 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.294+0000 7f12e7cf1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.294+0000 7f12e7cf1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e7cf1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 0x7f12e0103ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e7cf1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113a90 con 0x7f12e0109580 2024-09-21T01:48:03.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e7cf1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113910 con 0x7f12e0101c20 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e7cf1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113c10 con 0x7f12e0109ed0 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51760/0 (socket says 172.21.15.29:51760) 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 -- 172.21.15.29:0/2897615702 learned_addr learned my addr 172.21.15.29:0/2897615702 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e628e640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 0x7f12e0103ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 -- 172.21.15.29:0/2897615702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 msgr2=0x7f12e0103ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 0x7f12e0103ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 -- 172.21.15.29:0/2897615702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 msgr2=0x7f12e0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 -- 172.21.15.29:0/2897615702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e01a8630 con 0x7f12e0109580 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e628e640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 0x7f12e0103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e528c640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d950 tx=0x7f12d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12d6ffd640 1 -- 172.21.15.29:0/2897615702 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0017070 con 0x7f12e0109580 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e01a8920 con 0x7f12e0109580 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12d6ffd640 1 -- 172.21.15.29:0/2897615702 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d0009c30 con 0x7f12e0109580 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.295+0000 7f12d6ffd640 1 -- 172.21.15.29:0/2897615702 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00047a0 con 0x7f12e0109580 2024-09-21T01:48:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.296+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e01a8e60 con 0x7f12e0109580 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.298+0000 7f12ee265740 1 Processor -- start 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.298+0000 7f12ee265740 1 -- start start 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.298+0000 7f12d6ffd640 1 -- 172.21.15.29:0/2897615702 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f12d0004020 con 0x7f12e0109580 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.298+0000 7f12ee265740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d165f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12ee265740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d165f8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12ee265740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12ee265740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d1656fc10 con 0x555d16231280 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12ee265740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555d1655f040 con 0x555d16251f60 2024-09-21T01:48:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12ee265740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555d16562e50 con 0x555d162290a0 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e628e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d165f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51766/0 (socket says 172.21.15.29:51766) 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 -- 172.21.15.29:0/271581510 learned_addr learned my addr 172.21.15.29:0/271581510 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d165f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 -- 172.21.15.29:0/271581510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 msgr2=0x555d165f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d165f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 -- 172.21.15.29:0/271581510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 msgr2=0x555d165f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d165f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 -- 172.21.15.29:0/271581510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d162bd540 con 0x555d16231280 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d165f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.299+0000 7f12e528c640 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f12d00e4ad0 tx=0x7f12d00ef8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c2879f6da38f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12d6ffd640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 0x7f12cc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12cbfff640 1 -- 172.21.15.29:0/271581510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00fa070 con 0x555d16231280 2024-09-21T01:48:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12e628e640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 0x7f12cc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12cbfff640 1 -- 172.21.15.29:0/271581510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d00acd20 con 0x555d16231280 2024-09-21T01:48:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12cbfff640 1 -- 172.21.15.29:0/271581510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00acb60 con 0x555d16231280 2024-09-21T01:48:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.300+0000 7f12d6ffd640 1 -- 172.21.15.29:0/2897615702 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12d00e71d0 con 0x7f12e0109580 2024-09-21T01:48:03.302 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:02 smithi066 ceph-mon[35001]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- 172.21.15.29:0/271581510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 msgr2=0x555d165f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12e628e640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 0x7f12cc0b8ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12d8004af0 tx=0x7f12d8005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f12d00e4ad0 tx=0x7f12d00ef8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- 172.21.15.29:0/271581510 shutdown_connections 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d165f8980 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d165f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 --2- 172.21.15.29:0/271581510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d165f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- 172.21.15.29:0/271581510 >> 172.21.15.29:0/271581510 conn(0x555d16246350 msgr2=0x555d165f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- 172.21.15.29:0/271581510 shutdown_connections 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- 172.21.15.29:0/271581510 wait complete. 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.301+0000 7f12ee265740 1 -- start start 2024-09-21T01:48:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 Processor -- start 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16603750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16603d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d163c05f0 con 0x555d16251f60 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555d164fa010 con 0x555d16231280 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555d166066e0 con 0x555d162290a0 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12ee265740 10 client.0 _ll_register_callbacks cb 0x555d16559340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12e628e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12e628e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51774/0 (socket says 172.21.15.29:51774) 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12e628e640 1 -- 172.21.15.29:0/2471945288 learned_addr learned my addr 172.21.15.29:0/2471945288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16603750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.302+0000 7f12e528c640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16603d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 -- 172.21.15.29:0/2471945288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 msgr2=0x555d16603750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16603750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 -- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 msgr2=0x555d16603d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16603d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 -- 172.21.15.29:0/2471945288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d16527a00 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16603750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12e628e640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12d8033030 tx=0x7f12d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb024b26db565f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8004250 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12d80043f0 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d00acd20 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12d8004630 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12ee265740 10 client.15300 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.303+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d166125e0 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d00acd20 con 0x555d16251f60 2024-09-21T01:48:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f12d8050070 con 0x555d16251f60 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 10 client.15300 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 20 client.15300 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d162108e0 0x555d16616130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 4 client.15300 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d16571b60 con 0x555d162108e0 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.304+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d162108e0 0x555d16616130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.305+0000 7f12c97fa640 10 client.15300 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:48:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.305+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d162108e0 0x555d16616130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6756407701ed39a4 server_cookie=e22fd68cdca1a442 in_seq=0 out_seq=0 2024-09-21T01:48:03.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.305+0000 7f12c97fa640 1 -- 172.21.15.29:0/2471945288 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d16571b60 con 0x555d162108e0 2024-09-21T01:48:03.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.305+0000 7f12c97fa640 10 client.15300 handle_command_reply: tid=0 2024-09-21T01:48:03.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.305+0000 7f12ee265740 1 client.15300 shutdown 2024-09-21T01:48:03.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 msgr2=0x555d1660fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12d8033030 tx=0x7f12d802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 shutdown_connections 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d162108e0 0x555d16616130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16251f60 0x555d1660fa30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16603d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12c97fa640 0 client.15300 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 --2- 172.21.15.29:0/2471945288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16603750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 >> 172.21.15.29:0/2471945288 conn(0x555d16246350 msgr2=0x555d165fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 shutdown_connections 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 1 -- 172.21.15.29:0/2471945288 wait complete. 2024-09-21T01:48:03.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.306+0000 7f12ee265740 20 client.15300 trim_cache size 0 max 16384 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.324+0000 7f12ee265740 1 Processor -- start 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.324+0000 7f12ee265740 1 -- start start 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16602510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d16251f60 0x555d165f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d16614470 con 0x555d162290a0 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555d16614b40 con 0x555d16231280 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12ee265740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555d16615210 con 0x555d16251f60 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51782/0 (socket says 172.21.15.29:51782) 2024-09-21T01:48:03.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 -- 172.21.15.29:0/1070514049 learned_addr learned my addr 172.21.15.29:0/1070514049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e528c640 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16602510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 -- 172.21.15.29:0/1070514049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d16251f60 msgr2=0x555d165f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d16251f60 0x555d165f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 -- 172.21.15.29:0/1070514049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 msgr2=0x555d16602510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16602510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 -- 172.21.15.29:0/1070514049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d16604080 con 0x555d162290a0 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.325+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0058c0 tx=0x7f12dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ae99b909258419 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12cb7fe640 1 -- 172.21.15.29:0/1070514049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc013c80 con 0x555d162290a0 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12cb7fe640 1 -- 172.21.15.29:0/1070514049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12dc01e030 con 0x555d162290a0 2024-09-21T01:48:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12cb7fe640 1 -- 172.21.15.29:0/1070514049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011970 con 0x555d162290a0 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 -- 172.21.15.29:0/1070514049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 msgr2=0x555d16601fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0058c0 tx=0x7f12dc005850 comp rx=0 tx=0).stop 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 -- 172.21.15.29:0/1070514049 shutdown_connections 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d16251f60 0x555d165f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16231280 0x555d16602510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 --2- 172.21.15.29:0/1070514049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d162290a0 0x555d16601fd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 -- 172.21.15.29:0/1070514049 >> 172.21.15.29:0/1070514049 conn(0x555d16246350 msgr2=0x555d165fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 -- 172.21.15.29:0/1070514049 shutdown_connections 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.326+0000 7f12ee265740 1 -- 172.21.15.29:0/1070514049 wait complete. 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 -- start start 2024-09-21T01:48:03.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 Processor -- start 2024-09-21T01:48:03.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16618910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d1661ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d16615fb0 con 0x555d16231280 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555d166158e0 con 0x555d16251f60 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555d16616680 con 0x555d162290a0 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.327+0000 7f12ee265740 10 client.0 _ll_register_callbacks cb 0x7f12d802f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51790/0 (socket says 172.21.15.29:51790) 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 -- 172.21.15.29:0/2953482925 learned_addr learned my addr 172.21.15.29:0/2953482925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16618910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e628e640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d1661ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 -- 172.21.15.29:0/2953482925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 msgr2=0x555d16618910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16618910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 -- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 msgr2=0x555d1661ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d1661ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 -- 172.21.15.29:0/2953482925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d1661b260 con 0x555d16231280 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e628e640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d1661ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12e528c640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12d0005fd0 tx=0x7f12d00e4b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1954524978a5cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00860f0 con 0x555d16231280 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12d00a9480 con 0x555d16231280 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12dc01e030 con 0x555d16231280 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12d0084120 con 0x555d16231280 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12ee265740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:03.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.328+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d16681c40 con 0x555d16231280 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12dc01e030 con 0x555d16231280 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f12d00b67a0 con 0x555d16231280 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 10 client.15306 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 20 client.15306 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d161e5c90 0x555d16685020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 4 client.15306 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "status"}) -- 0x555d1656d610 con 0x555d161e5c90 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.329+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d161e5c90 0x555d16685020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12e5a8d640 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d161e5c90 0x555d16685020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a021700c660338e9 server_cookie=abfd6c7fe5c99a9c in_seq=0 out_seq=0 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12c97fa640 10 client.15306 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12c97fa640 1 -- 172.21.15.29:0/2953482925 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555d1656d610 con 0x555d161e5c90 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12c97fa640 10 client.15306 handle_command_reply: tid=0 2024-09-21T01:48:03.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 client.15306 shutdown 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 msgr2=0x555d16618e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12d0005fd0 tx=0x7f12d00e4b00 comp rx=0 tx=0).stop 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 shutdown_connections 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x555d161e5c90 0x555d16685020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d16251f60 0x555d1661ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d16231280 0x555d16618e50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12c97fa640 0 client.15306 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 --2- 172.21.15.29:0/2953482925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d162290a0 0x555d16618910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 >> 172.21.15.29:0/2953482925 conn(0x555d16246350 msgr2=0x555d165f5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 shutdown_connections 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.330+0000 7f12ee265740 1 -- 172.21.15.29:0/2953482925 wait complete. 2024-09-21T01:48:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12ee265740 20 client.15306 trim_cache size 0 max 16384 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 4, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24620, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 102.137281857, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 141.973698993 2024-09-21T01:48:03.336 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 msgr2=0x7f12cc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 0x7f12cc0b8ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12d8004af0 tx=0x7f12d8005d80 comp rx=0 tx=0).stop 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 msgr2=0x7f12e0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d950 tx=0x7f12d000de10 comp rx=0 tx=0).stop 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 shutdown_connections 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12cc0b65a0 0x7f12cc0b8ab0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e0109ed0 0x7f12e0103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12e0109580 0x7f12e0103990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 --2- 172.21.15.29:0/2897615702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12e0101c20 0x7f12e0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 >> 172.21.15.29:0/2897615702 conn(0x7f12e0077800 msgr2=0x7f12e0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 shutdown_connections 2024-09-21T01:48:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:03.331+0000 7f12e7cf1640 1 -- 172.21.15.29:0/2897615702 wait complete. 2024-09-21T01:48:04.118 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T01:48:04.381 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 -- 172.21.15.29:0/158589692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c8108cf0 msgr2=0x7f68c810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 --2- 172.21.15.29:0/158589692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c8108cf0 0x7f68c810f520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009ca0 tx=0x7f68c002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 -- 172.21.15.29:0/158589692 shutdown_connections 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 --2- 172.21.15.29:0/158589692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c8108cf0 0x7f68c810f520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 --2- 172.21.15.29:0/158589692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c81083a0 0x7f68c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 --2- 172.21.15.29:0/158589692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8102a60 0x7f68c8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 -- 172.21.15.29:0/158589692 >> 172.21.15.29:0/158589692 conn(0x7f68c8079730 msgr2=0x7f68c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 -- 172.21.15.29:0/158589692 shutdown_connections 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.877+0000 7f68cce4d640 1 -- 172.21.15.29:0/158589692 wait complete. 2024-09-21T01:48:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 Processor -- start 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 -- start start 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 0x7f68c8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8108cf0 0x7f68c8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8112960 con 0x7f68c81083a0 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81127e0 con 0x7f68c8108cf0 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68cce4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8112ae0 con 0x7f68c8102a60 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68c5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68c5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51806/0 (socket says 172.21.15.29:51806) 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.878+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1737873560 learned_addr learned my addr 172.21.15.29:0/1737873560 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8108cf0 0x7f68c8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c659c640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 0x7f68c8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1737873560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 msgr2=0x7f68c8199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 0x7f68c8199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1737873560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8108cf0 msgr2=0x7f68c8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8108cf0 0x7f68c8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1737873560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81a8680 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c659c640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 0x7f68c8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c970 tx=0x7f68b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68bf7fe640 1 -- 172.21.15.29:0/1737873560 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0007bb0 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81a8910 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68bf7fe640 1 -- 172.21.15.29:0/1737873560 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b0007d50 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68bf7fe640 1 -- 172.21.15.29:0/1737873560 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0018730 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.879+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81a8e00 con 0x7f68c81083a0 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 Processor -- start 2024-09-21T01:48:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 -- start start 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68bf7fe640 1 -- 172.21.15.29:0/1737873560 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f68b000f990 con 0x7f68c81083a0 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c706f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c707980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55901c67ec10 con 0x55901c3380a0 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55901c66e040 con 0x55901c360f60 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.882+0000 7f68d33c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55901c671e50 con 0x55901c340280 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51820/0 (socket says 172.21.15.29:51820) 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 -- 172.21.15.29:0/1129707748 learned_addr learned my addr 172.21.15.29:0/1129707748 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c706f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c707980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 -- 172.21.15.29:0/1129707748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 msgr2=0x55901c707980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c707980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 -- 172.21.15.29:0/1129707748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 msgr2=0x55901c706f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c706f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 -- 172.21.15.29:0/1129707748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55901c3cc540 con 0x55901c3380a0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c706f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c659c640 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f68b8008c30 tx=0x7f68b800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5688d6d709f4f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68bf7fe640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 0x7f68b40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68bcff9640 1 -- 172.21.15.29:0/1129707748 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8014070 con 0x55901c3380a0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.883+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 0x7f68b40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68bf7fe640 1 -- 172.21.15.29:0/1737873560 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f68b00e7530 con 0x7f68c81083a0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68bcff9640 1 -- 172.21.15.29:0/1129707748 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b80047a0 con 0x55901c3380a0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68bcff9640 1 -- 172.21.15.29:0/1129707748 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8010690 con 0x55901c3380a0 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68d33c1740 1 -- 172.21.15.29:0/1129707748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 msgr2=0x55901c707440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f68b8008c30 tx=0x7f68b800e720 comp rx=0 tx=0).stop 2024-09-21T01:48:04.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.884+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 0x7f68b40b8ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098d0 tx=0x7f68c002f470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- 172.21.15.29:0/1129707748 shutdown_connections 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c707980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c707440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1129707748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c706f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- 172.21.15.29:0/1129707748 >> 172.21.15.29:0/1129707748 conn(0x55901c355350 msgr2=0x55901c704df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- 172.21.15.29:0/1129707748 shutdown_connections 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- 172.21.15.29:0/1129707748 wait complete. 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- start start 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 Processor -- start 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c712f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c71ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55901c4cf5f0 con 0x55901c3380a0 2024-09-21T01:48:04.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55901c609010 con 0x55901c360f60 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.885+0000 7f68d33c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55901c7156e0 con 0x55901c340280 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68d33c1740 10 client.0 _ll_register_callbacks cb 0x55901c668340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51830/0 (socket says 172.21.15.29:51830) 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 -- 172.21.15.29:0/864479769 learned_addr learned my addr 172.21.15.29:0/864479769 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c712f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c71ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 -- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 msgr2=0x55901c712f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c712f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 -- 172.21.15.29:0/864479769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 msgr2=0x55901c71ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c71ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 -- 172.21.15.29:0/864479769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55901c636a00 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c71ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68c659c640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005da0 tx=0x7f68b8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b7107ed45edb5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68adffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8014030 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f68b8004050 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b80047a0 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68b8004e30 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.886+0000 7f68d33c1740 10 client.15318 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55901c721600 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b80047a0 con 0x55901c3380a0 2024-09-21T01:48:04.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68b8003c40 con 0x55901c3380a0 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 10 client.15318 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 20 client.15318 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c31f8e0 0x55901c724e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 4 client.15318 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55901c680b60 con 0x55901c31f8e0 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.887+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c31f8e0 0x55901c724e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.888+0000 7f68adffb640 10 client.15318 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.888+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c31f8e0 0x55901c724e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4306d7ef607b036f server_cookie=b46ed54ddf3358d1 in_seq=0 out_seq=0 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68adffb640 1 -- 172.21.15.29:0/864479769 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55901c680b60 con 0x55901c31f8e0 2024-09-21T01:48:04.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68adffb640 10 client.15318 handle_command_reply: tid=0 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 client.15318 shutdown 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 msgr2=0x55901c712a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005da0 tx=0x7f68b8005230 comp rx=0 tx=0).stop 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 shutdown_connections 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c360f60 0x55901c71ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c31f8e0 0x55901c724e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c712f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 --2- 172.21.15.29:0/864479769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c3380a0 0x55901c712a00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68adffb640 0 client.15318 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 >> 172.21.15.29:0/864479769 conn(0x55901c355350 msgr2=0x55901c70a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 shutdown_connections 2024-09-21T01:48:04.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 1 -- 172.21.15.29:0/864479769 wait complete. 2024-09-21T01:48:04.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.889+0000 7f68d33c1740 20 client.15318 trim_cache size 0 max 16384 2024-09-21T01:48:04.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 Processor -- start 2024-09-21T01:48:04.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 -- start start 2024-09-21T01:48:04.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c71e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c360f60 0x55901c704600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55901c7231c0 con 0x55901c340280 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55901c723890 con 0x55901c3380a0 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68d33c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55901c723f60 con 0x55901c360f60 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51840/0 (socket says 172.21.15.29:51840) 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1831161387 learned_addr learned my addr 172.21.15.29:0/1831161387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c659c640 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c71e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c360f60 0x55901c704600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1831161387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c360f60 msgr2=0x55901c704600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c360f60 0x55901c704600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1831161387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 msgr2=0x55901c71e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c71e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.908+0000 7f68c5d9b640 1 -- 172.21.15.29:0/1831161387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55901c6ff080 con 0x55901c340280 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.909+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004770 tx=0x7f68b00e9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8f046046982782 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.909+0000 7f68affff640 1 -- 172.21.15.29:0/1831161387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00ad080 con 0x55901c340280 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.909+0000 7f68affff640 1 -- 172.21.15.29:0/1831161387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b00adc70 con 0x55901c340280 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.909+0000 7f68affff640 1 -- 172.21.15.29:0/1831161387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00e7110 con 0x55901c340280 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- 172.21.15.29:0/1831161387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 msgr2=0x55901c71eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004770 tx=0x7f68b00e9a20 comp rx=0 tx=0).stop 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- 172.21.15.29:0/1831161387 shutdown_connections 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c360f60 0x55901c704600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c340280 0x55901c71eeb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 --2- 172.21.15.29:0/1831161387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c71e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- 172.21.15.29:0/1831161387 >> 172.21.15.29:0/1831161387 conn(0x55901c355350 msgr2=0x55901c711080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- 172.21.15.29:0/1831161387 shutdown_connections 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- 172.21.15.29:0/1831161387 wait complete. 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 -- start start 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.910+0000 7f68d33c1740 1 Processor -- start 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c727470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c7279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55901c724d00 con 0x55901c360f60 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55901c724630 con 0x55901c3380a0 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55901c7253d0 con 0x55901c340280 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68d33c1740 10 client.0 _ll_register_callbacks cb 0x7f68c00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51848/0 (socket says 172.21.15.29:51848) 2024-09-21T01:48:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 -- 172.21.15.29:0/3322987142 learned_addr learned my addr 172.21.15.29:0/3322987142 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c659c640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c727470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c5d9b640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c7279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 -- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 msgr2=0x55901c7279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c7279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 -- 172.21.15.29:0/3322987142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 msgr2=0x55901c727470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c727470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 -- 172.21.15.29:0/3322987142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55901c72d140 con 0x55901c360f60 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68c6d9d640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f68c0031a00 tx=0x7f68c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882e1131701fa40a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68adffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.911+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0059060 con 0x55901c360f60 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f68c0004310 con 0x55901c360f60 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b00adc70 con 0x55901c360f60 2024-09-21T01:48:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68c0004510 con 0x55901c360f60 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 10 client.15324 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55901c790970 con 0x55901c360f60 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b00adc70 con 0x55901c360f60 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68c00305c0 con 0x55901c360f60 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 10 client.15324 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 20 client.15324 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c2f4c90 0x55901c7942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 4 client.15324 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.912+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "status"}) -- 0x55901c67c610 con 0x55901c2f4c90 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68c659c640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c2f4c90 0x55901c7942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68adffb640 10 client.15324 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68c659c640 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c2f4c90 0x55901c7942d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=391fdd905197bb2a server_cookie=2f39f49aceecb98c in_seq=0 out_seq=0 2024-09-21T01:48:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68adffb640 1 -- 172.21.15.29:0/3322987142 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55901c67c610 con 0x55901c2f4c90 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68adffb640 10 client.15324 handle_command_reply: tid=0 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 client.15324 shutdown 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 msgr2=0x55901c72cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f68c0031a00 tx=0x7f68c00095c0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 shutdown_connections 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55901c360f60 0x55901c72cc00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55901c2f4c90 0x55901c7942d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68adffb640 0 client.15324 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901c340280 0x55901c7279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 --2- 172.21.15.29:0/3322987142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55901c3380a0 0x55901c727470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 >> 172.21.15.29:0/3322987142 conn(0x55901c355350 msgr2=0x55901c704600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 shutdown_connections 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.913+0000 7f68d33c1740 1 -- 172.21.15.29:0/3322987142 wait complete. 2024-09-21T01:48:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68d33c1740 20 client.15324 trim_cache size 0 max 16384 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 3, 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24625, 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 104.728665214, 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:48:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 138.74932692300001 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 msgr2=0x7f68b40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 0x7f68b40b8ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098d0 tx=0x7f68c002f470 comp rx=0 tx=0).stop 2024-09-21T01:48:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 msgr2=0x7f68c8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c970 tx=0x7f68b000ce30 comp rx=0 tx=0).stop 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 shutdown_connections 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f68b40b65a0 0x7f68b40b8ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68c8108cf0 0x7f68c8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68c81083a0 0x7f68c8199540 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 --2- 172.21.15.29:0/1737873560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102a60 0x7f68c8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 >> 172.21.15.29:0/1737873560 conn(0x7f68c8079730 msgr2=0x7f68c8104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 shutdown_connections 2024-09-21T01:48:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:04.914+0000 7f68cce4d640 1 -- 172.21.15.29:0/1737873560 wait complete. 2024-09-21T01:48:05.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:05 smithi190 ceph-mon[34368]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 340 B/s wr, 12 op/s 2024-09-21T01:48:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:05 smithi066 ceph-mon[35001]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 340 B/s wr, 12 op/s 2024-09-21T01:48:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:05 smithi029 ceph-mon[31228]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 340 B/s wr, 12 op/s 2024-09-21T01:48:05.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T01:48:05.991 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:06.411 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:06 smithi029 ceph-mon[31228]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T01:48:06.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:06 smithi190 ceph-mon[34368]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T01:48:06.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 -- 172.21.15.29:0/3950023873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d181083a0 msgr2=0x7f3d181087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/3950023873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d181083a0 0x7f3d181087b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009900 tx=0x7f3d0c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 -- 172.21.15.29:0/3950023873 shutdown_connections 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/3950023873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d18108cf0 0x7f3d1810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/3950023873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d181083a0 0x7f3d181087b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/3950023873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 0x7f3d18102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 -- 172.21.15.29:0/3950023873 >> 172.21.15.29:0/3950023873 conn(0x7f3d18079730 msgr2=0x7f3d18079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 -- 172.21.15.29:0/3950023873 shutdown_connections 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.505+0000 7f3d1f07f640 1 -- 172.21.15.29:0/3950023873 wait complete. 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 Processor -- start 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 -- start start 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 0x7f3d1819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181083a0 0x7f3d1819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18113a80 con 0x7f3d18108cf0 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18113900 con 0x7f3d181083a0 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.506+0000 7f3d1f07f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18113c00 con 0x7f3d18102a60 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51870/0 (socket says 172.21.15.29:51870) 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 -- 172.21.15.29:0/575932057 learned_addr learned my addr 172.21.15.29:0/575932057 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d17fff640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181083a0 0x7f3d1819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 0x7f3d1819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 -- 172.21.15.29:0/575932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 msgr2=0x7f3d1819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 0x7f3d1819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 -- 172.21.15.29:0/575932057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181083a0 msgr2=0x7f3d1819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181083a0 0x7f3d1819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 -- 172.21.15.29:0/575932057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d181a0480 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1d61c640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000eb10 tx=0x7f3d1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d15ffb640 1 -- 172.21.15.29:0/575932057 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10004b20 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d181a0770 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d15ffb640 1 -- 172.21.15.29:0/575932057 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d10004cc0 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.507+0000 7f3d15ffb640 1 -- 172.21.15.29:0/575932057 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10010670 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.508+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d181a6ed0 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.510+0000 7f3d255f3740 1 Processor -- start 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.511+0000 7f3d15ffb640 1 -- 172.21.15.29:0/575932057 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3d10005020 con 0x7f3d18108cf0 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.511+0000 7f3d255f3740 1 -- start start 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cba000f60 0x564cba3a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cba31ec10 con 0x564cb9fe0280 2024-09-21T01:48:06.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cba30e040 con 0x564cb9fd80a0 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d255f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cba311e50 con 0x564cba000f60 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51884/0 (socket says 172.21.15.29:51884) 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d17fff640 1 -- 172.21.15.29:0/2763994768 learned_addr learned my addr 172.21.15.29:0/2763994768 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d17fff640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cba000f60 0x564cba3a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d15ffb640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 0x7f3d000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 -- 172.21.15.29:0/2763994768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cba000f60 msgr2=0x564cba3a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cba000f60 0x564cba3a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 -- 172.21.15.29:0/2763994768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 msgr2=0x564cba3a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1ce1b640 1 -- 172.21.15.29:0/2763994768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cba06c540 con 0x564cb9fd80a0 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d17fff640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.512+0000 7f3d1d61c640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 0x7f3d000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.513+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3a7440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009b70 tx=0x7f3d080117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0766a6cdacd95b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.513+0000 7f3d15ffb640 1 -- 172.21.15.29:0/575932057 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3d100e7930 con 0x7f3d18108cf0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.513+0000 7f3cfeffd640 1 -- 172.21.15.29:0/2763994768 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08009e60 con 0x564cb9fd80a0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.513+0000 7f3cfeffd640 1 -- 172.21.15.29:0/2763994768 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d08012900 con 0x564cb9fd80a0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.513+0000 7f3cfeffd640 1 -- 172.21.15.29:0/2763994768 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800fa80 con 0x564cb9fd80a0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- 172.21.15.29:0/2763994768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 msgr2=0x564cba3a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d1d61c640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 0x7f3d000b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ada00 tx=0x7f3d10079670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3a7440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009b70 tx=0x7f3d080117b0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- 172.21.15.29:0/2763994768 shutdown_connections 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3a7440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2763994768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cba000f60 0x564cba3a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- 172.21.15.29:0/2763994768 >> 172.21.15.29:0/2763994768 conn(0x564cb9ff5350 msgr2=0x564cba3a4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- 172.21.15.29:0/2763994768 shutdown_connections 2024-09-21T01:48:06.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- 172.21.15.29:0/2763994768 wait complete. 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 -- start start 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.514+0000 7f3d255f3740 1 Processor -- start 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3b7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cba16f5f0 con 0x564cba000f60 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cba2a9010 con 0x564cb9fd80a0 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cba3b9980 con 0x564cb9fe0280 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d255f3740 10 client.0 _ll_register_callbacks cb 0x564cba308340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51896/0 (socket says 172.21.15.29:51896) 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 -- 172.21.15.29:0/1951225062 learned_addr learned my addr 172.21.15.29:0/1951225062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 -- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 msgr2=0x564cba3b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 -- 172.21.15.29:0/1951225062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 msgr2=0x564cba3b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.515+0000 7f3d17fff640 1 -- 172.21.15.29:0/1951225062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cba2d6a00 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3d17fff640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c032ef0 tx=0x7f3d0c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea173bd9912bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c047040 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d08012900 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3d0c031d10 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d0c053050 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3d255f3740 10 client.15336 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cba3c5880 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d08012900 con 0x564cba000f60 2024-09-21T01:48:06.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.516+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d0c003c40 con 0x564cba000f60 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 20 client.15336 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cb9fbf8e0 0x564cba3c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 4 client.15336 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cba320b60 con 0x564cb9fbf8e0 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cb9fbf8e0 0x564cba3c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cb9fbf8e0 0x564cba3c9280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=450d00929d0e6f46 server_cookie=a17b0d5025647bf4 in_seq=0 out_seq=0 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.517+0000 7f3cf7fff640 10 client.15336 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3cf7fff640 1 -- 172.21.15.29:0/1951225062 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564cba320b60 con 0x564cb9fbf8e0 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3cf7fff640 10 client.15336 handle_command_reply: tid=0 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 client.15336 shutdown 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 msgr2=0x564cba3c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c032ef0 tx=0x7f3d0c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:06.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 shutdown_connections 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cb9fbf8e0 0x564cba3c9280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cba000f60 0x564cba3c2ce0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 --2- 172.21.15.29:0/1951225062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3cf7fff640 0 client.15336 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.518+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 >> 172.21.15.29:0/1951225062 conn(0x564cb9ff5350 msgr2=0x564cba3a33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.519+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 shutdown_connections 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.519+0000 7f3d255f3740 1 -- 172.21.15.29:0/1951225062 wait complete. 2024-09-21T01:48:06.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.519+0000 7f3d255f3740 20 client.15336 trim_cache size 0 max 16384 2024-09-21T01:48:06.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 Processor -- start 2024-09-21T01:48:06.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 -- start start 2024-09-21T01:48:06.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c75c0 con 0x564cb9fbf8e0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c7c90 con 0x564cb9fd80a0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d255f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c8360 con 0x564cb9fe0280 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.537+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51906/0 (socket says 172.21.15.29:51906) 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 -- 172.21.15.29:0/2440235161 learned_addr learned my addr 172.21.15.29:0/2440235161 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 -- 172.21.15.29:0/2440235161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 msgr2=0x564cba3c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 -- 172.21.15.29:0/2440235161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 msgr2=0x564cba3c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 -- 172.21.15.29:0/2440235161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cba3a9d30 con 0x564cb9fbf8e0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3d17fff640 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001eb0 tx=0x7f3d0c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca12958e6bea8675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3cfe7fc640 1 -- 172.21.15.29:0/2440235161 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c035040 con 0x564cb9fbf8e0 2024-09-21T01:48:06.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3cfe7fc640 1 -- 172.21.15.29:0/2440235161 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c03dd80 con 0x564cb9fbf8e0 2024-09-21T01:48:06.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.538+0000 7f3cfe7fc640 1 -- 172.21.15.29:0/2440235161 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0044b0 con 0x564cb9fbf8e0 2024-09-21T01:48:06.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 -- 172.21.15.29:0/2440235161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 msgr2=0x564cba3c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001eb0 tx=0x7f3d0c005a20 comp rx=0 tx=0).stop 2024-09-21T01:48:06.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 -- 172.21.15.29:0/2440235161 shutdown_connections 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fe0280 0x564cba3c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fd80a0 0x564cba3c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 --2- 172.21.15.29:0/2440235161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fbf8e0 0x564cba3c6cf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 -- 172.21.15.29:0/2440235161 >> 172.21.15.29:0/2440235161 conn(0x564cb9ff5350 msgr2=0x564cba3b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 -- 172.21.15.29:0/2440235161 shutdown_connections 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.539+0000 7f3d255f3740 1 -- 172.21.15.29:0/2440235161 wait complete. 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 -- start start 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 Processor -- start 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fbf8e0 0x564cba3d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fd80a0 0x564cba3cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c9100 con 0x564cb9fe0280 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c8a30 con 0x564cb9fbf8e0 2024-09-21T01:48:06.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cba3c97d0 con 0x564cb9fd80a0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d255f3740 10 client.0 _ll_register_callbacks cb 0x7f3d0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51922/0 (socket says 172.21.15.29:51922) 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d17fff640 1 -- 172.21.15.29:0/4087238330 learned_addr learned my addr 172.21.15.29:0/4087238330 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.540+0000 7f3d17fff640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fbf8e0 0x564cba3d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fd80a0 0x564cba3cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 -- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fd80a0 msgr2=0x564cba3cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fd80a0 0x564cba3cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 -- 172.21.15.29:0/4087238330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fbf8e0 msgr2=0x564cba3d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fbf8e0 0x564cba3d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 -- 172.21.15.29:0/4087238330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cba3d0b20 con 0x564cb9fe0280 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d17fff640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0059d0 tx=0x7f3d0c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9a207e6d2e8015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c050070 con 0x564cb9fe0280 2024-09-21T01:48:06.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3d0c03db70 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c03dd80 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d0c035040 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d255f3740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.541+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cba4348a0 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c03dd80 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d0c063040 con 0x564cb9fe0280 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 10 client.15342 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 20 client.15342 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cba000f60 0x564cba437db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 4 client.15342 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "status"}) -- 0x564cba31c610 con 0x564cba000f60 2024-09-21T01:48:06.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.542+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cba000f60 0x564cba437db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d1ce1b640 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cba000f60 0x564cba437db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=554073c41fe020b3 server_cookie=8f52be2b453d99a2 in_seq=0 out_seq=0 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3cf7fff640 10 client.15342 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3cf7fff640 1 -- 172.21.15.29:0/4087238330 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564cba31c610 con 0x564cba000f60 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3cf7fff640 10 client.15342 handle_command_reply: tid=0 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 client.15342 shutdown 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 msgr2=0x564cba3cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0059d0 tx=0x7f3d0c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 shutdown_connections 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564cba000f60 0x564cba437db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cb9fe0280 0x564cba3cbaa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cb9fd80a0 0x564cba3cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3cf7fff640 0 client.15342 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:48:06.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 --2- 172.21.15.29:0/4087238330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cb9fbf8e0 0x564cba3d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 >> 172.21.15.29:0/4087238330 conn(0x564cb9ff5350 msgr2=0x564cba3b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 shutdown_connections 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 1 -- 172.21.15.29:0/4087238330 wait complete. 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.543+0000 7f3d255f3740 20 client.15342 trim_cache size 0 max 16384 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 2, 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24640, 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:48:06.548 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 107.371570766, 2024-09-21T01:48:06.549 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:48:06.549 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:48:06.549 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:48:06.549 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 135.303125163 2024-09-21T01:48:06.549 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 msgr2=0x7f3d000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 0x7f3d000b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ada00 tx=0x7f3d10079670 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 msgr2=0x7f3d1819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000eb10 tx=0x7f3d1000c2d0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 shutdown_connections 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3d000b6510 0x7f3d000b8a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d18108cf0 0x7f3d1819fd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181083a0 0x7f3d1819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 --2- 172.21.15.29:0/575932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d18102a60 0x7f3d1819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 >> 172.21.15.29:0/575932057 conn(0x7f3d18079730 msgr2=0x7f3d18104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 shutdown_connections 2024-09-21T01:48:06.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:06.544+0000 7f3d1f07f640 1 -- 172.21.15.29:0/575932057 wait complete. 2024-09-21T01:48:06.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:06 smithi066 ceph-mon[35001]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T01:48:07.423 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T01:48:07.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:07 smithi190 ceph-mon[34368]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:07.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:07 smithi066 ceph-mon[35001]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:07.565 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:07 smithi029 ceph-mon[31228]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 -- 172.21.15.29:0/2913691194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 msgr2=0x7f853810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 --2- 172.21.15.29:0/2913691194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853810f520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8530009ca0 tx=0x7f853002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 -- 172.21.15.29:0/2913691194 shutdown_connections 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 --2- 172.21.15.29:0/2913691194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853810f520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 --2- 172.21.15.29:0/2913691194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 0x7f85381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 --2- 172.21.15.29:0/2913691194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 0x7f8538102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 -- 172.21.15.29:0/2913691194 >> 172.21.15.29:0/2913691194 conn(0x7f8538079730 msgr2=0x7f8538079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 -- 172.21.15.29:0/2913691194 shutdown_connections 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.181+0000 7f853fb20640 1 -- 172.21.15.29:0/2913691194 wait complete. 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 Processor -- start 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 -- start start 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 0x7f853819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 0x7f853819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8538113a80 con 0x7f8538108cf0 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8538113900 con 0x7f85381083a0 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853fb20640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8538113c00 con 0x7f8538102a60 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.182+0000 7f853e0bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51950/0 (socket says 172.21.15.29:51950) 2024-09-21T01:48:08.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 -- 172.21.15.29:0/4086535511 learned_addr learned my addr 172.21.15.29:0/4086535511 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853d0bb640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 0x7f853819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853d8bc640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 0x7f853819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 -- 172.21.15.29:0/4086535511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 msgr2=0x7f853819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 0x7f853819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 -- 172.21.15.29:0/4086535511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 msgr2=0x7f853819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 0x7f853819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 -- 172.21.15.29:0/4086535511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381a0320 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853e0bd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8530005cc0 tx=0x7f853002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f852effd640 1 -- 172.21.15.29:0/4086535511 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530003bf0 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f852effd640 1 -- 172.21.15.29:0/4086535511 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8530003d90 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85381a05b0 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f852effd640 1 -- 172.21.15.29:0/4086535511 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853002fbc0 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.183+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85381a0af0 con 0x7f8538108cf0 2024-09-21T01:48:08.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.186+0000 7f8546094740 1 Processor -- start 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f852effd640 1 -- 172.21.15.29:0/4086535511 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f853002fd60 con 0x7f8538108cf0 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 -- start start 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fb8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561109f2fc10 con 0x561109be90a0 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561109f1f040 con 0x561109c11f60 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f8546094740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561109f22e50 con 0x561109bf1280 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f853d8bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f853d8bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51956/0 (socket says 172.21.15.29:51956) 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.187+0000 7f853d8bc640 1 -- 172.21.15.29:0/3606875813 learned_addr learned my addr 172.21.15.29:0/3606875813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d0bb640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f852effd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 0x7f85240b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d0bb640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fb8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 -- 172.21.15.29:0/3606875813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 msgr2=0x561109fb8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fb8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 -- 172.21.15.29:0/3606875813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 msgr2=0x561109fb7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fb7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 -- 172.21.15.29:0/3606875813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561109c7d540 con 0x561109be90a0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d0bb640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fb8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f852effd640 1 -- 172.21.15.29:0/4086535511 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f85301085f0 con 0x7f8538108cf0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853e0bd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 0x7f85240b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f853d8bc640 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f853400cc60 tx=0x7f853400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86888eeac4d60f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f8523fff640 1 -- 172.21.15.29:0/3606875813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534014070 con 0x561109be90a0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f8523fff640 1 -- 172.21.15.29:0/3606875813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85340047a0 con 0x561109be90a0 2024-09-21T01:48:08.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.188+0000 7f8523fff640 1 -- 172.21.15.29:0/3606875813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534010690 con 0x561109be90a0 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 -- 172.21.15.29:0/3606875813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 msgr2=0x561109fb8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f853e0bd640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 0x7f85240b8a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8530009620 tx=0x7f853002f660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f853400cc60 tx=0x7f853400e720 comp rx=0 tx=0).stop 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 -- 172.21.15.29:0/3606875813 shutdown_connections 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fb8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fb8440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 --2- 172.21.15.29:0/3606875813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fb7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 -- 172.21.15.29:0/3606875813 >> 172.21.15.29:0/3606875813 conn(0x561109c06350 msgr2=0x561109fb5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 -- 172.21.15.29:0/3606875813 shutdown_connections 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.189+0000 7f8546094740 1 -- 172.21.15.29:0/3606875813 wait complete. 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 -- start start 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 Processor -- start 2024-09-21T01:48:08.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fc3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fc3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561109d805f0 con 0x561109c11f60 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561109eba010 con 0x561109be90a0 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561109fc66e0 con 0x561109bf1280 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.190+0000 7f8546094740 10 client.0 _ll_register_callbacks cb 0x561109f19340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51958/0 (socket says 172.21.15.29:51958) 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 -- 172.21.15.29:0/618747544 learned_addr learned my addr 172.21.15.29:0/618747544 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fc3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d8bc640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fc3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 -- 172.21.15.29:0/618747544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 msgr2=0x561109fc3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fc3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 -- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 msgr2=0x561109fc3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fc3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 -- 172.21.15.29:0/618747544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561109ee7a00 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f853d0bb640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8528012b90 tx=0x7f8528014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab1700083e457 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f85217fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528010c00 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8528010da0 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85340047a0 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.191+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8528004030 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 10 client.15354 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561109fd25f0 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85340047a0 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8528003c00 con 0x561109c11f60 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:08.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 10 client.15354 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 20 client.15354 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109bd08e0 0x561109fd6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 4 client.15354 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561109f31b60 con 0x561109bd08e0 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.192+0000 7f853d8bc640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109bd08e0 0x561109fd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.193+0000 7f853d8bc640 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109bd08e0 0x561109fd6030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52a7ecd3d2bc4caa server_cookie=d6101ce09f72a675 in_seq=0 out_seq=0 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.193+0000 7f85217fa640 10 client.15354 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f85217fa640 1 -- 172.21.15.29:0/618747544 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561109f31b60 con 0x561109bd08e0 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f85217fa640 10 client.15354 handle_command_reply: tid=0 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 client.15354 shutdown 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 msgr2=0x561109fcf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8528012b90 tx=0x7f8528014040 comp rx=0 tx=0).stop 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 shutdown_connections 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109bd08e0 0x561109fd6030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109c11f60 0x561109fcf9d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fc3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 --2- 172.21.15.29:0/618747544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fc3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f85217fa640 0 client.15354 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 >> 172.21.15.29:0/618747544 conn(0x561109c06350 msgr2=0x561109fbb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 shutdown_connections 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 1 -- 172.21.15.29:0/618747544 wait complete. 2024-09-21T01:48:08.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.194+0000 7f8546094740 20 client.15354 trim_cache size 0 max 16384 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 Processor -- start 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 -- start start 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fcfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fde700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561109fd4370 con 0x561109be90a0 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561109fd4a40 con 0x561109c11f60 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f8546094740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561109fd5110 con 0x561109bf1280 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51966/0 (socket says 172.21.15.29:51966) 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 -- 172.21.15.29:0/458924342 learned_addr learned my addr 172.21.15.29:0/458924342 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 -- 172.21.15.29:0/458924342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 msgr2=0x561109fcfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d8bc640 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fcfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fcfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 -- 172.21.15.29:0/458924342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 msgr2=0x561109fde700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fde700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.213+0000 7f853d0bb640 1 -- 172.21.15.29:0/458924342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561109fc4b00 con 0x561109be90a0 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.214+0000 7f853d0bb640 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f852800bd60 tx=0x7f85280127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485db2ccdd95a570 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.214+0000 7f85237fe640 1 -- 172.21.15.29:0/458924342 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528010bd0 con 0x561109be90a0 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.214+0000 7f85237fe640 1 -- 172.21.15.29:0/458924342 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852800b390 con 0x561109be90a0 2024-09-21T01:48:08.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.214+0000 7f85237fe640 1 -- 172.21.15.29:0/458924342 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280054f0 con 0x561109be90a0 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- 172.21.15.29:0/458924342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 msgr2=0x561109fcf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f852800bd60 tx=0x7f85280127b0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- 172.21.15.29:0/458924342 shutdown_connections 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109c11f60 0x561109fde700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109bf1280 0x561109fcfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 --2- 172.21.15.29:0/458924342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109be90a0 0x561109fcf940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- 172.21.15.29:0/458924342 >> 172.21.15.29:0/458924342 conn(0x561109c06350 msgr2=0x561109fc20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- 172.21.15.29:0/458924342 shutdown_connections 2024-09-21T01:48:08.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- 172.21.15.29:0/458924342 wait complete. 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 -- start start 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.215+0000 7f8546094740 1 Processor -- start 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 0x56110a03e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561109fd5eb0 con 0x561109bf1280 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561109fd57e0 con 0x561109be90a0 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561109fd6580 con 0x561109c11f60 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f8546094740 10 client.0 _ll_register_callbacks cb 0x7f852800b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51978/0 (socket says 172.21.15.29:51978) 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 -- 172.21.15.29:0/1442967537 learned_addr learned my addr 172.21.15.29:0/1442967537 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:08.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d0bb640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d0bb640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 0x56110a03e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 -- 172.21.15.29:0/1442967537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 msgr2=0x56110a03e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 0x56110a03e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 -- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 msgr2=0x561109fd7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fd7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d8bc640 1 -- 172.21.15.29:0/1442967537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110a03ec50 con 0x561109bf1280 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.216+0000 7f853d0bb640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 0x56110a03e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f853d8bc640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f853400f010 tx=0x7f8534005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc6c202e4e4c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f85217fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534014040 con 0x561109bf1280 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852800b390 con 0x561109bf1280 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8534005020 con 0x561109bf1280 2024-09-21T01:48:08.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8534004510 con 0x561109bf1280 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f8546094740 10 client.15360 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56110a045fa0 con 0x561109bf1280 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.217+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852800b390 con 0x561109bf1280 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f853402a030 con 0x561109bf1280 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 10 client.15360 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 20 client.15360 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109ba5c90 0x56110a049520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 4 client.15360 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:48:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "status"}) -- 0x561109f2d610 con 0x561109ba5c90 2024-09-21T01:48:08.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.218+0000 7f853d0bb640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109ba5c90 0x56110a049520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:08.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f853d0bb640 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109ba5c90 0x56110a049520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b199cc970a4d962 server_cookie=20e0724fdb605e80 in_seq=0 out_seq=0 2024-09-21T01:48:08.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f85217fa640 10 client.15360 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:48:08.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f85217fa640 1 -- 172.21.15.29:0/1442967537 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561109f2d610 con 0x561109ba5c90 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f85217fa640 10 client.15360 handle_command_reply: tid=0 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 client.15360 shutdown 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 msgr2=0x561109fd7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f853400f010 tx=0x7f8534005d60 comp rx=0 tx=0).stop 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 shutdown_connections 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561109ba5c90 0x56110a049520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561109c11f60 0x56110a03e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561109bf1280 0x561109fd7920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f85217fa640 0 client.15360 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 --2- 172.21.15.29:0/1442967537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561109be90a0 0x561109fd7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 >> 172.21.15.29:0/1442967537 conn(0x561109c06350 msgr2=0x561109fb5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 shutdown_connections 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 1 -- 172.21.15.29:0/1442967537 wait complete. 2024-09-21T01:48:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.219+0000 7f8546094740 20 client.15360 trim_cache size 0 max 16384 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 1, 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24644, 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 110.05592423, 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:48:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 139.37442824999999 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 msgr2=0x7f85240b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 0x7f85240b8a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8530009620 tx=0x7f853002f660 comp rx=0 tx=0).stop 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 msgr2=0x7f853819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8530005cc0 tx=0x7f853002f380 comp rx=0 tx=0).stop 2024-09-21T01:48:08.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 shutdown_connections 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85240b6540 0x7f85240b8a50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8538108cf0 0x7f853819fcb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85381083a0 0x7f853819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 --2- 172.21.15.29:0/4086535511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8538102a60 0x7f853819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 >> 172.21.15.29:0/4086535511 conn(0x7f8538079730 msgr2=0x7f85381052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 shutdown_connections 2024-09-21T01:48:08.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:08.220+0000 7f853fb20640 1 -- 172.21.15.29:0/4086535511 wait complete. 2024-09-21T01:48:09.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:48:09.270 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:09.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:09 smithi029 ceph-mon[31228]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:09.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:09 smithi190 ceph-mon[34368]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:09 smithi066 ceph-mon[35001]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:09.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.799+0000 7feede032640 1 -- 172.21.15.29:0/3635532051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8103b60 msgr2=0x7feed8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 --2- 172.21.15.29:0/3635532051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8103b60 0x7feed8103f70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec802a410 comp rx=0 tx=0).stop 2024-09-21T01:48:09.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 -- 172.21.15.29:0/3635532051 shutdown_connections 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 --2- 172.21.15.29:0/3635532051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8109df0 0x7feed8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 --2- 172.21.15.29:0/3635532051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 --2- 172.21.15.29:0/3635532051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8103b60 0x7feed8103f70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 -- 172.21.15.29:0/3635532051 >> 172.21.15.29:0/3635532051 conn(0x7feed807b850 msgr2=0x7feed807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 -- 172.21.15.29:0/3635532051 shutdown_connections 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 -- 172.21.15.29:0/3635532051 wait complete. 2024-09-21T01:48:09.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.800+0000 7feede032640 1 Processor -- start 2024-09-21T01:48:09.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 -- start start 2024-09-21T01:48:09.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8103b60 0x7feed819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feed8113ab0 con 0x7feed8109df0 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feed8113930 con 0x7feed81094a0 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feede032640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feed8113c30 con 0x7feed8103b60 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51998/0 (socket says 172.21.15.29:51998) 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 -- 172.21.15.29:0/3128857298 learned_addr learned my addr 172.21.15.29:0/3128857298 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:09.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed77fe640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8103b60 0x7feed819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed6ffd640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 -- 172.21.15.29:0/3128857298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8103b60 msgr2=0x7feed819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8103b60 0x7feed819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.801+0000 7feed7fff640 1 -- 172.21.15.29:0/3128857298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 msgr2=0x7feed819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed7fff640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed7fff640 1 -- 172.21.15.29:0/3128857298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed81a0450 con 0x7feed8109df0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed6ffd640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed7fff640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feed000eaa0 tx=0x7feed000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed4ff9640 1 -- 172.21.15.29:0/3128857298 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000cdb0 con 0x7feed8109df0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed81a0740 con 0x7feed8109df0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed4ff9640 1 -- 172.21.15.29:0/3128857298 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0005020 con 0x7feed8109df0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feed4ff9640 1 -- 172.21.15.29:0/3128857298 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00116d0 con 0x7feed8109df0 2024-09-21T01:48:09.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.802+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81a6ea0 con 0x7feed8109df0 2024-09-21T01:48:09.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 Processor -- start 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feed4ff9640 1 -- 172.21.15.29:0/3128857298 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7feed00051d0 con 0x7feed8109df0 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 -- start start 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741a90440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b741a07c10 con 0x55b7416e9f60 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7419f7040 con 0x55b7416c10a0 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feee45a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b7419fae50 con 0x55b7416c9280 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52014/0 (socket says 172.21.15.29:52014) 2024-09-21T01:48:09.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.805+0000 7feed6ffd640 1 -- 172.21.15.29:0/530261936 learned_addr learned my addr 172.21.15.29:0/530261936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed77fe640 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741a90440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed4ff9640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 0x7feec00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 -- 172.21.15.29:0/530261936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 msgr2=0x55b741a90980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a90980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 -- 172.21.15.29:0/530261936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 msgr2=0x55b741a90440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741a90440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 -- 172.21.15.29:0/530261936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b741755540 con 0x55b7416e9f60 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed7fff640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 0x7feec00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed4ff9640 1 -- 172.21.15.29:0/3128857298 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feed00e7470 con 0x7feed8109df0 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feed6ffd640 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feecc013560 tx=0x7feecc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849b57d9d2df8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feebdffb640 1 -- 172.21.15.29:0/530261936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc019030 con 0x55b7416e9f60 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feebdffb640 1 -- 172.21.15.29:0/530261936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc011970 con 0x55b7416e9f60 2024-09-21T01:48:09.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.806+0000 7feebdffb640 1 -- 172.21.15.29:0/530261936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc01d7e0 con 0x55b7416e9f60 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 -- 172.21.15.29:0/530261936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 msgr2=0x55b741a8ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feed7fff640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 0x7feec00b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feed00ad310 tx=0x7feed00de0b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feecc013560 tx=0x7feecc00b6d0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 -- 172.21.15.29:0/530261936 shutdown_connections 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741a90440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 --2- 172.21.15.29:0/530261936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416e9f60 0x55b741a8ff00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 -- 172.21.15.29:0/530261936 >> 172.21.15.29:0/530261936 conn(0x55b7416de350 msgr2=0x55b741a8ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 -- 172.21.15.29:0/530261936 shutdown_connections 2024-09-21T01:48:09.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.807+0000 7feee45a6740 1 -- 172.21.15.29:0/530261936 wait complete. 2024-09-21T01:48:09.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.808+0000 7feee45a6740 1 -- start start 2024-09-21T01:48:09.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.808+0000 7feee45a6740 1 Processor -- start 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.808+0000 7feee45a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c10a0 0x55b741a9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.808+0000 7feee45a6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.808+0000 7feee45a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aabd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feee45a6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7418585f0 con 0x55b7416c9280 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feee45a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b741992010 con 0x55b7416e9f60 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feee45a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b741aa2a40 con 0x55b7416c10a0 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feee45a6740 10 client.0 _ll_register_callbacks cb 0x55b7419f1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52030/0 (socket says 172.21.15.29:52030) 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 -- 172.21.15.29:0/999460701 learned_addr learned my addr 172.21.15.29:0/999460701 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed6ffd640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c10a0 0x55b741a9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed6ffd640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aabd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 -- 172.21.15.29:0/999460701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c10a0 msgr2=0x55b741a9fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c10a0 0x55b741a9fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 -- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 msgr2=0x55b741aabd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aabd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 -- 172.21.15.29:0/999460701 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7419bfa00 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed6ffd640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aabd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feed77fe640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec80316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145dce4fe3cc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feeb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8035040 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feec8004040 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.809+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc011970 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feec8004240 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b741aae950 con 0x55b7416c9280 2024-09-21T01:48:09.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc011970 con 0x55b7416c9280 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feec8004850 con 0x55b7416c9280 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 20 client.15372 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7416a88e0 0x55b741ab23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 4 client.15372 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.810+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b741a09b60 con 0x55b7416a88e0 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.811+0000 7feed6ffd640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7416a88e0 0x55b741ab23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.811+0000 7feed6ffd640 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7416a88e0 0x55b741ab23f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e7a7be1dcf0ee5 server_cookie=e4698d9c89fdc12b in_seq=0 out_seq=0 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.811+0000 7feeb6ffd640 10 client.15372 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feeb6ffd640 1 -- 172.21.15.29:0/999460701 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b741a09b60 con 0x55b7416a88e0 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feeb6ffd640 10 client.15372 handle_command_reply: tid=0 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 client.15372 shutdown 2024-09-21T01:48:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 msgr2=0x55b741aa0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec80316d0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 shutdown_connections 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7416a88e0 0x55b741ab23f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aabd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741aa0300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 --2- 172.21.15.29:0/999460701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c10a0 0x55b741a9fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 >> 172.21.15.29:0/999460701 conn(0x55b7416de350 msgr2=0x55b741a8c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feeb6ffd640 0 client.15372 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 shutdown_connections 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 1 -- 172.21.15.29:0/999460701 wait complete. 2024-09-21T01:48:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.812+0000 7feee45a6740 20 client.15372 trim_cache size 0 max 16384 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.830+0000 7feee45a6740 1 Processor -- start 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 -- start start 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aa02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab0730 con 0x55b7416c10a0 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab0e00 con 0x55b7416e9f60 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feee45a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab14d0 con 0x55b7416c9280 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52042/0 (socket says 172.21.15.29:52042) 2024-09-21T01:48:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 -- 172.21.15.29:0/1575314126 learned_addr learned my addr 172.21.15.29:0/1575314126 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aa02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 -- 172.21.15.29:0/1575314126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 msgr2=0x55b741a9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 -- 172.21.15.29:0/1575314126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 msgr2=0x55b741aa02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aa02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 -- 172.21.15.29:0/1575314126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b741a8e310 con 0x55b7416c10a0 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.831+0000 7feed6ffd640 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b410 tx=0x7feecc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f058d7b37012fd0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.832+0000 7feebd7fa640 1 -- 172.21.15.29:0/1575314126 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc028040 con 0x55b7416c10a0 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.832+0000 7feebd7fa640 1 -- 172.21.15.29:0/1575314126 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc013230 con 0x55b7416c10a0 2024-09-21T01:48:09.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.832+0000 7feebd7fa640 1 -- 172.21.15.29:0/1575314126 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc019040 con 0x55b7416c10a0 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- 172.21.15.29:0/1575314126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 msgr2=0x55b741a9f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b410 tx=0x7feecc015040 comp rx=0 tx=0).stop 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- 172.21.15.29:0/1575314126 shutdown_connections 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416e9f60 0x55b741aa02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416c9280 0x55b741a9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 --2- 172.21.15.29:0/1575314126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c10a0 0x55b741a9f0b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- 172.21.15.29:0/1575314126 >> 172.21.15.29:0/1575314126 conn(0x55b7416de350 msgr2=0x55b741aafe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- 172.21.15.29:0/1575314126 shutdown_connections 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- 172.21.15.29:0/1575314126 wait complete. 2024-09-21T01:48:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 -- start start 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.833+0000 7feee45a6740 1 Processor -- start 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741aba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 0x55b741ab3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab2270 con 0x55b7416c9280 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab1ba0 con 0x55b7416c10a0 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b741ab2940 con 0x55b7416e9f60 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feee45a6740 10 client.0 _ll_register_callbacks cb 0x7feec8005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52046/0 (socket says 172.21.15.29:52046) 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 -- 172.21.15.29:0/2579096160 learned_addr learned my addr 172.21.15.29:0/2579096160 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed6ffd640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 0x55b741ab3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed6ffd640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741aba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 -- 172.21.15.29:0/2579096160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 msgr2=0x55b741ab3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 0x55b741ab3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 -- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 msgr2=0x55b741aba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741aba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 -- 172.21.15.29:0/2579096160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b741ab4710 con 0x55b7416c9280 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed6ffd640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 0x55b741ab3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feed77fe640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c00 tx=0x7feec8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94ec10a7d8f8b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.834+0000 7feeb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8035070 con 0x55b7416c9280 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feec8003940 con 0x55b7416c9280 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc013230 con 0x55b7416c9280 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feec8003b40 con 0x55b7416c9280 2024-09-21T01:48:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 10 client.15378 fetch_fsmap learned FSMap version 27 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b741aba0c0 con 0x55b7416c9280 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc013230 con 0x55b7416c9280 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feec8031c10 con 0x55b7416c9280 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 10 client.15378 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 20 client.15378 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b74167dc90 0x55b741b20c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 4 client.15378 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.835+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "status"}) -- 0x55b741a05610 con 0x55b74167dc90 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feed6ffd640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b74167dc90 0x55b741b20c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feed6ffd640 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b74167dc90 0x55b741b20c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f719ad29110bd873 server_cookie=7406d3e3086827e0 in_seq=0 out_seq=0 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feeb6ffd640 10 client.15378 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feeb6ffd640 1 -- 172.21.15.29:0/2579096160 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b741a05610 con 0x55b74167dc90 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feeb6ffd640 10 client.15378 handle_command_reply: tid=0 2024-09-21T01:48:09.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.836+0000 7feee45a6740 1 client.15378 shutdown 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 msgr2=0x55b741abafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c00 tx=0x7feec8009230 comp rx=0 tx=0).stop 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 shutdown_connections 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b74167dc90 0x55b741b20c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7416e9f60 0x55b741ab3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7416c9280 0x55b741abafc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 --2- 172.21.15.29:0/2579096160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7416c10a0 0x55b741aba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 >> 172.21.15.29:0/2579096160 conn(0x55b7416de350 msgr2=0x55b741a9fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feeb6ffd640 0 client.15378 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 shutdown_connections 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 1 -- 172.21.15.29:0/2579096160 wait complete. 2024-09-21T01:48:09.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feee45a6740 20 client.15378 trim_cache size 0 max 16384 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "e38bc6b6-77b9-11ef-baf5-efdc52797490", 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 24671, 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 118.988735017, 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 27, 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 76, 2024-09-21T01:48:09.842 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 129.614602346 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 msgr2=0x7feec00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 0x7feec00b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feed00ad310 tx=0x7feed00de0b0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 msgr2=0x7feed819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:09.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feed000eaa0 tx=0x7feed000ef60 comp rx=0 tx=0).stop 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 shutdown_connections 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7feec00b6510 0x7feec00b8a20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed8109df0 0x7feed819fd20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed81094a0 0x7feed819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 --2- 172.21.15.29:0/3128857298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed8103b60 0x7feed819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.837+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 >> 172.21.15.29:0/3128857298 conn(0x7feed807b850 msgr2=0x7feed8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.838+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 shutdown_connections 2024-09-21T01:48:09.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:09.838+0000 7feede032640 1 -- 172.21.15.29:0/3128857298 wait complete. 2024-09-21T01:48:10.630 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T01:48:10.654 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T01:48:10.654 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T01:48:10.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:10.655 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-09-21T01:48:10.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:10.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-09-21T01:48:10.741 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-09-21T01:48:10.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:48:10.743 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-21T01:48:10.743 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T01:48:10.743 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-21T01:48:10.743 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-21T01:48:10.743 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T01:48:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:11.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:11 smithi190 ceph-mon[34368]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:11.502 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:11 smithi029 ceph-mon[31228]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:11 smithi066 ceph-mon[35001]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:11.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 -- 172.21.15.29:0/1069438701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c1083a0 msgr2=0x7f934c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:11.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 --2- 172.21.15.29:0/1069438701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c1083a0 0x7f934c1087b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9340009900 tx=0x7f934002a410 comp rx=0 tx=0).stop 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 -- 172.21.15.29:0/1069438701 shutdown_connections 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 --2- 172.21.15.29:0/1069438701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c108cf0 0x7f934c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 --2- 172.21.15.29:0/1069438701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c1083a0 0x7f934c1087b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 --2- 172.21.15.29:0/1069438701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 0x7f934c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 -- 172.21.15.29:0/1069438701 >> 172.21.15.29:0/1069438701 conn(0x7f934c079730 msgr2=0x7f934c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 -- 172.21.15.29:0/1069438701 shutdown_connections 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.594+0000 7f935144b640 1 -- 172.21.15.29:0/1069438701 wait complete. 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 Processor -- start 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 -- start start 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 0x7f934c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c1083a0 0x7f934c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f934c112960 con 0x7f934c108cf0 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f934c1127e0 con 0x7f934c102a60 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f935144b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f934c112ae0 con 0x7f934c1083a0 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52068/0 (socket says 172.21.15.29:52068) 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f934b7fe640 1 -- 172.21.15.29:0/3609682891 learned_addr learned my addr 172.21.15.29:0/3609682891 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f934affd640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 0x7f934c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.595+0000 7f934a7fc640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c1083a0 0x7f934c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 -- 172.21.15.29:0/3609682891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c1083a0 msgr2=0x7f934c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c1083a0 0x7f934c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 -- 172.21.15.29:0/3609682891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 msgr2=0x7f934c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:11.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 0x7f934c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 -- 172.21.15.29:0/3609682891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c1a0450 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f934b7fe640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f934400eaa0 tx=0x7f934400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934400cdb0 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9344005020 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c1a0740 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440116d0 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.596+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c1a6ea0 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.597+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c102e70 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.599+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f93440051d0 con 0x7f934c108cf0 2024-09-21T01:48:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.599+0000 7f9333fff640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 0x7f93340b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:11.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.600+0000 7f934affd640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 0x7f93340b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:11.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.600+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f93440e7740 con 0x7f934c108cf0 2024-09-21T01:48:11.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.600+0000 7f934affd640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 0x7f93340b8aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f933c00bbe0 tx=0x7f933c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:11.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.603+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9344016030 con 0x7f934c108cf0 2024-09-21T01:48:11.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.834+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f934c19b610 con 0x7f934c108cf0 2024-09-21T01:48:11.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.834+0000 7f9333fff640 1 -- 172.21.15.29:0/3609682891 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f934c19b610 con 0x7f934c108cf0 2024-09-21T01:48:11.835 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T01:48:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 msgr2=0x7f93340b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 0x7f93340b8aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f933c00bbe0 tx=0x7f933c00b390 comp rx=0 tx=0).stop 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 msgr2=0x7f934c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f934400eaa0 tx=0x7f934400ef60 comp rx=0 tx=0).stop 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 shutdown_connections 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93340b6590 0x7f93340b8aa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f934c108cf0 0x7f934c19fd20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f934c1083a0 0x7f934c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 --2- 172.21.15.29:0/3609682891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f934c102a60 0x7f934c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:11.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 >> 172.21.15.29:0/3609682891 conn(0x7f934c079730 msgr2=0x7f934c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:11.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 shutdown_connections 2024-09-21T01:48:11.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:11.837+0000 7f935144b640 1 -- 172.21.15.29:0/3609682891 wait complete. 2024-09-21T01:48:12.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:12 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3609682891' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:48:12.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:12 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3609682891' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:48:12.579 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:12 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3609682891' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T01:48:12.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-21T01:48:12.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:12.609 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:48:12.626 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42102sec preferred_lft 42102sec 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T01:48:12.627 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-09-21T01:48:12.628 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add name ceph-brx type bridge 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr flush dev ceph-brx 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set ceph-brx up 2024-09-21T01:48:12.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T01:48:12.629 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-09-21T01:48:12.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:12.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:12.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:12.842 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T01:48:12.868 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-09-21T01:48:12.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:12.869 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-09-21T01:48:12.942 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-09-21T01:48:12.942 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T01:48:12.942 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T01:48:12.942 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T01:48:12.942 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T01:48:12.942 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:48:12.943 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-09-21T01:48:12.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T01:48:12.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T01:48:12.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T01:48:12.943 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-09-21T01:48:13.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:13.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:13.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:13.275 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-09-21T01:48:13.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:13.291 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-09-21T01:48:13.347 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:48:13.348 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-09-21T01:48:13.348 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-21T01:48:13.348 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-21T01:48:13.348 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-09-21T01:48:13.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:13.446 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:13 smithi029 ceph-mon[31228]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:13.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:13.468 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-21T01:48:13.468 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:48:13.468 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-09-21T01:48:13.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-21T01:48:13.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T01:48:13.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-21T01:48:13.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-21T01:48:13.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-21T01:48:13.469 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-09-21T01:48:13.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:13 smithi190 ceph-mon[34368]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:13.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:13.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:13 smithi066 ceph-mon[35001]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T01:48:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:13.598 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:48:13.598 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-09-21T01:48:13.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-09-21T01:48:13.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T01:48:13.598 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-09-21T01:48:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:13.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:13.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:48:13.679 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /tmp/cephfs 2024-09-21T01:48:13.694 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-21T01:48:13.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:48:13.694 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /tmp/cephfs 2024-09-21T01:48:13.749 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T01:48:13.750 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T01:48:13.750 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c64ca240-4c76-4594-a4c1-13b15f731d9d 2024-09-21T01:48:13.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:13.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c64ca240-4c76-4594-a4c1-13b15f731d9d /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-21T01:48:13.863 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T01:48:13.863 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T01:48:13.863 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-09-21T01:48:14.000 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T01:48:14.000 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T01:48:14.000 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-09-21T01:48:14.001 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.29:3300,172.21.15.66:3300,172.21.15.190:3300" 2024-09-21T01:48:14.001 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.29:3300,172.21.15.66:3300,172.21.15.190:3300:/volumes/qa/sv_1/c64ca240-4c76-4594-a4c1-13b15f731d9d 2024-09-21T01:48:14.001 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=e38bc6b6-77b9-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T01:48:14.003 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T01:48:14.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:48:14.004 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /tmp/cephfs 2024-09-21T01:48:14.031 DEBUG:teuthology.orchestra.run.smithi029:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:48:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:48:14.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:14.129 INFO:teuthology.orchestra.run.smithi029.stdout:client25099 2024-09-21T01:48:14.130 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T01:48:14.130 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:48:14.147 DEBUG:teuthology.orchestra.run.smithi029:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:48:14.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:48:14.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:14.247 INFO:teuthology.orchestra.run.smithi029.stdout:client25099 2024-09-21T01:48:14.248 DEBUG:teuthology.orchestra.run.smithi029:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:48:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:48:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:14.305 INFO:teuthology.orchestra.run.smithi029.stdout:client25099 2024-09-21T01:48:14.306 DEBUG:teuthology.orchestra.run.smithi029:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490.client25099/status') 2024-09-21T01:48:14.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:48:14.365 INFO:teuthology.orchestra.run.smithi029.stdout:instance: client.25099 (3)172.21.15.29:0/2252473151 2024-09-21T01:48:14.365 INFO:teuthology.orchestra.run.smithi029.stdout:blocklisted: false 2024-09-21T01:48:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records in 2024-09-21T01:48:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records out 2024-09-21T01:48:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:71 bytes copied, 2.3988e-05 s, 3.0 MB/s 2024-09-21T01:48:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:48:14.373 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T01:48:14.383 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T01:48:14.384 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:48:14.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T01:48:14.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/490162793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103fe0 msgr2=0x7fa1a810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:14.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/490162793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103fe0 0x7fa1a810a980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009ca0 tx=0x7fa1a002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/490162793 shutdown_connections 2024-09-21T01:48:14.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/490162793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103fe0 0x7fa1a810a980 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/490162793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a8103640 0x7fa1a8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/490162793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a81025c0 0x7fa1a8102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/490162793 >> 172.21.15.29:0/490162793 conn(0x7fa1a80fdf00 msgr2=0x7fa1a81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/490162793 shutdown_connections 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.498+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/490162793 wait complete. 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.499+0000 7fa1ae2c5640 1 Processor -- start 2024-09-21T01:48:14.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.499+0000 7fa1ae2c5640 1 -- start start 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.499+0000 7fa1ae2c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a81025c0 0x7fa1a8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.499+0000 7fa1ae2c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.499+0000 7fa1ae2c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a8103fe0 0x7fa1a819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ae2c5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810e1f0 con 0x7fa1a8103640 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ae2c5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810e070 con 0x7fa1a81025c0 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ae2c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810e370 con 0x7fa1a8103fe0 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44240/0 (socket says 172.21.15.29:44240) 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 -- 172.21.15.29:0/713682858 learned_addr learned my addr 172.21.15.29:0/713682858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:14.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a7fff640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a81025c0 0x7fa1a8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ac862640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a8103fe0 0x7fa1a819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 -- 172.21.15.29:0/713682858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a8103fe0 msgr2=0x7fa1a819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a8103fe0 0x7fa1a819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 -- 172.21.15.29:0/713682858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a81025c0 msgr2=0x7fa1a8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a81025c0 0x7fa1a8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 -- 172.21.15.29:0/713682858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819a820 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a77fe640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00e9f0 tx=0x7fa19c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00cd80 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0108e0 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819ab10 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c018980 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.500+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a1200 con 0x7fa1a8103640 2024-09-21T01:48:14.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.501+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8109770 con 0x7fa1a8103640 2024-09-21T01:48:14.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.503+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa19c010430 con 0x7fa1a8103640 2024-09-21T01:48:14.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.504+0000 7fa1a57fa640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 0x7fa1900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:14.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.504+0000 7fa1a7fff640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 0x7fa1900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:14.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.504+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa19c0e7b40 con 0x7fa1a8103640 2024-09-21T01:48:14.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.504+0000 7fa1a7fff640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 0x7fa1900b8a20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa19800bbe0 tx=0x7fa19800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:14.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.507+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa19c0ae3e0 con 0x7fa1a8103640 2024-09-21T01:48:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:14.735+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fa1a802e640 con 0x7fa1a8103640 2024-09-21T01:48:15.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.222+0000 7fa1a57fa640 1 -- 172.21.15.29:0/713682858 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fa19c0b27c0 con 0x7fa1a8103640 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 msgr2=0x7fa1900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 0x7fa1900b8a20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa19800bbe0 tx=0x7fa19800b390 comp rx=0 tx=0).stop 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 msgr2=0x7fa1a8199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00e9f0 tx=0x7fa19c00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 shutdown_connections 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1900b6510 0x7fa1900b8a20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a8103fe0 0x7fa1a819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1a8103640 0x7fa1a8199450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.224+0000 7fa1ae2c5640 1 --2- 172.21.15.29:0/713682858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a81025c0 0x7fa1a8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.225+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 >> 172.21.15.29:0/713682858 conn(0x7fa1a80fdf00 msgr2=0x7fa1a8106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:15.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.225+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 shutdown_connections 2024-09-21T01:48:15.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.225+0000 7fa1ae2c5640 1 -- 172.21.15.29:0/713682858 wait complete. 2024-09-21T01:48:15.238 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T01:48:15.249 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T01:48:15.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 -- 172.21.15.29:0/3845941333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930102570 msgr2=0x7f19301029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 --2- 172.21.15.29:0/3845941333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930102570 0x7f19301029e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f192c009900 tx=0x7f192c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 -- 172.21.15.29:0/3845941333 shutdown_connections 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 --2- 172.21.15.29:0/3845941333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930103f90 0x7f193010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 --2- 172.21.15.29:0/3845941333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 0x7f1930103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 --2- 172.21.15.29:0/3845941333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930102570 0x7f19301029e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 -- 172.21.15.29:0/3845941333 >> 172.21.15.29:0/3845941333 conn(0x7f19300fdeb0 msgr2=0x7f1930100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 -- 172.21.15.29:0/3845941333 shutdown_connections 2024-09-21T01:48:15.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.378+0000 7f19378b6640 1 -- 172.21.15.29:0/3845941333 wait complete. 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 Processor -- start 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 -- start start 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930102570 0x7f1930199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 0x7f193019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f193010f300 con 0x7f1930103f90 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f193010f180 con 0x7f19301035f0 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f19378b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f193010f480 con 0x7f1930102570 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44262/0 (socket says 172.21.15.29:44262) 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 -- 172.21.15.29:0/3764543200 learned_addr learned my addr 172.21.15.29:0/3764543200 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935652640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930102570 0x7f1930199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1934e51640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 0x7f193019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 -- 172.21.15.29:0/3764543200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930102570 msgr2=0x7f1930199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930102570 0x7f1930199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 -- 172.21.15.29:0/3764543200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 msgr2=0x7f193019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 0x7f193019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.379+0000 7f1935e53640 1 -- 172.21.15.29:0/3764543200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019b960 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f1935e53640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f192800eaa0 tx=0x7f192800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192800cdb0 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019bc50 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1928005020 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280116d0 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.380+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301a2340 con 0x7f1930103f90 2024-09-21T01:48:15.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.381+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930109740 con 0x7f1930103f90 2024-09-21T01:48:15.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.382+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f19280051d0 con 0x7f1930103f90 2024-09-21T01:48:15.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.384+0000 7f19267fc640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.384+0000 7f1935652640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.384+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f19280e77e0 con 0x7f1930103f90 2024-09-21T01:48:15.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.384+0000 7f1935652640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 0x7f191c0b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:15.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.387+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1928016030 con 0x7f1930103f90 2024-09-21T01:48:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:15 smithi190 ceph-mon[34368]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:15.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:15 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T01:48:15.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:15 smithi066 ceph-mon[35001]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:15.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:15 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T01:48:15.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.577+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f193002e810 con 0x7f1930103f90 2024-09-21T01:48:15.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.578+0000 7f19267fc640 1 -- 172.21.15.29:0/3764543200 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7f19280b2460 con 0x7f1930103f90 2024-09-21T01:48:15.579 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:15.579 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 msgr2=0x7f191c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 0x7f191c0b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c031400 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 msgr2=0x7f193019b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f192800eaa0 tx=0x7f192800ef60 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 shutdown_connections 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1930103f90 0x7f193019b140 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19301035f0 0x7f193019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 --2- 172.21.15.29:0/3764543200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1930102570 0x7f1930199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 >> 172.21.15.29:0/3764543200 conn(0x7f19300fdeb0 msgr2=0x7f1930106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:15.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.580+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 shutdown_connections 2024-09-21T01:48:15.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.581+0000 7f19378b6640 1 -- 172.21.15.29:0/3764543200 wait complete. 2024-09-21T01:48:15.591 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:15 smithi029 ceph-mon[31228]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T01:48:15.591 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:15 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T01:48:15.592 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:41:30.465145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T01:46:06.321761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T01:46:08.562000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T01:48:15.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T01:48:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 -- 172.21.15.29:0/2231123602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 msgr2=0x7f4b30102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 --2- 172.21.15.29:0/2231123602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b30102a10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 -- 172.21.15.29:0/2231123602 shutdown_connections 2024-09-21T01:48:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 --2- 172.21.15.29:0/2231123602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103fc0 0x7f4b3010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 --2- 172.21.15.29:0/2231123602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103620 0x7f4b30103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 --2- 172.21.15.29:0/2231123602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b30102a10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 -- 172.21.15.29:0/2231123602 >> 172.21.15.29:0/2231123602 conn(0x7f4b300fdee0 msgr2=0x7f4b30100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 -- 172.21.15.29:0/2231123602 shutdown_connections 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 -- 172.21.15.29:0/2231123602 wait complete. 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.709+0000 7f4b382e5640 1 Processor -- start 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 -- start start 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103620 0x7f4b3019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103fc0 0x7f4b301a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010e1d0 con 0x7f4b301025a0 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010e050 con 0x7f4b30103620 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b382e5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010e350 con 0x7f4b30103fc0 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44280/0 (socket says 172.21.15.29:44280) 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 -- 172.21.15.29:0/4035845730 learned_addr learned my addr 172.21.15.29:0/4035845730 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b35880640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103620 0x7f4b3019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36882640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103fc0 0x7f4b301a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 -- 172.21.15.29:0/4035845730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103fc0 msgr2=0x7f4b301a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103fc0 0x7f4b301a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 -- 172.21.15.29:0/4035845730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103620 msgr2=0x7f4b3019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103620 0x7f4b3019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 -- 172.21.15.29:0/4035845730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b301a1a10 con 0x7f4b301025a0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.710+0000 7f4b36081640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.711+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c03f030 con 0x7f4b301025a0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.711+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2c003cb0 con 0x7f4b301025a0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.711+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b301a1ca0 con 0x7f4b301025a0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.711+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c035070 con 0x7f4b301025a0 2024-09-21T01:48:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.711+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a2190 con 0x7f4b301025a0 2024-09-21T01:48:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.712+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b30109770 con 0x7f4b301025a0 2024-09-21T01:48:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.714+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4b2c042030 con 0x7f4b301025a0 2024-09-21T01:48:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.715+0000 7f4b277fe640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 0x7f4b1c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.715+0000 7f4b35880640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 0x7f4b1c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.715+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4b2c1080f0 con 0x7f4b301025a0 2024-09-21T01:48:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.716+0000 7f4b35880640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 0x7f4b1c0b8aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005d00 tx=0x7f4b20005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:15.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.718+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b2c0ce930 con 0x7f4b301025a0 2024-09-21T01:48:15.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.907+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b3002e890 con 0x7f4b301025a0 2024-09-21T01:48:15.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.908+0000 7f4b277fe640 1 -- 172.21.15.29:0/4035845730 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f4b2c003e60 con 0x7f4b301025a0 2024-09-21T01:48:15.908 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-09-21T01:48:15.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 msgr2=0x7f4b1c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 0x7f4b1c0b8aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005d00 tx=0x7f4b20005c50 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 msgr2=0x7f4b3006c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c004340 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 shutdown_connections 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4b1c0b6590 0x7f4b1c0b8aa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b30103fc0 0x7f4b301a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b30103620 0x7f4b3019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 --2- 172.21.15.29:0/4035845730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b301025a0 0x7f4b3006c150 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 >> 172.21.15.29:0/4035845730 conn(0x7f4b300fdee0 msgr2=0x7f4b30106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 shutdown_connections 2024-09-21T01:48:15.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:15.910+0000 7f4b382e5640 1 -- 172.21.15.29:0/4035845730 wait complete. 2024-09-21T01:48:15.922 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 -- 172.21.15.29:0/2150587818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 msgr2=0x7f463c1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 --2- 172.21.15.29:0/2150587818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c1029f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f462c009900 tx=0x7f462c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 -- 172.21.15.29:0/2150587818 shutdown_connections 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 --2- 172.21.15.29:0/2150587818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f463c103fa0 0x7f463c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 --2- 172.21.15.29:0/2150587818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103600 0x7f463c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 --2- 172.21.15.29:0/2150587818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c1029f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 -- 172.21.15.29:0/2150587818 >> 172.21.15.29:0/2150587818 conn(0x7f463c0fdee0 msgr2=0x7f463c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 -- 172.21.15.29:0/2150587818 shutdown_connections 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.039+0000 7f4642790640 1 -- 172.21.15.29:0/2150587818 wait complete. 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 Processor -- start 2024-09-21T01:48:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 -- start start 2024-09-21T01:48:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f463c103600 0x7f463c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 0x7f463c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e1b0 con 0x7f463c102580 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e030 con 0x7f463c103600 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4642790640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e330 con 0x7f463c103fa0 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44308/0 (socket says 172.21.15.29:44308) 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 -- 172.21.15.29:0/1706226542 learned_addr learned my addr 172.21.15.29:0/1706226542 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4640d2d640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 0x7f463c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 -- 172.21.15.29:0/1706226542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 msgr2=0x7f463c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 0x7f463c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 -- 172.21.15.29:0/1706226542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f463c103600 msgr2=0x7f463c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f463c103600 0x7f463c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f463bfff640 1 -- 172.21.15.29:0/1706226542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c19a840 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.040+0000 7f4640d2d640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 0x7f463c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f463bfff640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c006010 tx=0x7f462c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c03f030 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c003cb0 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c19aad0 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c035070 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.041+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c19ade0 con 0x7f463c102580 2024-09-21T01:48:16.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.042+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c109750 con 0x7f463c102580 2024-09-21T01:48:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.044+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f462c042030 con 0x7f463c102580 2024-09-21T01:48:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.044+0000 7f46397fa640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 0x7f46240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.044+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f462c1080f0 con 0x7f463c102580 2024-09-21T01:48:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.044+0000 7f463b7fe640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 0x7f46240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.045+0000 7f463b7fe640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 0x7f46240b8a20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:16.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.047+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f462c0ce990 con 0x7f463c102580 2024-09-21T01:48:16.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.238+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f463c063fc0 con 0x7f463c102580 2024-09-21T01:48:16.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.238+0000 7f46397fa640 1 -- 172.21.15.29:0/1706226542 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) ==== 104+0+11 (secure 0 0 0) 0x7f462c0d2d70 con 0x7f463c102580 2024-09-21T01:48:16.238 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.240+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 msgr2=0x7f46240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.240+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 0x7f46240b8a20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005850 comp rx=0 tx=0).stop 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.240+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 msgr2=0x7f463c198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.240+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c006010 tx=0x7f462c004340 comp rx=0 tx=0).stop 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 shutdown_connections 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46240b6510 0x7f46240b8a20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c103fa0 0x7f463c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f463c103600 0x7f463c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 --2- 172.21.15.29:0/1706226542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f463c102580 0x7f463c198e00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 >> 172.21.15.29:0/1706226542 conn(0x7f463c0fdee0 msgr2=0x7f463c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:16.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 shutdown_connections 2024-09-21T01:48:16.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.241+0000 7f4642790640 1 -- 172.21.15.29:0/1706226542 wait complete. 2024-09-21T01:48:16.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T01:48:16.478 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T01:48:16.478 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:16 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:16.478 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3764543200' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:16.478 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:16 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4035845730' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:48:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T01:48:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:16 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3764543200' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:16.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:16 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4035845730' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:48:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/294269994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08102580 msgr2=0x7fdf081029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/294269994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08102580 0x7fdf081029f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009900 tx=0x7fdef802a410 comp rx=0 tx=0).stop 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/294269994 shutdown_connections 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/294269994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08103fa0 0x7fdf0810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/294269994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103600 0x7fdf08103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/294269994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08102580 0x7fdf081029f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/294269994 >> 172.21.15.29:0/294269994 conn(0x7fdf080fdee0 msgr2=0x7fdf08100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/294269994 shutdown_connections 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.572+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/294269994 wait complete. 2024-09-21T01:48:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 Processor -- start 2024-09-21T01:48:16.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 -- start start 2024-09-21T01:48:16.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08102580 0x7fdf08106170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 0x7fdf08104f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e1b0 con 0x7fdf08103600 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e030 con 0x7fdf08102580 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf0d5ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e330 con 0x7fdf08103fa0 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44314/0 (socket says 172.21.15.29:44314) 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf067fc640 1 -- 172.21.15.29:0/107859481 learned_addr learned my addr 172.21.15.29:0/107859481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf06ffd640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08102580 0x7fdf08106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.573+0000 7fdf077fe640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 0x7fdf08104f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 -- 172.21.15.29:0/107859481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 msgr2=0x7fdf08104f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 0x7fdf08104f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 -- 172.21.15.29:0/107859481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08102580 msgr2=0x7fdf08106170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08102580 0x7fdf08106170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 -- 172.21.15.29:0/107859481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf081056a0 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf077fe640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 0x7fdf08104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf067fc640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d950 tx=0x7fdefc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc017070 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc009c30 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a3f50 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00f620 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.574+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a4360 con 0x7fdf08103600 2024-09-21T01:48:16.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.575+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf081029f0 con 0x7fdf08103600 2024-09-21T01:48:16.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.577+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdefc010460 con 0x7fdf08103600 2024-09-21T01:48:16.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.577+0000 7fdeeffff640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 0x7fdef00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:16.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.577+0000 7fdf06ffd640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 0x7fdef00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:16.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.578+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdefc0e6e90 con 0x7fdf08103600 2024-09-21T01:48:16.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.578+0000 7fdf06ffd640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 0x7fdef00b8aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdef8000c00 tx=0x7fdef8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:16.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.581+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdefc0ad7b0 con 0x7fdf08103600 2024-09-21T01:48:16.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/713682858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T01:48:16.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:16 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:16.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3764543200' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:16.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:16 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4035845730' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:48:16.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.770+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf081a47f0 con 0x7fdf08103600 2024-09-21T01:48:16.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.770+0000 7fdeeffff640 1 -- 172.21.15.29:0/107859481 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) ==== 100+0+11 (secure 0 0 0) 0x7fdefc009de0 con 0x7fdf08103600 2024-09-21T01:48:16.770 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 msgr2=0x7fdef00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 0x7fdef00b8aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdef8000c00 tx=0x7fdef8031400 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 msgr2=0x7fdf08104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d950 tx=0x7fdefc00de10 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 shutdown_connections 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdef00b6590 0x7fdef00b8aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf08103fa0 0x7fdf08104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf08103600 0x7fdf08104a30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 --2- 172.21.15.29:0/107859481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf08102580 0x7fdf08106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 >> 172.21.15.29:0/107859481 conn(0x7fdf080fdee0 msgr2=0x7fdf080ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 shutdown_connections 2024-09-21T01:48:16.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:16.772+0000 7fdf0d5ba640 1 -- 172.21.15.29:0/107859481 wait complete. 2024-09-21T01:48:16.784 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T01:48:16.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:48:17.089 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:17 smithi029 ceph-mon[31228]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:48:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1706226542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:17 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/107859481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:17 smithi066 ceph-mon[35001]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:48:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1706226542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:17 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/107859481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 -- 172.21.15.29:0/95458262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee4102a60 msgr2=0x7efee4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 --2- 172.21.15.29:0/95458262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee4102a60 0x7efee4102e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efed4009900 tx=0x7efed402a410 comp rx=0 tx=0).stop 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 -- 172.21.15.29:0/95458262 shutdown_connections 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 --2- 172.21.15.29:0/95458262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 0x7efee410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 --2- 172.21.15.29:0/95458262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee41083a0 0x7efee41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 --2- 172.21.15.29:0/95458262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee4102a60 0x7efee4102e70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 -- 172.21.15.29:0/95458262 >> 172.21.15.29:0/95458262 conn(0x7efee4079730 msgr2=0x7efee4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:17.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 -- 172.21.15.29:0/95458262 shutdown_connections 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.583+0000 7efee96ab640 1 -- 172.21.15.29:0/95458262 wait complete. 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.584+0000 7efee96ab640 1 Processor -- start 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.584+0000 7efee96ab640 1 -- start start 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.584+0000 7efee96ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee4102a60 0x7efee4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.584+0000 7efee96ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee96ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 0x7efee419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee96ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112960 con 0x7efee41083a0 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee96ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efee41127e0 con 0x7efee4102a60 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee96ab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112ae0 con 0x7efee4108cf0 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44336/0 (socket says 172.21.15.29:44336) 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 -- 172.21.15.29:0/4277398116 learned_addr learned my addr 172.21.15.29:0/4277398116 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee37fe640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 0x7efee419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:17.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee2ffd640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee4102a60 0x7efee4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 -- 172.21.15.29:0/4277398116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 msgr2=0x7efee419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 0x7efee419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 -- 172.21.15.29:0/4277398116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee4102a60 msgr2=0x7efee4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee4102a60 0x7efee4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 -- 172.21.15.29:0/4277398116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419f2b0 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.585+0000 7efee27fc640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efecc00ea80 tx=0x7efecc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.586+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00ce40 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.586+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efecc011920 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.586+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419f5a0 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.586+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc01ab00 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.586+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee406ae60 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.587+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee4102e70 con 0x7efee41083a0 2024-09-21T01:48:17.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.590+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efecc01aca0 con 0x7efee41083a0 2024-09-21T01:48:17.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.590+0000 7efedbfff640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 0x7efed00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:17.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.590+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7efecc0e79f0 con 0x7efee41083a0 2024-09-21T01:48:17.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.591+0000 7efee2ffd640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 0x7efed00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:17.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.591+0000 7efee2ffd640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 0x7efed00b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efed4031440 tx=0x7efed4009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:17.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.593+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efecc058710 con 0x7efee41083a0 2024-09-21T01:48:17.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:17 smithi190 ceph-mon[34368]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:48:17.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1706226542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:17 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/107859481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:48:17.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.821+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee406b930 con 0x7efee41083a0 2024-09-21T01:48:17.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.821+0000 7efedbfff640 1 -- 172.21.15.29:0/4277398116 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11970 (secure 0 0 0) 0x7efecc016030 con 0x7efee41083a0 2024-09-21T01:48:17.822 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:17.822 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":28,"btime":"2024-09-21T01:48:15:216396+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:48:14.736858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:48:17.822 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 28 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 msgr2=0x7efed00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 0x7efed00b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efed4031440 tx=0x7efed4009560 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 msgr2=0x7efee4199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efecc00ea80 tx=0x7efecc00ef40 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 shutdown_connections 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7efed00b6510 0x7efed00b8a20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee4108cf0 0x7efee419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efee41083a0 0x7efee4199910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 --2- 172.21.15.29:0/4277398116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee4102a60 0x7efee4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.823+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 >> 172.21.15.29:0/4277398116 conn(0x7efee4079730 msgr2=0x7efee41014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.824+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 shutdown_connections 2024-09-21T01:48:17.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:17.824+0000 7efee96ab640 1 -- 172.21.15.29:0/4277398116 wait complete. 2024-09-21T01:48:18.524 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:18 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4277398116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:18.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:18 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4277398116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:18.555 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T01:48:18.555 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T01:48:18.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 -- 172.21.15.29:0/3439184317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4818102c40 msgr2=0x7f4818105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 --2- 172.21.15.29:0/3439184317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4818102c40 0x7f4818105080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4808009900 tx=0x7f480802a410 comp rx=0 tx=0).stop 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 -- 172.21.15.29:0/3439184317 shutdown_connections 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 --2- 172.21.15.29:0/3439184317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181082e0 0x7f481810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 --2- 172.21.15.29:0/3439184317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48181056c0 0x7f4818107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 --2- 172.21.15.29:0/3439184317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4818102c40 0x7f4818105080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.674+0000 7f481cab6640 1 -- 172.21.15.29:0/3439184317 >> 172.21.15.29:0/3439184317 conn(0x7f48180fd690 msgr2=0x7f48180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:18.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.675+0000 7f481cab6640 1 -- 172.21.15.29:0/3439184317 shutdown_connections 2024-09-21T01:48:18.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.675+0000 7f481cab6640 1 -- 172.21.15.29:0/3439184317 wait complete. 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.675+0000 7f481cab6640 1 Processor -- start 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.675+0000 7f481cab6640 1 -- start start 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4818102c40 0x7f4818102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181056c0 0x7f481819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f481810e1d0 con 0x7f48181082e0 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f481810e050 con 0x7f48181056c0 2024-09-21T01:48:18.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481cab6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f481810e350 con 0x7f4818102c40 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44346/0 (socket says 172.21.15.29:44346) 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 -- 172.21.15.29:0/3582162338 learned_addr learned my addr 172.21.15.29:0/3582162338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f480ffff640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181056c0 0x7f481819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f481659c640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4818102c40 0x7f4818102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 -- 172.21.15.29:0/3582162338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4818102c40 msgr2=0x7f4818102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4818102c40 0x7f4818102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 -- 172.21.15.29:0/3582162338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181056c0 msgr2=0x7f481819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181056c0 0x7f481819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 -- 172.21.15.29:0/3582162338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48181a1950 con 0x7f48181082e0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.676+0000 7f4816d9d640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f481000cce0 tx=0x7f481000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.677+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481000f450 con 0x7f48181082e0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.677+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481000faa0 con 0x7f48181082e0 2024-09-21T01:48:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.677+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48181a1c40 con 0x7f48181082e0 2024-09-21T01:48:18.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.677+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810018b70 con 0x7f48181082e0 2024-09-21T01:48:18.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.677+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181a2140 con 0x7f48181082e0 2024-09-21T01:48:18.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.678+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48181a2310 con 0x7f48181082e0 2024-09-21T01:48:18.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.680+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4810018d10 con 0x7f48181082e0 2024-09-21T01:48:18.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.681+0000 7f480f7fe640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 0x7f48040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:18.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.681+0000 7f481659c640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 0x7f48040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:18.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.681+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f48100e77b0 con 0x7f48181082e0 2024-09-21T01:48:18.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.681+0000 7f481659c640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 0x7f48040b8a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4808031440 tx=0x7f48080314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:18.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.684+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4810014030 con 0x7f48181082e0 2024-09-21T01:48:18.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:18 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4277398116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.871+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48181a24e0 con 0x7f48040b6510 2024-09-21T01:48:18.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.876+0000 7f480f7fe640 1 -- 172.21.15.29:0/3582162338 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258872 (secure 0 0 0) 0x7f48181a24e0 con 0x7f48040b6510 2024-09-21T01:48:18.878 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 msgr2=0x7f48040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 0x7f48040b8a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4808031440 tx=0x7f48080314a0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 msgr2=0x7f48181a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f481000cce0 tx=0x7f481000a620 comp rx=0 tx=0).stop 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 shutdown_connections 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48040b6510 0x7f48040b8a20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48181082e0 0x7f48181a11c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48181056c0 0x7f481819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 --2- 172.21.15.29:0/3582162338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4818102c40 0x7f4818102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:18.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 >> 172.21.15.29:0/3582162338 conn(0x7f48180fd690 msgr2=0x7f48181091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:18.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 shutdown_connections 2024-09-21T01:48:18.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:18.879+0000 7f481cab6640 1 -- 172.21.15.29:0/3582162338 wait complete. 2024-09-21T01:48:18.891 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-21T01:48:18.140033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661707,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1240,"num_read_kb":721,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402728,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539096,"statfs":{"total":1151940427776,"available":1151528034304,"internally_reserved":0,"allocated":12165120,"data_stored":6757081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003147"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500723+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:48:02.500723+0000","last_peered":"2024-09-21T01:48:02.500723+0000","last_clean":"2024-09-21T01:48:02.500723+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:48:02.500723+0000","last_undegraded":"2024-09-21T01:48:02.500723+0000","last_fullsized":"2024-09-21T01:48:02.500723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481380+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:48:02.481380+0000","last_peered":"2024-09-21T01:48:02.481380+0000","last_clean":"2024-09-21T01:48:02.481380+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:48:02.481380+0000","last_undegraded":"2024-09-21T01:48:02.481380+0000","last_fullsized":"2024-09-21T01:48:02.481380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500426+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:48:02.500426+0000","last_peered":"2024-09-21T01:48:02.500426+0000","last_clean":"2024-09-21T01:48:02.500426+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:48:02.500426+0000","last_undegraded":"2024-09-21T01:48:02.500426+0000","last_fullsized":"2024-09-21T01:48:02.500426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473094+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:48:02.473094+0000","last_peered":"2024-09-21T01:48:02.473094+0000","last_clean":"2024-09-21T01:48:02.473094+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:48:02.473094+0000","last_undegraded":"2024-09-21T01:48:02.473094+0000","last_fullsized":"2024-09-21T01:48:02.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469231+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:48:02.469231+0000","last_peered":"2024-09-21T01:48:02.469231+0000","last_clean":"2024-09-21T01:48:02.469231+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:48:02.469231+0000","last_undegraded":"2024-09-21T01:48:02.469231+0000","last_fullsized":"2024-09-21T01:48:02.469231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491842+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:48:02.491842+0000","last_peered":"2024-09-21T01:48:02.491842+0000","last_clean":"2024-09-21T01:48:02.491842+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:48:02.491842+0000","last_undegraded":"2024-09-21T01:48:02.491842+0000","last_fullsized":"2024-09-21T01:48:02.491842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495253+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:48:02.495253+0000","last_peered":"2024-09-21T01:48:02.495253+0000","last_clean":"2024-09-21T01:48:02.495253+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:48:02.495253+0000","last_undegraded":"2024-09-21T01:48:02.495253+0000","last_fullsized":"2024-09-21T01:48:02.495253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472978+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:48:02.472978+0000","last_peered":"2024-09-21T01:48:02.472978+0000","last_clean":"2024-09-21T01:48:02.472978+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:48:02.472978+0000","last_undegraded":"2024-09-21T01:48:02.472978+0000","last_fullsized":"2024-09-21T01:48:02.472978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500825+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:48:02.500825+0000","last_peered":"2024-09-21T01:48:02.500825+0000","last_clean":"2024-09-21T01:48:02.500825+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:48:02.500825+0000","last_undegraded":"2024-09-21T01:48:02.500825+0000","last_fullsized":"2024-09-21T01:48:02.500825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491994+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:48:02.491994+0000","last_peered":"2024-09-21T01:48:02.491994+0000","last_clean":"2024-09-21T01:48:02.491994+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:48:02.491994+0000","last_undegraded":"2024-09-21T01:48:02.491994+0000","last_fullsized":"2024-09-21T01:48:02.491994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498101+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:48:02.498101+0000","last_peered":"2024-09-21T01:48:02.498101+0000","last_clean":"2024-09-21T01:48:02.498101+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:48:02.498101+0000","last_undegraded":"2024-09-21T01:48:02.498101+0000","last_fullsized":"2024-09-21T01:48:02.498101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485929+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:48:02.485929+0000","last_peered":"2024-09-21T01:48:02.485929+0000","last_clean":"2024-09-21T01:48:02.485929+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:48:02.485929+0000","last_undegraded":"2024-09-21T01:48:02.485929+0000","last_fullsized":"2024-09-21T01:48:02.485929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472853+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:48:02.472853+0000","last_peered":"2024-09-21T01:48:02.472853+0000","last_clean":"2024-09-21T01:48:02.472853+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:48:02.472853+0000","last_undegraded":"2024-09-21T01:48:02.472853+0000","last_fullsized":"2024-09-21T01:48:02.472853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481186+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:48:02.481186+0000","last_peered":"2024-09-21T01:48:02.481186+0000","last_clean":"2024-09-21T01:48:02.481186+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:48:02.481186+0000","last_undegraded":"2024-09-21T01:48:02.481186+0000","last_fullsized":"2024-09-21T01:48:02.481186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481111+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:48:02.481111+0000","last_peered":"2024-09-21T01:48:02.481111+0000","last_clean":"2024-09-21T01:48:02.481111+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:48:02.481111+0000","last_undegraded":"2024-09-21T01:48:02.481111+0000","last_fullsized":"2024-09-21T01:48:02.481111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500912+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:48:02.500912+0000","last_peered":"2024-09-21T01:48:02.500912+0000","last_clean":"2024-09-21T01:48:02.500912+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:48:02.500912+0000","last_undegraded":"2024-09-21T01:48:02.500912+0000","last_fullsized":"2024-09-21T01:48:02.500912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500720+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:48:02.500720+0000","last_peered":"2024-09-21T01:48:02.500720+0000","last_clean":"2024-09-21T01:48:02.500720+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:48:02.500720+0000","last_undegraded":"2024-09-21T01:48:02.500720+0000","last_fullsized":"2024-09-21T01:48:02.500720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.476661+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:48:02.476661+0000","last_peered":"2024-09-21T01:48:02.476661+0000","last_clean":"2024-09-21T01:48:02.476661+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:48:02.476661+0000","last_undegraded":"2024-09-21T01:48:02.476661+0000","last_fullsized":"2024-09-21T01:48:02.476661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478158+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:48:02.478158+0000","last_peered":"2024-09-21T01:48:02.478158+0000","last_clean":"2024-09-21T01:48:02.478158+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:48:02.478158+0000","last_undegraded":"2024-09-21T01:48:02.478158+0000","last_fullsized":"2024-09-21T01:48:02.478158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485136+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:48:02.485136+0000","last_peered":"2024-09-21T01:48:02.485136+0000","last_clean":"2024-09-21T01:48:02.485136+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:48:02.485136+0000","last_undegraded":"2024-09-21T01:48:02.485136+0000","last_fullsized":"2024-09-21T01:48:02.485136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.475250+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:48:02.475250+0000","last_peered":"2024-09-21T01:48:02.475250+0000","last_clean":"2024-09-21T01:48:02.475250+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:48:02.475250+0000","last_undegraded":"2024-09-21T01:48:02.475250+0000","last_fullsized":"2024-09-21T01:48:02.475250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.501498+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:48:02.501498+0000","last_peered":"2024-09-21T01:48:02.501498+0000","last_clean":"2024-09-21T01:48:02.501498+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:48:02.501498+0000","last_undegraded":"2024-09-21T01:48:02.501498+0000","last_fullsized":"2024-09-21T01:48:02.501498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484582+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:48:02.484582+0000","last_peered":"2024-09-21T01:48:02.484582+0000","last_clean":"2024-09-21T01:48:02.484582+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:48:02.484582+0000","last_undegraded":"2024-09-21T01:48:02.484582+0000","last_fullsized":"2024-09-21T01:48:02.484582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473115+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:48:02.473115+0000","last_peered":"2024-09-21T01:48:02.473115+0000","last_clean":"2024-09-21T01:48:02.473115+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:48:02.473115+0000","last_undegraded":"2024-09-21T01:48:02.473115+0000","last_fullsized":"2024-09-21T01:48:02.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483131+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:48:02.483131+0000","last_peered":"2024-09-21T01:48:02.483131+0000","last_clean":"2024-09-21T01:48:02.483131+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:48:02.483131+0000","last_undegraded":"2024-09-21T01:48:02.483131+0000","last_fullsized":"2024-09-21T01:48:02.483131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477192+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:48:02.477192+0000","last_peered":"2024-09-21T01:48:02.477192+0000","last_clean":"2024-09-21T01:48:02.477192+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:48:02.477192+0000","last_undegraded":"2024-09-21T01:48:02.477192+0000","last_fullsized":"2024-09-21T01:48:02.477192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478134+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:48:02.478134+0000","last_peered":"2024-09-21T01:48:02.478134+0000","last_clean":"2024-09-21T01:48:02.478134+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:48:02.478134+0000","last_undegraded":"2024-09-21T01:48:02.478134+0000","last_fullsized":"2024-09-21T01:48:02.478134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486640+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:48:02.486640+0000","last_peered":"2024-09-21T01:48:02.486640+0000","last_clean":"2024-09-21T01:48:02.486640+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:48:02.486640+0000","last_undegraded":"2024-09-21T01:48:02.486640+0000","last_fullsized":"2024-09-21T01:48:02.486640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481987+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:48:02.481987+0000","last_peered":"2024-09-21T01:48:02.481987+0000","last_clean":"2024-09-21T01:48:02.481987+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:48:02.481987+0000","last_undegraded":"2024-09-21T01:48:02.481987+0000","last_fullsized":"2024-09-21T01:48:02.481987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.480441+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:48:02.480441+0000","last_peered":"2024-09-21T01:48:02.480441+0000","last_clean":"2024-09-21T01:48:02.480441+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:48:02.480441+0000","last_undegraded":"2024-09-21T01:48:02.480441+0000","last_fullsized":"2024-09-21T01:48:02.480441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483409+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:48:02.483409+0000","last_peered":"2024-09-21T01:48:02.483409+0000","last_clean":"2024-09-21T01:48:02.483409+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:48:02.483409+0000","last_undegraded":"2024-09-21T01:48:02.483409+0000","last_fullsized":"2024-09-21T01:48:02.483409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484143+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:48:02.484143+0000","last_peered":"2024-09-21T01:48:02.484143+0000","last_clean":"2024-09-21T01:48:02.484143+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:48:02.484143+0000","last_undegraded":"2024-09-21T01:48:02.484143+0000","last_fullsized":"2024-09-21T01:48:02.484143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.468518+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:48:02.468518+0000","last_peered":"2024-09-21T01:48:02.468518+0000","last_clean":"2024-09-21T01:48:02.468518+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:48:02.468518+0000","last_undegraded":"2024-09-21T01:48:02.468518+0000","last_fullsized":"2024-09-21T01:48:02.468518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:11.380028+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:48:11.380028+0000","last_peered":"2024-09-21T01:48:11.380028+0000","last_clean":"2024-09-21T01:48:11.380028+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:48:11.380028+0000","last_undegraded":"2024-09-21T01:48:11.380028+0000","last_fullsized":"2024-09-21T01:48:11.380028+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491593+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:48:02.491593+0000","last_peered":"2024-09-21T01:48:02.491593+0000","last_clean":"2024-09-21T01:48:02.491593+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:48:02.491593+0000","last_undegraded":"2024-09-21T01:48:02.491593+0000","last_fullsized":"2024-09-21T01:48:02.491593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486175+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:48:02.486175+0000","last_peered":"2024-09-21T01:48:02.486175+0000","last_clean":"2024-09-21T01:48:02.486175+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:48:02.486175+0000","last_undegraded":"2024-09-21T01:48:02.486175+0000","last_fullsized":"2024-09-21T01:48:02.486175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486993+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:48:02.486993+0000","last_peered":"2024-09-21T01:48:02.486993+0000","last_clean":"2024-09-21T01:48:02.486993+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:48:02.486993+0000","last_undegraded":"2024-09-21T01:48:02.486993+0000","last_fullsized":"2024-09-21T01:48:02.486993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":251,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381826+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:48:13.381826+0000","last_peered":"2024-09-21T01:48:13.381826+0000","last_clean":"2024-09-21T01:48:13.381826+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:48:13.381826+0000","last_undegraded":"2024-09-21T01:48:13.381826+0000","last_fullsized":"2024-09-21T01:48:13.381826+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.496142+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:48:02.496142+0000","last_peered":"2024-09-21T01:48:02.496142+0000","last_clean":"2024-09-21T01:48:02.496142+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:48:02.496142+0000","last_undegraded":"2024-09-21T01:48:02.496142+0000","last_fullsized":"2024-09-21T01:48:02.496142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483673+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:48:02.483673+0000","last_peered":"2024-09-21T01:48:02.483673+0000","last_clean":"2024-09-21T01:48:02.483673+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:48:02.483673+0000","last_undegraded":"2024-09-21T01:48:02.483673+0000","last_fullsized":"2024-09-21T01:48:02.483673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500539+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:48:02.500539+0000","last_peered":"2024-09-21T01:48:02.500539+0000","last_clean":"2024-09-21T01:48:02.500539+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:48:02.500539+0000","last_undegraded":"2024-09-21T01:48:02.500539+0000","last_fullsized":"2024-09-21T01:48:02.500539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":290,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381821+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:48:13.381821+0000","last_peered":"2024-09-21T01:48:13.381821+0000","last_clean":"2024-09-21T01:48:13.381821+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:48:13.381821+0000","last_undegraded":"2024-09-21T01:48:13.381821+0000","last_fullsized":"2024-09-21T01:48:13.381821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":179,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499321+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:48:02.499321+0000","last_peered":"2024-09-21T01:48:02.499321+0000","last_clean":"2024-09-21T01:48:02.499321+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:48:02.499321+0000","last_undegraded":"2024-09-21T01:48:02.499321+0000","last_fullsized":"2024-09-21T01:48:02.499321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486655+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:48:02.486655+0000","last_peered":"2024-09-21T01:48:02.486655+0000","last_clean":"2024-09-21T01:48:02.486655+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:48:02.486655+0000","last_undegraded":"2024-09-21T01:48:02.486655+0000","last_fullsized":"2024-09-21T01:48:02.486655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497852+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:48:02.497852+0000","last_peered":"2024-09-21T01:48:02.497852+0000","last_clean":"2024-09-21T01:48:02.497852+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:48:02.497852+0000","last_undegraded":"2024-09-21T01:48:02.497852+0000","last_fullsized":"2024-09-21T01:48:02.497852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":145,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:15.385340+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:48:15.385340+0000","last_peered":"2024-09-21T01:48:15.385340+0000","last_clean":"2024-09-21T01:48:15.385340+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:48:15.385340+0000","last_undegraded":"2024-09-21T01:48:15.385340+0000","last_fullsized":"2024-09-21T01:48:15.385340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499554+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:48:02.499554+0000","last_peered":"2024-09-21T01:48:02.499554+0000","last_clean":"2024-09-21T01:48:02.499554+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:48:02.499554+0000","last_undegraded":"2024-09-21T01:48:02.499554+0000","last_fullsized":"2024-09-21T01:48:02.499554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498081+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:48:02.498081+0000","last_peered":"2024-09-21T01:48:02.498081+0000","last_clean":"2024-09-21T01:48:02.498081+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:48:02.498081+0000","last_undegraded":"2024-09-21T01:48:02.498081+0000","last_fullsized":"2024-09-21T01:48:02.498081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477597+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:48:02.477597+0000","last_peered":"2024-09-21T01:48:02.477597+0000","last_clean":"2024-09-21T01:48:02.477597+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:48:02.477597+0000","last_undegraded":"2024-09-21T01:48:02.477597+0000","last_fullsized":"2024-09-21T01:48:02.477597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.494315+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:48:02.494315+0000","last_peered":"2024-09-21T01:48:02.494315+0000","last_clean":"2024-09-21T01:48:02.494315+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:48:02.494315+0000","last_undegraded":"2024-09-21T01:48:02.494315+0000","last_fullsized":"2024-09-21T01:48:02.494315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495872+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:48:02.495872+0000","last_peered":"2024-09-21T01:48:02.495872+0000","last_clean":"2024-09-21T01:48:02.495872+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:48:02.495872+0000","last_undegraded":"2024-09-21T01:48:02.495872+0000","last_fullsized":"2024-09-21T01:48:02.495872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.384064+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:48:14.384064+0000","last_peered":"2024-09-21T01:48:14.384064+0000","last_clean":"2024-09-21T01:48:14.384064+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:48:14.384064+0000","last_undegraded":"2024-09-21T01:48:14.384064+0000","last_fullsized":"2024-09-21T01:48:14.384064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498216+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:48:02.498216+0000","last_peered":"2024-09-21T01:48:02.498216+0000","last_clean":"2024-09-21T01:48:02.498216+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:48:02.498216+0000","last_undegraded":"2024-09-21T01:48:02.498216+0000","last_fullsized":"2024-09-21T01:48:02.498216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231673+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231673+0000","last_peered":"2024-09-21T01:46:33.231673+0000","last_clean":"2024-09-21T01:46:33.231673+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231673+0000","last_undegraded":"2024-09-21T01:46:33.231673+0000","last_fullsized":"2024-09-21T01:46:33.231673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237139+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237139+0000","last_peered":"2024-09-21T01:46:33.237139+0000","last_clean":"2024-09-21T01:46:33.237139+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237139+0000","last_undegraded":"2024-09-21T01:46:33.237139+0000","last_fullsized":"2024-09-21T01:46:33.237139+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.502137+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:48:02.502137+0000","last_peered":"2024-09-21T01:48:02.502137+0000","last_clean":"2024-09-21T01:48:02.502137+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:48:02.502137+0000","last_undegraded":"2024-09-21T01:48:02.502137+0000","last_fullsized":"2024-09-21T01:48:02.502137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231523+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231523+0000","last_peered":"2024-09-21T01:46:33.231523+0000","last_clean":"2024-09-21T01:46:33.231523+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231523+0000","last_undegraded":"2024-09-21T01:46:33.231523+0000","last_fullsized":"2024-09-21T01:46:33.231523+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469566+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:48:02.469566+0000","last_peered":"2024-09-21T01:48:02.469566+0000","last_clean":"2024-09-21T01:48:02.469566+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:48:02.469566+0000","last_undegraded":"2024-09-21T01:48:02.469566+0000","last_fullsized":"2024-09-21T01:48:02.469566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:12.380241+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:48:12.380241+0000","last_peered":"2024-09-21T01:48:12.380241+0000","last_clean":"2024-09-21T01:48:12.380241+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:48:12.380241+0000","last_undegraded":"2024-09-21T01:48:12.380241+0000","last_fullsized":"2024-09-21T01:48:12.380241+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.474910+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:48:02.474910+0000","last_peered":"2024-09-21T01:48:02.474910+0000","last_clean":"2024-09-21T01:48:02.474910+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:48:02.474910+0000","last_undegraded":"2024-09-21T01:48:02.474910+0000","last_fullsized":"2024-09-21T01:48:02.474910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"76'7","reported_seq":147,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.383550+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:48:14.383550+0000","last_peered":"2024-09-21T01:48:14.383550+0000","last_clean":"2024-09-21T01:48:14.383550+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:48:14.383550+0000","last_undegraded":"2024-09-21T01:48:14.383550+0000","last_fullsized":"2024-09-21T01:48:14.383550+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.490525+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:48:02.490525+0000","last_peered":"2024-09-21T01:48:02.490525+0000","last_clean":"2024-09-21T01:48:02.490525+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:48:02.490525+0000","last_undegraded":"2024-09-21T01:48:02.490525+0000","last_fullsized":"2024-09-21T01:48:02.490525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225768+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225768+0000","last_peered":"2024-09-21T01:46:33.225768+0000","last_clean":"2024-09-21T01:46:33.225768+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225768+0000","last_undegraded":"2024-09-21T01:46:33.225768+0000","last_fullsized":"2024-09-21T01:46:33.225768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.487242+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:48:02.487242+0000","last_peered":"2024-09-21T01:48:02.487242+0000","last_clean":"2024-09-21T01:48:02.487242+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:48:02.487242+0000","last_undegraded":"2024-09-21T01:48:02.487242+0000","last_fullsized":"2024-09-21T01:48:02.487242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230587+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230587+0000","last_peered":"2024-09-21T01:46:33.230587+0000","last_clean":"2024-09-21T01:46:33.230587+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230587+0000","last_undegraded":"2024-09-21T01:46:33.230587+0000","last_fullsized":"2024-09-21T01:46:33.230587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469299+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:48:02.469299+0000","last_peered":"2024-09-21T01:48:02.469299+0000","last_clean":"2024-09-21T01:48:02.469299+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:48:02.469299+0000","last_undegraded":"2024-09-21T01:48:02.469299+0000","last_fullsized":"2024-09-21T01:48:02.469299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'15","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:41.858878+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:41.858878+0000","last_peered":"2024-09-21T01:46:41.858878+0000","last_clean":"2024-09-21T01:46:41.858878+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:41.858878+0000","last_undegraded":"2024-09-21T01:46:41.858878+0000","last_fullsized":"2024-09-21T01:46:41.858878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484327+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:48:02.484327+0000","last_peered":"2024-09-21T01:48:02.484327+0000","last_clean":"2024-09-21T01:48:02.484327+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:48:02.484327+0000","last_undegraded":"2024-09-21T01:48:02.484327+0000","last_fullsized":"2024-09-21T01:48:02.484327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237630+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237630+0000","last_peered":"2024-09-21T01:46:33.237630+0000","last_clean":"2024-09-21T01:46:33.237630+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237630+0000","last_undegraded":"2024-09-21T01:46:33.237630+0000","last_fullsized":"2024-09-21T01:46:33.237630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485439+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:48:02.485439+0000","last_peered":"2024-09-21T01:48:02.485439+0000","last_clean":"2024-09-21T01:48:02.485439+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:48:02.485439+0000","last_undegraded":"2024-09-21T01:48:02.485439+0000","last_fullsized":"2024-09-21T01:48:02.485439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.441006+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:48:02.441006+0000","last_peered":"2024-09-21T01:48:02.441006+0000","last_clean":"2024-09-21T01:48:02.441006+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:48:02.441006+0000","last_undegraded":"2024-09-21T01:48:02.441006+0000","last_fullsized":"2024-09-21T01:48:02.441006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483903+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:48:02.483903+0000","last_peered":"2024-09-21T01:48:02.483903+0000","last_clean":"2024-09-21T01:48:02.483903+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:48:02.483903+0000","last_undegraded":"2024-09-21T01:48:02.483903+0000","last_fullsized":"2024-09-21T01:48:02.483903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486618+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:48:02.486618+0000","last_peered":"2024-09-21T01:48:02.486618+0000","last_clean":"2024-09-21T01:48:02.486618+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:48:02.486618+0000","last_undegraded":"2024-09-21T01:48:02.486618+0000","last_fullsized":"2024-09-21T01:48:02.486618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500620+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:48:02.500620+0000","last_peered":"2024-09-21T01:48:02.500620+0000","last_clean":"2024-09-21T01:48:02.500620+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:48:02.500620+0000","last_undegraded":"2024-09-21T01:48:02.500620+0000","last_fullsized":"2024-09-21T01:48:02.500620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499041+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:48:02.499041+0000","last_peered":"2024-09-21T01:48:02.499041+0000","last_clean":"2024-09-21T01:48:02.499041+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:48:02.499041+0000","last_undegraded":"2024-09-21T01:48:02.499041+0000","last_fullsized":"2024-09-21T01:48:02.499041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497926+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:48:02.497926+0000","last_peered":"2024-09-21T01:48:02.497926+0000","last_clean":"2024-09-21T01:48:02.497926+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:48:02.497926+0000","last_undegraded":"2024-09-21T01:48:02.497926+0000","last_fullsized":"2024-09-21T01:48:02.497926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500315+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:48:02.500315+0000","last_peered":"2024-09-21T01:48:02.500315+0000","last_clean":"2024-09-21T01:48:02.500315+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:48:02.500315+0000","last_undegraded":"2024-09-21T01:48:02.500315+0000","last_fullsized":"2024-09-21T01:48:02.500315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224354+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224354+0000","last_peered":"2024-09-21T01:46:33.224354+0000","last_clean":"2024-09-21T01:46:33.224354+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224354+0000","last_undegraded":"2024-09-21T01:46:33.224354+0000","last_fullsized":"2024-09-21T01:46:33.224354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483729+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:48:02.483729+0000","last_peered":"2024-09-21T01:48:02.483729+0000","last_clean":"2024-09-21T01:48:02.483729+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:48:02.483729+0000","last_undegraded":"2024-09-21T01:48:02.483729+0000","last_fullsized":"2024-09-21T01:48:02.483729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381551+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:48:13.381551+0000","last_peered":"2024-09-21T01:48:13.381551+0000","last_clean":"2024-09-21T01:48:13.381551+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:48:13.381551+0000","last_undegraded":"2024-09-21T01:48:13.381551+0000","last_fullsized":"2024-09-21T01:48:13.381551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71105,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1164,"num_read_kb":657,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":219374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1053984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1445888,"data_stored":991771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":398983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712544,"statfs":{"total":95995035648,"available":95961645056,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":464043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":464355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:48:18.895 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.044+0000 7fc245120640 1 -- 172.21.15.29:0/3476782780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240103600 msgr2=0x7fc240103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.044+0000 7fc245120640 1 --2- 172.21.15.29:0/3476782780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240103600 0x7fc240103a10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc228009900 tx=0x7fc22802a410 comp rx=0 tx=0).stop 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 -- 172.21.15.29:0/3476782780 shutdown_connections 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 --2- 172.21.15.29:0/3476782780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc240103fa0 0x7fc24010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 --2- 172.21.15.29:0/3476782780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240103600 0x7fc240103a10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 --2- 172.21.15.29:0/3476782780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240102580 0x7fc2401029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 -- 172.21.15.29:0/3476782780 >> 172.21.15.29:0/3476782780 conn(0x7fc2400fdee0 msgr2=0x7fc240100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:19.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 -- 172.21.15.29:0/3476782780 shutdown_connections 2024-09-21T01:48:19.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 -- 172.21.15.29:0/3476782780 wait complete. 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 Processor -- start 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.045+0000 7fc245120640 1 -- start start 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc240103600 0x7fc24010b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 0x7fc24010baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc24010f310 con 0x7fc240102580 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc24010f190 con 0x7fc240103fa0 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc245120640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc24010f490 con 0x7fc240103600 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44368/0 (socket says 172.21.15.29:44368) 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 -- 172.21.15.29:0/3043472581 learned_addr learned my addr 172.21.15.29:0/3043472581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23f59e640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 0x7fc24010baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 -- 172.21.15.29:0/3043472581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc240103600 msgr2=0x7fc24010b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc240103600 0x7fc24010b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 -- 172.21.15.29:0/3043472581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 msgr2=0x7fc24010baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 0x7fc24010baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 -- 172.21.15.29:0/3043472581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24010c220 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23f59e640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 0x7fc24010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23ed9d640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e9f0 tx=0x7fc23000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.046+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000cd80 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.047+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2401a3f40 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.047+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2300108e0 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.047+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230018980 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.047+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401a4350 con 0x7fc240102580 2024-09-21T01:48:19.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.048+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc240109730 con 0x7fc240102580 2024-09-21T01:48:19.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.049+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc230010430 con 0x7fc240102580 2024-09-21T01:48:19.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.050+0000 7fc23cd99640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 0x7fc22c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.051+0000 7fc237fff640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 0x7fc22c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.051+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2300e7bf0 con 0x7fc240102580 2024-09-21T01:48:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.051+0000 7fc237fff640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 0x7fc22c0b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc228031500 tx=0x7fc228031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.054+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2300ae490 con 0x7fc240102580 2024-09-21T01:48:19.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.239+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc240063800 con 0x7fc22c0b6510 2024-09-21T01:48:19.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.244+0000 7fc23cd99640 1 -- 172.21.15.29:0/3043472581 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258872 (secure 0 0 0) 0x7fc240063800 con 0x7fc22c0b6510 2024-09-21T01:48:19.246 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:19.247 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 msgr2=0x7fc22c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 0x7fc22c0b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc228031500 tx=0x7fc228031400 comp rx=0 tx=0).stop 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 msgr2=0x7fc24010ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e9f0 tx=0x7fc23000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 shutdown_connections 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc22c0b6510 0x7fc22c0b8a20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc240103fa0 0x7fc24010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc240103600 0x7fc24010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 --2- 172.21.15.29:0/3043472581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240102580 0x7fc24010ccf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 >> 172.21.15.29:0/3043472581 conn(0x7fc2400fdee0 msgr2=0x7fc240106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:19.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 shutdown_connections 2024-09-21T01:48:19.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.247+0000 7fc245120640 1 -- 172.21.15.29:0/3043472581 wait complete. 2024-09-21T01:48:19.259 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-21T01:48:18.140033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661707,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1240,"num_read_kb":721,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402728,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539096,"statfs":{"total":1151940427776,"available":1151528034304,"internally_reserved":0,"allocated":12165120,"data_stored":6757081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003147"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500723+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:48:02.500723+0000","last_peered":"2024-09-21T01:48:02.500723+0000","last_clean":"2024-09-21T01:48:02.500723+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:48:02.500723+0000","last_undegraded":"2024-09-21T01:48:02.500723+0000","last_fullsized":"2024-09-21T01:48:02.500723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481380+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:48:02.481380+0000","last_peered":"2024-09-21T01:48:02.481380+0000","last_clean":"2024-09-21T01:48:02.481380+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:48:02.481380+0000","last_undegraded":"2024-09-21T01:48:02.481380+0000","last_fullsized":"2024-09-21T01:48:02.481380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500426+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:48:02.500426+0000","last_peered":"2024-09-21T01:48:02.500426+0000","last_clean":"2024-09-21T01:48:02.500426+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:48:02.500426+0000","last_undegraded":"2024-09-21T01:48:02.500426+0000","last_fullsized":"2024-09-21T01:48:02.500426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473094+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:48:02.473094+0000","last_peered":"2024-09-21T01:48:02.473094+0000","last_clean":"2024-09-21T01:48:02.473094+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:48:02.473094+0000","last_undegraded":"2024-09-21T01:48:02.473094+0000","last_fullsized":"2024-09-21T01:48:02.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469231+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:48:02.469231+0000","last_peered":"2024-09-21T01:48:02.469231+0000","last_clean":"2024-09-21T01:48:02.469231+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:48:02.469231+0000","last_undegraded":"2024-09-21T01:48:02.469231+0000","last_fullsized":"2024-09-21T01:48:02.469231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491842+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:48:02.491842+0000","last_peered":"2024-09-21T01:48:02.491842+0000","last_clean":"2024-09-21T01:48:02.491842+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:48:02.491842+0000","last_undegraded":"2024-09-21T01:48:02.491842+0000","last_fullsized":"2024-09-21T01:48:02.491842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495253+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:48:02.495253+0000","last_peered":"2024-09-21T01:48:02.495253+0000","last_clean":"2024-09-21T01:48:02.495253+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:48:02.495253+0000","last_undegraded":"2024-09-21T01:48:02.495253+0000","last_fullsized":"2024-09-21T01:48:02.495253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472978+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:48:02.472978+0000","last_peered":"2024-09-21T01:48:02.472978+0000","last_clean":"2024-09-21T01:48:02.472978+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:48:02.472978+0000","last_undegraded":"2024-09-21T01:48:02.472978+0000","last_fullsized":"2024-09-21T01:48:02.472978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500825+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:48:02.500825+0000","last_peered":"2024-09-21T01:48:02.500825+0000","last_clean":"2024-09-21T01:48:02.500825+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:48:02.500825+0000","last_undegraded":"2024-09-21T01:48:02.500825+0000","last_fullsized":"2024-09-21T01:48:02.500825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491994+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:48:02.491994+0000","last_peered":"2024-09-21T01:48:02.491994+0000","last_clean":"2024-09-21T01:48:02.491994+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:48:02.491994+0000","last_undegraded":"2024-09-21T01:48:02.491994+0000","last_fullsized":"2024-09-21T01:48:02.491994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498101+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:48:02.498101+0000","last_peered":"2024-09-21T01:48:02.498101+0000","last_clean":"2024-09-21T01:48:02.498101+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:48:02.498101+0000","last_undegraded":"2024-09-21T01:48:02.498101+0000","last_fullsized":"2024-09-21T01:48:02.498101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485929+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:48:02.485929+0000","last_peered":"2024-09-21T01:48:02.485929+0000","last_clean":"2024-09-21T01:48:02.485929+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:48:02.485929+0000","last_undegraded":"2024-09-21T01:48:02.485929+0000","last_fullsized":"2024-09-21T01:48:02.485929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472853+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:48:02.472853+0000","last_peered":"2024-09-21T01:48:02.472853+0000","last_clean":"2024-09-21T01:48:02.472853+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:48:02.472853+0000","last_undegraded":"2024-09-21T01:48:02.472853+0000","last_fullsized":"2024-09-21T01:48:02.472853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481186+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:48:02.481186+0000","last_peered":"2024-09-21T01:48:02.481186+0000","last_clean":"2024-09-21T01:48:02.481186+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:48:02.481186+0000","last_undegraded":"2024-09-21T01:48:02.481186+0000","last_fullsized":"2024-09-21T01:48:02.481186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481111+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:48:02.481111+0000","last_peered":"2024-09-21T01:48:02.481111+0000","last_clean":"2024-09-21T01:48:02.481111+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:48:02.481111+0000","last_undegraded":"2024-09-21T01:48:02.481111+0000","last_fullsized":"2024-09-21T01:48:02.481111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500912+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:48:02.500912+0000","last_peered":"2024-09-21T01:48:02.500912+0000","last_clean":"2024-09-21T01:48:02.500912+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:48:02.500912+0000","last_undegraded":"2024-09-21T01:48:02.500912+0000","last_fullsized":"2024-09-21T01:48:02.500912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500720+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:48:02.500720+0000","last_peered":"2024-09-21T01:48:02.500720+0000","last_clean":"2024-09-21T01:48:02.500720+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:48:02.500720+0000","last_undegraded":"2024-09-21T01:48:02.500720+0000","last_fullsized":"2024-09-21T01:48:02.500720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.476661+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:48:02.476661+0000","last_peered":"2024-09-21T01:48:02.476661+0000","last_clean":"2024-09-21T01:48:02.476661+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:48:02.476661+0000","last_undegraded":"2024-09-21T01:48:02.476661+0000","last_fullsized":"2024-09-21T01:48:02.476661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478158+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:48:02.478158+0000","last_peered":"2024-09-21T01:48:02.478158+0000","last_clean":"2024-09-21T01:48:02.478158+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:48:02.478158+0000","last_undegraded":"2024-09-21T01:48:02.478158+0000","last_fullsized":"2024-09-21T01:48:02.478158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485136+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:48:02.485136+0000","last_peered":"2024-09-21T01:48:02.485136+0000","last_clean":"2024-09-21T01:48:02.485136+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:48:02.485136+0000","last_undegraded":"2024-09-21T01:48:02.485136+0000","last_fullsized":"2024-09-21T01:48:02.485136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.475250+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:48:02.475250+0000","last_peered":"2024-09-21T01:48:02.475250+0000","last_clean":"2024-09-21T01:48:02.475250+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:48:02.475250+0000","last_undegraded":"2024-09-21T01:48:02.475250+0000","last_fullsized":"2024-09-21T01:48:02.475250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.501498+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:48:02.501498+0000","last_peered":"2024-09-21T01:48:02.501498+0000","last_clean":"2024-09-21T01:48:02.501498+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:48:02.501498+0000","last_undegraded":"2024-09-21T01:48:02.501498+0000","last_fullsized":"2024-09-21T01:48:02.501498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484582+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:48:02.484582+0000","last_peered":"2024-09-21T01:48:02.484582+0000","last_clean":"2024-09-21T01:48:02.484582+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:48:02.484582+0000","last_undegraded":"2024-09-21T01:48:02.484582+0000","last_fullsized":"2024-09-21T01:48:02.484582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473115+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:48:02.473115+0000","last_peered":"2024-09-21T01:48:02.473115+0000","last_clean":"2024-09-21T01:48:02.473115+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:48:02.473115+0000","last_undegraded":"2024-09-21T01:48:02.473115+0000","last_fullsized":"2024-09-21T01:48:02.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483131+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:48:02.483131+0000","last_peered":"2024-09-21T01:48:02.483131+0000","last_clean":"2024-09-21T01:48:02.483131+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:48:02.483131+0000","last_undegraded":"2024-09-21T01:48:02.483131+0000","last_fullsized":"2024-09-21T01:48:02.483131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477192+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:48:02.477192+0000","last_peered":"2024-09-21T01:48:02.477192+0000","last_clean":"2024-09-21T01:48:02.477192+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:48:02.477192+0000","last_undegraded":"2024-09-21T01:48:02.477192+0000","last_fullsized":"2024-09-21T01:48:02.477192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478134+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:48:02.478134+0000","last_peered":"2024-09-21T01:48:02.478134+0000","last_clean":"2024-09-21T01:48:02.478134+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:48:02.478134+0000","last_undegraded":"2024-09-21T01:48:02.478134+0000","last_fullsized":"2024-09-21T01:48:02.478134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486640+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:48:02.486640+0000","last_peered":"2024-09-21T01:48:02.486640+0000","last_clean":"2024-09-21T01:48:02.486640+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:48:02.486640+0000","last_undegraded":"2024-09-21T01:48:02.486640+0000","last_fullsized":"2024-09-21T01:48:02.486640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481987+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:48:02.481987+0000","last_peered":"2024-09-21T01:48:02.481987+0000","last_clean":"2024-09-21T01:48:02.481987+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:48:02.481987+0000","last_undegraded":"2024-09-21T01:48:02.481987+0000","last_fullsized":"2024-09-21T01:48:02.481987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.480441+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:48:02.480441+0000","last_peered":"2024-09-21T01:48:02.480441+0000","last_clean":"2024-09-21T01:48:02.480441+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:48:02.480441+0000","last_undegraded":"2024-09-21T01:48:02.480441+0000","last_fullsized":"2024-09-21T01:48:02.480441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483409+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:48:02.483409+0000","last_peered":"2024-09-21T01:48:02.483409+0000","last_clean":"2024-09-21T01:48:02.483409+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:48:02.483409+0000","last_undegraded":"2024-09-21T01:48:02.483409+0000","last_fullsized":"2024-09-21T01:48:02.483409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484143+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:48:02.484143+0000","last_peered":"2024-09-21T01:48:02.484143+0000","last_clean":"2024-09-21T01:48:02.484143+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:48:02.484143+0000","last_undegraded":"2024-09-21T01:48:02.484143+0000","last_fullsized":"2024-09-21T01:48:02.484143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.468518+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:48:02.468518+0000","last_peered":"2024-09-21T01:48:02.468518+0000","last_clean":"2024-09-21T01:48:02.468518+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:48:02.468518+0000","last_undegraded":"2024-09-21T01:48:02.468518+0000","last_fullsized":"2024-09-21T01:48:02.468518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:11.380028+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:48:11.380028+0000","last_peered":"2024-09-21T01:48:11.380028+0000","last_clean":"2024-09-21T01:48:11.380028+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:48:11.380028+0000","last_undegraded":"2024-09-21T01:48:11.380028+0000","last_fullsized":"2024-09-21T01:48:11.380028+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491593+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:48:02.491593+0000","last_peered":"2024-09-21T01:48:02.491593+0000","last_clean":"2024-09-21T01:48:02.491593+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:48:02.491593+0000","last_undegraded":"2024-09-21T01:48:02.491593+0000","last_fullsized":"2024-09-21T01:48:02.491593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486175+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:48:02.486175+0000","last_peered":"2024-09-21T01:48:02.486175+0000","last_clean":"2024-09-21T01:48:02.486175+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:48:02.486175+0000","last_undegraded":"2024-09-21T01:48:02.486175+0000","last_fullsized":"2024-09-21T01:48:02.486175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486993+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:48:02.486993+0000","last_peered":"2024-09-21T01:48:02.486993+0000","last_clean":"2024-09-21T01:48:02.486993+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:48:02.486993+0000","last_undegraded":"2024-09-21T01:48:02.486993+0000","last_fullsized":"2024-09-21T01:48:02.486993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":251,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381826+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:48:13.381826+0000","last_peered":"2024-09-21T01:48:13.381826+0000","last_clean":"2024-09-21T01:48:13.381826+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:48:13.381826+0000","last_undegraded":"2024-09-21T01:48:13.381826+0000","last_fullsized":"2024-09-21T01:48:13.381826+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.496142+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:48:02.496142+0000","last_peered":"2024-09-21T01:48:02.496142+0000","last_clean":"2024-09-21T01:48:02.496142+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:48:02.496142+0000","last_undegraded":"2024-09-21T01:48:02.496142+0000","last_fullsized":"2024-09-21T01:48:02.496142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483673+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:48:02.483673+0000","last_peered":"2024-09-21T01:48:02.483673+0000","last_clean":"2024-09-21T01:48:02.483673+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:48:02.483673+0000","last_undegraded":"2024-09-21T01:48:02.483673+0000","last_fullsized":"2024-09-21T01:48:02.483673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500539+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:48:02.500539+0000","last_peered":"2024-09-21T01:48:02.500539+0000","last_clean":"2024-09-21T01:48:02.500539+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:48:02.500539+0000","last_undegraded":"2024-09-21T01:48:02.500539+0000","last_fullsized":"2024-09-21T01:48:02.500539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":290,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381821+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:48:13.381821+0000","last_peered":"2024-09-21T01:48:13.381821+0000","last_clean":"2024-09-21T01:48:13.381821+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:48:13.381821+0000","last_undegraded":"2024-09-21T01:48:13.381821+0000","last_fullsized":"2024-09-21T01:48:13.381821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":179,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499321+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:48:02.499321+0000","last_peered":"2024-09-21T01:48:02.499321+0000","last_clean":"2024-09-21T01:48:02.499321+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:48:02.499321+0000","last_undegraded":"2024-09-21T01:48:02.499321+0000","last_fullsized":"2024-09-21T01:48:02.499321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486655+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:48:02.486655+0000","last_peered":"2024-09-21T01:48:02.486655+0000","last_clean":"2024-09-21T01:48:02.486655+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:48:02.486655+0000","last_undegraded":"2024-09-21T01:48:02.486655+0000","last_fullsized":"2024-09-21T01:48:02.486655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497852+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:48:02.497852+0000","last_peered":"2024-09-21T01:48:02.497852+0000","last_clean":"2024-09-21T01:48:02.497852+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:48:02.497852+0000","last_undegraded":"2024-09-21T01:48:02.497852+0000","last_fullsized":"2024-09-21T01:48:02.497852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":145,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:15.385340+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:48:15.385340+0000","last_peered":"2024-09-21T01:48:15.385340+0000","last_clean":"2024-09-21T01:48:15.385340+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:48:15.385340+0000","last_undegraded":"2024-09-21T01:48:15.385340+0000","last_fullsized":"2024-09-21T01:48:15.385340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499554+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:48:02.499554+0000","last_peered":"2024-09-21T01:48:02.499554+0000","last_clean":"2024-09-21T01:48:02.499554+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:48:02.499554+0000","last_undegraded":"2024-09-21T01:48:02.499554+0000","last_fullsized":"2024-09-21T01:48:02.499554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498081+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:48:02.498081+0000","last_peered":"2024-09-21T01:48:02.498081+0000","last_clean":"2024-09-21T01:48:02.498081+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:48:02.498081+0000","last_undegraded":"2024-09-21T01:48:02.498081+0000","last_fullsized":"2024-09-21T01:48:02.498081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477597+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:48:02.477597+0000","last_peered":"2024-09-21T01:48:02.477597+0000","last_clean":"2024-09-21T01:48:02.477597+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:48:02.477597+0000","last_undegraded":"2024-09-21T01:48:02.477597+0000","last_fullsized":"2024-09-21T01:48:02.477597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.494315+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:48:02.494315+0000","last_peered":"2024-09-21T01:48:02.494315+0000","last_clean":"2024-09-21T01:48:02.494315+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:48:02.494315+0000","last_undegraded":"2024-09-21T01:48:02.494315+0000","last_fullsized":"2024-09-21T01:48:02.494315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495872+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:48:02.495872+0000","last_peered":"2024-09-21T01:48:02.495872+0000","last_clean":"2024-09-21T01:48:02.495872+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:48:02.495872+0000","last_undegraded":"2024-09-21T01:48:02.495872+0000","last_fullsized":"2024-09-21T01:48:02.495872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.384064+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:48:14.384064+0000","last_peered":"2024-09-21T01:48:14.384064+0000","last_clean":"2024-09-21T01:48:14.384064+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:48:14.384064+0000","last_undegraded":"2024-09-21T01:48:14.384064+0000","last_fullsized":"2024-09-21T01:48:14.384064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498216+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:48:02.498216+0000","last_peered":"2024-09-21T01:48:02.498216+0000","last_clean":"2024-09-21T01:48:02.498216+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:48:02.498216+0000","last_undegraded":"2024-09-21T01:48:02.498216+0000","last_fullsized":"2024-09-21T01:48:02.498216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231673+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231673+0000","last_peered":"2024-09-21T01:46:33.231673+0000","last_clean":"2024-09-21T01:46:33.231673+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231673+0000","last_undegraded":"2024-09-21T01:46:33.231673+0000","last_fullsized":"2024-09-21T01:46:33.231673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237139+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237139+0000","last_peered":"2024-09-21T01:46:33.237139+0000","last_clean":"2024-09-21T01:46:33.237139+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237139+0000","last_undegraded":"2024-09-21T01:46:33.237139+0000","last_fullsized":"2024-09-21T01:46:33.237139+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.502137+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:48:02.502137+0000","last_peered":"2024-09-21T01:48:02.502137+0000","last_clean":"2024-09-21T01:48:02.502137+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:48:02.502137+0000","last_undegraded":"2024-09-21T01:48:02.502137+0000","last_fullsized":"2024-09-21T01:48:02.502137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231523+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231523+0000","last_peered":"2024-09-21T01:46:33.231523+0000","last_clean":"2024-09-21T01:46:33.231523+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231523+0000","last_undegraded":"2024-09-21T01:46:33.231523+0000","last_fullsized":"2024-09-21T01:46:33.231523+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469566+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:48:02.469566+0000","last_peered":"2024-09-21T01:48:02.469566+0000","last_clean":"2024-09-21T01:48:02.469566+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:48:02.469566+0000","last_undegraded":"2024-09-21T01:48:02.469566+0000","last_fullsized":"2024-09-21T01:48:02.469566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:12.380241+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:48:12.380241+0000","last_peered":"2024-09-21T01:48:12.380241+0000","last_clean":"2024-09-21T01:48:12.380241+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:48:12.380241+0000","last_undegraded":"2024-09-21T01:48:12.380241+0000","last_fullsized":"2024-09-21T01:48:12.380241+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.474910+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:48:02.474910+0000","last_peered":"2024-09-21T01:48:02.474910+0000","last_clean":"2024-09-21T01:48:02.474910+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:48:02.474910+0000","last_undegraded":"2024-09-21T01:48:02.474910+0000","last_fullsized":"2024-09-21T01:48:02.474910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"76'7","reported_seq":147,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.383550+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:48:14.383550+0000","last_peered":"2024-09-21T01:48:14.383550+0000","last_clean":"2024-09-21T01:48:14.383550+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:48:14.383550+0000","last_undegraded":"2024-09-21T01:48:14.383550+0000","last_fullsized":"2024-09-21T01:48:14.383550+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.490525+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:48:02.490525+0000","last_peered":"2024-09-21T01:48:02.490525+0000","last_clean":"2024-09-21T01:48:02.490525+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:48:02.490525+0000","last_undegraded":"2024-09-21T01:48:02.490525+0000","last_fullsized":"2024-09-21T01:48:02.490525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225768+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225768+0000","last_peered":"2024-09-21T01:46:33.225768+0000","last_clean":"2024-09-21T01:46:33.225768+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225768+0000","last_undegraded":"2024-09-21T01:46:33.225768+0000","last_fullsized":"2024-09-21T01:46:33.225768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.487242+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:48:02.487242+0000","last_peered":"2024-09-21T01:48:02.487242+0000","last_clean":"2024-09-21T01:48:02.487242+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:48:02.487242+0000","last_undegraded":"2024-09-21T01:48:02.487242+0000","last_fullsized":"2024-09-21T01:48:02.487242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230587+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230587+0000","last_peered":"2024-09-21T01:46:33.230587+0000","last_clean":"2024-09-21T01:46:33.230587+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230587+0000","last_undegraded":"2024-09-21T01:46:33.230587+0000","last_fullsized":"2024-09-21T01:46:33.230587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469299+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:48:02.469299+0000","last_peered":"2024-09-21T01:48:02.469299+0000","last_clean":"2024-09-21T01:48:02.469299+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:48:02.469299+0000","last_undegraded":"2024-09-21T01:48:02.469299+0000","last_fullsized":"2024-09-21T01:48:02.469299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'15","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:41.858878+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:41.858878+0000","last_peered":"2024-09-21T01:46:41.858878+0000","last_clean":"2024-09-21T01:46:41.858878+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:41.858878+0000","last_undegraded":"2024-09-21T01:46:41.858878+0000","last_fullsized":"2024-09-21T01:46:41.858878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484327+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:48:02.484327+0000","last_peered":"2024-09-21T01:48:02.484327+0000","last_clean":"2024-09-21T01:48:02.484327+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:48:02.484327+0000","last_undegraded":"2024-09-21T01:48:02.484327+0000","last_fullsized":"2024-09-21T01:48:02.484327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237630+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237630+0000","last_peered":"2024-09-21T01:46:33.237630+0000","last_clean":"2024-09-21T01:46:33.237630+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237630+0000","last_undegraded":"2024-09-21T01:46:33.237630+0000","last_fullsized":"2024-09-21T01:46:33.237630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485439+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:48:02.485439+0000","last_peered":"2024-09-21T01:48:02.485439+0000","last_clean":"2024-09-21T01:48:02.485439+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:48:02.485439+0000","last_undegraded":"2024-09-21T01:48:02.485439+0000","last_fullsized":"2024-09-21T01:48:02.485439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.441006+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:48:02.441006+0000","last_peered":"2024-09-21T01:48:02.441006+0000","last_clean":"2024-09-21T01:48:02.441006+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:48:02.441006+0000","last_undegraded":"2024-09-21T01:48:02.441006+0000","last_fullsized":"2024-09-21T01:48:02.441006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483903+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:48:02.483903+0000","last_peered":"2024-09-21T01:48:02.483903+0000","last_clean":"2024-09-21T01:48:02.483903+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:48:02.483903+0000","last_undegraded":"2024-09-21T01:48:02.483903+0000","last_fullsized":"2024-09-21T01:48:02.483903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486618+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:48:02.486618+0000","last_peered":"2024-09-21T01:48:02.486618+0000","last_clean":"2024-09-21T01:48:02.486618+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:48:02.486618+0000","last_undegraded":"2024-09-21T01:48:02.486618+0000","last_fullsized":"2024-09-21T01:48:02.486618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500620+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:48:02.500620+0000","last_peered":"2024-09-21T01:48:02.500620+0000","last_clean":"2024-09-21T01:48:02.500620+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:48:02.500620+0000","last_undegraded":"2024-09-21T01:48:02.500620+0000","last_fullsized":"2024-09-21T01:48:02.500620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499041+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:48:02.499041+0000","last_peered":"2024-09-21T01:48:02.499041+0000","last_clean":"2024-09-21T01:48:02.499041+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:48:02.499041+0000","last_undegraded":"2024-09-21T01:48:02.499041+0000","last_fullsized":"2024-09-21T01:48:02.499041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497926+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:48:02.497926+0000","last_peered":"2024-09-21T01:48:02.497926+0000","last_clean":"2024-09-21T01:48:02.497926+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:48:02.497926+0000","last_undegraded":"2024-09-21T01:48:02.497926+0000","last_fullsized":"2024-09-21T01:48:02.497926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500315+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:48:02.500315+0000","last_peered":"2024-09-21T01:48:02.500315+0000","last_clean":"2024-09-21T01:48:02.500315+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:48:02.500315+0000","last_undegraded":"2024-09-21T01:48:02.500315+0000","last_fullsized":"2024-09-21T01:48:02.500315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224354+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224354+0000","last_peered":"2024-09-21T01:46:33.224354+0000","last_clean":"2024-09-21T01:46:33.224354+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224354+0000","last_undegraded":"2024-09-21T01:46:33.224354+0000","last_fullsized":"2024-09-21T01:46:33.224354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483729+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:48:02.483729+0000","last_peered":"2024-09-21T01:48:02.483729+0000","last_clean":"2024-09-21T01:48:02.483729+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:48:02.483729+0000","last_undegraded":"2024-09-21T01:48:02.483729+0000","last_fullsized":"2024-09-21T01:48:02.483729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381551+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:48:13.381551+0000","last_peered":"2024-09-21T01:48:13.381551+0000","last_clean":"2024-09-21T01:48:13.381551+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:48:13.381551+0000","last_undegraded":"2024-09-21T01:48:13.381551+0000","last_fullsized":"2024-09-21T01:48:13.381551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71105,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1164,"num_read_kb":657,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":219374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1053984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1445888,"data_stored":991771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":398983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712544,"statfs":{"total":95995035648,"available":95961645056,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":464043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":464355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:48:19.263 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T01:48:19.263 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.412+0000 7fe6a36f9640 1 -- 172.21.15.29:0/1072113978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 msgr2=0x7fe69c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.412+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/1072113978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c10ba80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe694009ca0 tx=0x7fe69402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.412+0000 7fe6a36f9640 1 -- 172.21.15.29:0/1072113978 shutdown_connections 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.412+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/1072113978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c10ba80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.412+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/1072113978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/1072113978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 0x7fe69c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- 172.21.15.29:0/1072113978 >> 172.21.15.29:0/1072113978 conn(0x7fe69c0ff000 msgr2=0x7fe69c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- 172.21.15.29:0/1072113978 shutdown_connections 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- 172.21.15.29:0/1072113978 wait complete. 2024-09-21T01:48:19.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 Processor -- start 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- start start 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 0x7fe69c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10f310 con 0x7fe69c1050e0 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10f190 con 0x7fe69c1036c0 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.413+0000 7fe6a36f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10f490 con 0x7fe69c104740 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44386/0 (socket says 172.21.15.29:44386) 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 -- 172.21.15.29:0/2096031811 learned_addr learned my addr 172.21.15.29:0/2096031811 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1495640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 0x7fe69c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a0c94640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 -- 172.21.15.29:0/2096031811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 msgr2=0x7fe69c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 -- 172.21.15.29:0/2096031811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 msgr2=0x7fe69c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 0x7fe69c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 -- 172.21.15.29:0/2096031811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c19b8e0 con 0x7fe69c1050e0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a0c94640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a1c96640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe694006010 tx=0x7fe694003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694004240 con 0x7fe69c1050e0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6940317e0 con 0x7fe69c1050e0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c19bb70 con 0x7fe69c1050e0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69402fac0 con 0x7fe69c1050e0 2024-09-21T01:48:19.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.414+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1a22c0 con 0x7fe69c1050e0 2024-09-21T01:48:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.415+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c103b30 con 0x7fe69c1050e0 2024-09-21T01:48:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.417+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe69402fc60 con 0x7fe69c1050e0 2024-09-21T01:48:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.418+0000 7fe68e7fc640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 0x7fe6880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.418+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe694108880 con 0x7fe69c1050e0 2024-09-21T01:48:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.418+0000 7fe6a1495640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 0x7fe6880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:19.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.418+0000 7fe6a1495640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 0x7fe6880b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe698009bd0 tx=0x7fe698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.421+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6940cf120 con 0x7fe69c1050e0 2024-09-21T01:48:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:19 smithi066 ceph-mon[35001]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:48:19.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:19 smithi029 ceph-mon[31228]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:48:19.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.608+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 --> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe69c063800 con 0x7fe6880b6510 2024-09-21T01:48:19.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.612+0000 7fe68e7fc640 1 -- 172.21.15.29:0/2096031811 <== mgr.14152 v2:172.21.15.29:6800/796732990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258872 (secure 0 0 0) 0x7fe69c063800 con 0x7fe6880b6510 2024-09-21T01:48:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:19.616 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-21T01:48:19.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 msgr2=0x7fe6880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 0x7fe6880b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe698009bd0 tx=0x7fe698005f90 comp rx=0 tx=0).stop 2024-09-21T01:48:19.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 msgr2=0x7fe69c19b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:19.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe694006010 tx=0x7fe694003c20 comp rx=0 tx=0).stop 2024-09-21T01:48:19.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 shutdown_connections 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe6880b6510 0x7fe6880b8a20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe69c1050e0 0x7fe69c19b0c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe69c104740 0x7fe69c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 --2- 172.21.15.29:0/2096031811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69c1036c0 0x7fe69c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 >> 172.21.15.29:0/2096031811 conn(0x7fe69c0ff000 msgr2=0x7fe69c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 shutdown_connections 2024-09-21T01:48:19.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:19.615+0000 7fe6a36f9640 1 -- 172.21.15.29:0/2096031811 wait complete. 2024-09-21T01:48:19.627 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-21T01:48:18.140033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661707,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1240,"num_read_kb":721,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952422,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402728,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124539096,"statfs":{"total":1151940427776,"available":1151528034304,"internally_reserved":0,"allocated":12165120,"data_stored":6757081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003147"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500723+0000","last_change":"2024-09-21T01:46:09.853070+0000","last_active":"2024-09-21T01:48:02.500723+0000","last_peered":"2024-09-21T01:48:02.500723+0000","last_clean":"2024-09-21T01:48:02.500723+0000","last_became_active":"2024-09-21T01:46:09.852709+0000","last_became_peered":"2024-09-21T01:46:09.852709+0000","last_unstale":"2024-09-21T01:48:02.500723+0000","last_undegraded":"2024-09-21T01:48:02.500723+0000","last_fullsized":"2024-09-21T01:48:02.500723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:35.744163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481380+0000","last_change":"2024-09-21T01:46:09.849385+0000","last_active":"2024-09-21T01:48:02.481380+0000","last_peered":"2024-09-21T01:48:02.481380+0000","last_clean":"2024-09-21T01:48:02.481380+0000","last_became_active":"2024-09-21T01:46:09.848974+0000","last_became_peered":"2024-09-21T01:46:09.848974+0000","last_unstale":"2024-09-21T01:48:02.481380+0000","last_undegraded":"2024-09-21T01:48:02.481380+0000","last_fullsized":"2024-09-21T01:48:02.481380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:24.511304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500426+0000","last_change":"2024-09-21T01:46:09.848619+0000","last_active":"2024-09-21T01:48:02.500426+0000","last_peered":"2024-09-21T01:48:02.500426+0000","last_clean":"2024-09-21T01:48:02.500426+0000","last_became_active":"2024-09-21T01:46:09.848223+0000","last_became_peered":"2024-09-21T01:46:09.848223+0000","last_unstale":"2024-09-21T01:48:02.500426+0000","last_undegraded":"2024-09-21T01:48:02.500426+0000","last_fullsized":"2024-09-21T01:48:02.500426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:39.064387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473094+0000","last_change":"2024-09-21T01:46:33.234532+0000","last_active":"2024-09-21T01:48:02.473094+0000","last_peered":"2024-09-21T01:48:02.473094+0000","last_clean":"2024-09-21T01:48:02.473094+0000","last_became_active":"2024-09-21T01:46:33.234254+0000","last_became_peered":"2024-09-21T01:46:33.234254+0000","last_unstale":"2024-09-21T01:48:02.473094+0000","last_undegraded":"2024-09-21T01:48:02.473094+0000","last_fullsized":"2024-09-21T01:48:02.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:35.236256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469231+0000","last_change":"2024-09-21T01:46:09.851182+0000","last_active":"2024-09-21T01:48:02.469231+0000","last_peered":"2024-09-21T01:48:02.469231+0000","last_clean":"2024-09-21T01:48:02.469231+0000","last_became_active":"2024-09-21T01:46:09.850741+0000","last_became_peered":"2024-09-21T01:46:09.850741+0000","last_unstale":"2024-09-21T01:48:02.469231+0000","last_undegraded":"2024-09-21T01:48:02.469231+0000","last_fullsized":"2024-09-21T01:48:02.469231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:41.048051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491842+0000","last_change":"2024-09-21T01:46:09.849995+0000","last_active":"2024-09-21T01:48:02.491842+0000","last_peered":"2024-09-21T01:48:02.491842+0000","last_clean":"2024-09-21T01:48:02.491842+0000","last_became_active":"2024-09-21T01:46:09.849597+0000","last_became_peered":"2024-09-21T01:46:09.849597+0000","last_unstale":"2024-09-21T01:48:02.491842+0000","last_undegraded":"2024-09-21T01:48:02.491842+0000","last_fullsized":"2024-09-21T01:48:02.491842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:07.336824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495253+0000","last_change":"2024-09-21T01:46:09.848749+0000","last_active":"2024-09-21T01:48:02.495253+0000","last_peered":"2024-09-21T01:48:02.495253+0000","last_clean":"2024-09-21T01:48:02.495253+0000","last_became_active":"2024-09-21T01:46:09.848342+0000","last_became_peered":"2024-09-21T01:46:09.848342+0000","last_unstale":"2024-09-21T01:48:02.495253+0000","last_undegraded":"2024-09-21T01:48:02.495253+0000","last_fullsized":"2024-09-21T01:48:02.495253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:32.667939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472978+0000","last_change":"2024-09-21T01:46:33.231081+0000","last_active":"2024-09-21T01:48:02.472978+0000","last_peered":"2024-09-21T01:48:02.472978+0000","last_clean":"2024-09-21T01:48:02.472978+0000","last_became_active":"2024-09-21T01:46:33.230718+0000","last_became_peered":"2024-09-21T01:46:33.230718+0000","last_unstale":"2024-09-21T01:48:02.472978+0000","last_undegraded":"2024-09-21T01:48:02.472978+0000","last_fullsized":"2024-09-21T01:48:02.472978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:44.967864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500825+0000","last_change":"2024-09-21T01:46:09.846405+0000","last_active":"2024-09-21T01:48:02.500825+0000","last_peered":"2024-09-21T01:48:02.500825+0000","last_clean":"2024-09-21T01:48:02.500825+0000","last_became_active":"2024-09-21T01:46:09.846018+0000","last_became_peered":"2024-09-21T01:46:09.846018+0000","last_unstale":"2024-09-21T01:48:02.500825+0000","last_undegraded":"2024-09-21T01:48:02.500825+0000","last_fullsized":"2024-09-21T01:48:02.500825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:35.252202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491994+0000","last_change":"2024-09-21T01:46:09.856074+0000","last_active":"2024-09-21T01:48:02.491994+0000","last_peered":"2024-09-21T01:48:02.491994+0000","last_clean":"2024-09-21T01:48:02.491994+0000","last_became_active":"2024-09-21T01:46:09.855677+0000","last_became_peered":"2024-09-21T01:46:09.855677+0000","last_unstale":"2024-09-21T01:48:02.491994+0000","last_undegraded":"2024-09-21T01:48:02.491994+0000","last_fullsized":"2024-09-21T01:48:02.491994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.849306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498101+0000","last_change":"2024-09-21T01:46:09.860414+0000","last_active":"2024-09-21T01:48:02.498101+0000","last_peered":"2024-09-21T01:48:02.498101+0000","last_clean":"2024-09-21T01:48:02.498101+0000","last_became_active":"2024-09-21T01:46:09.859849+0000","last_became_peered":"2024-09-21T01:46:09.859849+0000","last_unstale":"2024-09-21T01:48:02.498101+0000","last_undegraded":"2024-09-21T01:48:02.498101+0000","last_fullsized":"2024-09-21T01:48:02.498101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:38.049329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485929+0000","last_change":"2024-09-21T01:46:09.848744+0000","last_active":"2024-09-21T01:48:02.485929+0000","last_peered":"2024-09-21T01:48:02.485929+0000","last_clean":"2024-09-21T01:48:02.485929+0000","last_became_active":"2024-09-21T01:46:09.848031+0000","last_became_peered":"2024-09-21T01:46:09.848031+0000","last_unstale":"2024-09-21T01:48:02.485929+0000","last_undegraded":"2024-09-21T01:48:02.485929+0000","last_fullsized":"2024-09-21T01:48:02.485929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.981274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.472853+0000","last_change":"2024-09-21T01:46:09.850616+0000","last_active":"2024-09-21T01:48:02.472853+0000","last_peered":"2024-09-21T01:48:02.472853+0000","last_clean":"2024-09-21T01:48:02.472853+0000","last_became_active":"2024-09-21T01:46:09.850159+0000","last_became_peered":"2024-09-21T01:46:09.850159+0000","last_unstale":"2024-09-21T01:48:02.472853+0000","last_undegraded":"2024-09-21T01:48:02.472853+0000","last_fullsized":"2024-09-21T01:48:02.472853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:47.184664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481186+0000","last_change":"2024-09-21T01:46:09.842110+0000","last_active":"2024-09-21T01:48:02.481186+0000","last_peered":"2024-09-21T01:48:02.481186+0000","last_clean":"2024-09-21T01:48:02.481186+0000","last_became_active":"2024-09-21T01:46:09.841727+0000","last_became_peered":"2024-09-21T01:46:09.841727+0000","last_unstale":"2024-09-21T01:48:02.481186+0000","last_undegraded":"2024-09-21T01:48:02.481186+0000","last_fullsized":"2024-09-21T01:48:02.481186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:05.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481111+0000","last_change":"2024-09-21T01:46:09.846154+0000","last_active":"2024-09-21T01:48:02.481111+0000","last_peered":"2024-09-21T01:48:02.481111+0000","last_clean":"2024-09-21T01:48:02.481111+0000","last_became_active":"2024-09-21T01:46:09.845705+0000","last_became_peered":"2024-09-21T01:46:09.845705+0000","last_unstale":"2024-09-21T01:48:02.481111+0000","last_undegraded":"2024-09-21T01:48:02.481111+0000","last_fullsized":"2024-09-21T01:48:02.481111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:58.867813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500912+0000","last_change":"2024-09-21T01:46:09.860028+0000","last_active":"2024-09-21T01:48:02.500912+0000","last_peered":"2024-09-21T01:48:02.500912+0000","last_clean":"2024-09-21T01:48:02.500912+0000","last_became_active":"2024-09-21T01:46:09.859657+0000","last_became_peered":"2024-09-21T01:46:09.859657+0000","last_unstale":"2024-09-21T01:48:02.500912+0000","last_undegraded":"2024-09-21T01:48:02.500912+0000","last_fullsized":"2024-09-21T01:48:02.500912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500720+0000","last_change":"2024-09-21T01:46:09.845161+0000","last_active":"2024-09-21T01:48:02.500720+0000","last_peered":"2024-09-21T01:48:02.500720+0000","last_clean":"2024-09-21T01:48:02.500720+0000","last_became_active":"2024-09-21T01:46:09.844752+0000","last_became_peered":"2024-09-21T01:46:09.844752+0000","last_unstale":"2024-09-21T01:48:02.500720+0000","last_undegraded":"2024-09-21T01:48:02.500720+0000","last_fullsized":"2024-09-21T01:48:02.500720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:54.881291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.476661+0000","last_change":"2024-09-21T01:46:09.845475+0000","last_active":"2024-09-21T01:48:02.476661+0000","last_peered":"2024-09-21T01:48:02.476661+0000","last_clean":"2024-09-21T01:48:02.476661+0000","last_became_active":"2024-09-21T01:46:09.845057+0000","last_became_peered":"2024-09-21T01:46:09.845057+0000","last_unstale":"2024-09-21T01:48:02.476661+0000","last_undegraded":"2024-09-21T01:48:02.476661+0000","last_fullsized":"2024-09-21T01:48:02.476661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:37.686184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478158+0000","last_change":"2024-09-21T01:46:09.851807+0000","last_active":"2024-09-21T01:48:02.478158+0000","last_peered":"2024-09-21T01:48:02.478158+0000","last_clean":"2024-09-21T01:48:02.478158+0000","last_became_active":"2024-09-21T01:46:09.851398+0000","last_became_peered":"2024-09-21T01:46:09.851398+0000","last_unstale":"2024-09-21T01:48:02.478158+0000","last_undegraded":"2024-09-21T01:48:02.478158+0000","last_fullsized":"2024-09-21T01:48:02.478158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:28.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485136+0000","last_change":"2024-09-21T01:46:09.846354+0000","last_active":"2024-09-21T01:48:02.485136+0000","last_peered":"2024-09-21T01:48:02.485136+0000","last_clean":"2024-09-21T01:48:02.485136+0000","last_became_active":"2024-09-21T01:46:09.845678+0000","last_became_peered":"2024-09-21T01:46:09.845678+0000","last_unstale":"2024-09-21T01:48:02.485136+0000","last_undegraded":"2024-09-21T01:48:02.485136+0000","last_fullsized":"2024-09-21T01:48:02.485136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:29.386680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.475250+0000","last_change":"2024-09-21T01:46:09.855931+0000","last_active":"2024-09-21T01:48:02.475250+0000","last_peered":"2024-09-21T01:48:02.475250+0000","last_clean":"2024-09-21T01:48:02.475250+0000","last_became_active":"2024-09-21T01:46:09.855593+0000","last_became_peered":"2024-09-21T01:46:09.855593+0000","last_unstale":"2024-09-21T01:48:02.475250+0000","last_undegraded":"2024-09-21T01:48:02.475250+0000","last_fullsized":"2024-09-21T01:48:02.475250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:02.302428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.501498+0000","last_change":"2024-09-21T01:46:09.861275+0000","last_active":"2024-09-21T01:48:02.501498+0000","last_peered":"2024-09-21T01:48:02.501498+0000","last_clean":"2024-09-21T01:48:02.501498+0000","last_became_active":"2024-09-21T01:46:09.860923+0000","last_became_peered":"2024-09-21T01:46:09.860923+0000","last_unstale":"2024-09-21T01:48:02.501498+0000","last_undegraded":"2024-09-21T01:48:02.501498+0000","last_fullsized":"2024-09-21T01:48:02.501498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:56.006314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484582+0000","last_change":"2024-09-21T01:46:09.857060+0000","last_active":"2024-09-21T01:48:02.484582+0000","last_peered":"2024-09-21T01:48:02.484582+0000","last_clean":"2024-09-21T01:48:02.484582+0000","last_became_active":"2024-09-21T01:46:09.856731+0000","last_became_peered":"2024-09-21T01:46:09.856731+0000","last_unstale":"2024-09-21T01:48:02.484582+0000","last_undegraded":"2024-09-21T01:48:02.484582+0000","last_fullsized":"2024-09-21T01:48:02.484582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:17.221893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.473115+0000","last_change":"2024-09-21T01:46:09.841436+0000","last_active":"2024-09-21T01:48:02.473115+0000","last_peered":"2024-09-21T01:48:02.473115+0000","last_clean":"2024-09-21T01:48:02.473115+0000","last_became_active":"2024-09-21T01:46:09.840856+0000","last_became_peered":"2024-09-21T01:46:09.840856+0000","last_unstale":"2024-09-21T01:48:02.473115+0000","last_undegraded":"2024-09-21T01:48:02.473115+0000","last_fullsized":"2024-09-21T01:48:02.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483131+0000","last_change":"2024-09-21T01:46:09.846406+0000","last_active":"2024-09-21T01:48:02.483131+0000","last_peered":"2024-09-21T01:48:02.483131+0000","last_clean":"2024-09-21T01:48:02.483131+0000","last_became_active":"2024-09-21T01:46:09.846028+0000","last_became_peered":"2024-09-21T01:46:09.846028+0000","last_unstale":"2024-09-21T01:48:02.483131+0000","last_undegraded":"2024-09-21T01:48:02.483131+0000","last_fullsized":"2024-09-21T01:48:02.483131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:51.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477192+0000","last_change":"2024-09-21T01:46:09.854450+0000","last_active":"2024-09-21T01:48:02.477192+0000","last_peered":"2024-09-21T01:48:02.477192+0000","last_clean":"2024-09-21T01:48:02.477192+0000","last_became_active":"2024-09-21T01:46:09.853371+0000","last_became_peered":"2024-09-21T01:46:09.853371+0000","last_unstale":"2024-09-21T01:48:02.477192+0000","last_undegraded":"2024-09-21T01:48:02.477192+0000","last_fullsized":"2024-09-21T01:48:02.477192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:09:14.458372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.478134+0000","last_change":"2024-09-21T01:46:09.845780+0000","last_active":"2024-09-21T01:48:02.478134+0000","last_peered":"2024-09-21T01:48:02.478134+0000","last_clean":"2024-09-21T01:48:02.478134+0000","last_became_active":"2024-09-21T01:46:09.845412+0000","last_became_peered":"2024-09-21T01:46:09.845412+0000","last_unstale":"2024-09-21T01:48:02.478134+0000","last_undegraded":"2024-09-21T01:48:02.478134+0000","last_fullsized":"2024-09-21T01:48:02.478134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:15.472494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486640+0000","last_change":"2024-09-21T01:46:09.839323+0000","last_active":"2024-09-21T01:48:02.486640+0000","last_peered":"2024-09-21T01:48:02.486640+0000","last_clean":"2024-09-21T01:48:02.486640+0000","last_became_active":"2024-09-21T01:46:09.838990+0000","last_became_peered":"2024-09-21T01:46:09.838990+0000","last_unstale":"2024-09-21T01:48:02.486640+0000","last_undegraded":"2024-09-21T01:48:02.486640+0000","last_fullsized":"2024-09-21T01:48:02.486640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:36.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.481987+0000","last_change":"2024-09-21T01:46:09.843532+0000","last_active":"2024-09-21T01:48:02.481987+0000","last_peered":"2024-09-21T01:48:02.481987+0000","last_clean":"2024-09-21T01:48:02.481987+0000","last_became_active":"2024-09-21T01:46:09.843144+0000","last_became_peered":"2024-09-21T01:46:09.843144+0000","last_unstale":"2024-09-21T01:48:02.481987+0000","last_undegraded":"2024-09-21T01:48:02.481987+0000","last_fullsized":"2024-09-21T01:48:02.481987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:41.921793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.480441+0000","last_change":"2024-09-21T01:46:09.851761+0000","last_active":"2024-09-21T01:48:02.480441+0000","last_peered":"2024-09-21T01:48:02.480441+0000","last_clean":"2024-09-21T01:48:02.480441+0000","last_became_active":"2024-09-21T01:46:09.851339+0000","last_became_peered":"2024-09-21T01:46:09.851339+0000","last_unstale":"2024-09-21T01:48:02.480441+0000","last_undegraded":"2024-09-21T01:48:02.480441+0000","last_fullsized":"2024-09-21T01:48:02.480441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:14.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483409+0000","last_change":"2024-09-21T01:46:09.851419+0000","last_active":"2024-09-21T01:48:02.483409+0000","last_peered":"2024-09-21T01:48:02.483409+0000","last_clean":"2024-09-21T01:48:02.483409+0000","last_became_active":"2024-09-21T01:46:09.851052+0000","last_became_peered":"2024-09-21T01:46:09.851052+0000","last_unstale":"2024-09-21T01:48:02.483409+0000","last_undegraded":"2024-09-21T01:48:02.483409+0000","last_fullsized":"2024-09-21T01:48:02.483409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:16.841806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484143+0000","last_change":"2024-09-21T01:46:09.853399+0000","last_active":"2024-09-21T01:48:02.484143+0000","last_peered":"2024-09-21T01:48:02.484143+0000","last_clean":"2024-09-21T01:48:02.484143+0000","last_became_active":"2024-09-21T01:46:09.849535+0000","last_became_peered":"2024-09-21T01:46:09.849535+0000","last_unstale":"2024-09-21T01:48:02.484143+0000","last_undegraded":"2024-09-21T01:48:02.484143+0000","last_fullsized":"2024-09-21T01:48:02.484143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:41.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.468518+0000","last_change":"2024-09-21T01:46:09.843692+0000","last_active":"2024-09-21T01:48:02.468518+0000","last_peered":"2024-09-21T01:48:02.468518+0000","last_clean":"2024-09-21T01:48:02.468518+0000","last_became_active":"2024-09-21T01:46:09.842661+0000","last_became_peered":"2024-09-21T01:46:09.842661+0000","last_unstale":"2024-09-21T01:48:02.468518+0000","last_undegraded":"2024-09-21T01:48:02.468518+0000","last_fullsized":"2024-09-21T01:48:02.468518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:31.749504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:11.380028+0000","last_change":"2024-09-21T01:46:07.835074+0000","last_active":"2024-09-21T01:48:11.380028+0000","last_peered":"2024-09-21T01:48:11.380028+0000","last_clean":"2024-09-21T01:48:11.380028+0000","last_became_active":"2024-09-21T01:46:07.834532+0000","last_became_peered":"2024-09-21T01:46:07.834532+0000","last_unstale":"2024-09-21T01:48:11.380028+0000","last_undegraded":"2024-09-21T01:48:11.380028+0000","last_fullsized":"2024-09-21T01:48:11.380028+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:27.497972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.491593+0000","last_change":"2024-09-21T01:46:09.838880+0000","last_active":"2024-09-21T01:48:02.491593+0000","last_peered":"2024-09-21T01:48:02.491593+0000","last_clean":"2024-09-21T01:48:02.491593+0000","last_became_active":"2024-09-21T01:46:09.838481+0000","last_became_peered":"2024-09-21T01:46:09.838481+0000","last_unstale":"2024-09-21T01:48:02.491593+0000","last_undegraded":"2024-09-21T01:48:02.491593+0000","last_fullsized":"2024-09-21T01:48:02.491593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:50.120948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225466+0000","last_change":"2024-09-21T01:46:07.833454+0000","last_active":"2024-09-21T01:46:33.225466+0000","last_peered":"2024-09-21T01:46:33.225466+0000","last_clean":"2024-09-21T01:46:33.225466+0000","last_became_active":"2024-09-21T01:46:07.832971+0000","last_became_peered":"2024-09-21T01:46:07.832971+0000","last_unstale":"2024-09-21T01:46:33.225466+0000","last_undegraded":"2024-09-21T01:46:33.225466+0000","last_fullsized":"2024-09-21T01:46:33.225466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:46.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.226209+0000","last_change":"2024-09-21T01:46:07.826679+0000","last_active":"2024-09-21T01:46:33.226209+0000","last_peered":"2024-09-21T01:46:33.226209+0000","last_clean":"2024-09-21T01:46:33.226209+0000","last_became_active":"2024-09-21T01:46:07.826040+0000","last_became_peered":"2024-09-21T01:46:07.826040+0000","last_unstale":"2024-09-21T01:46:33.226209+0000","last_undegraded":"2024-09-21T01:46:33.226209+0000","last_fullsized":"2024-09-21T01:46:33.226209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:30.954925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486175+0000","last_change":"2024-09-21T01:46:09.838919+0000","last_active":"2024-09-21T01:48:02.486175+0000","last_peered":"2024-09-21T01:48:02.486175+0000","last_clean":"2024-09-21T01:48:02.486175+0000","last_became_active":"2024-09-21T01:46:09.838551+0000","last_became_peered":"2024-09-21T01:46:09.838551+0000","last_unstale":"2024-09-21T01:48:02.486175+0000","last_undegraded":"2024-09-21T01:48:02.486175+0000","last_fullsized":"2024-09-21T01:48:02.486175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228338+0000","last_change":"2024-09-21T01:46:07.836268+0000","last_active":"2024-09-21T01:46:33.228338+0000","last_peered":"2024-09-21T01:46:33.228338+0000","last_clean":"2024-09-21T01:46:33.228338+0000","last_became_active":"2024-09-21T01:46:07.835580+0000","last_became_peered":"2024-09-21T01:46:07.835580+0000","last_unstale":"2024-09-21T01:46:33.228338+0000","last_undegraded":"2024-09-21T01:46:33.228338+0000","last_fullsized":"2024-09-21T01:46:33.228338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:39.280818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486993+0000","last_change":"2024-09-21T01:46:09.844032+0000","last_active":"2024-09-21T01:48:02.486993+0000","last_peered":"2024-09-21T01:48:02.486993+0000","last_clean":"2024-09-21T01:48:02.486993+0000","last_became_active":"2024-09-21T01:46:09.843694+0000","last_became_peered":"2024-09-21T01:46:09.843694+0000","last_unstale":"2024-09-21T01:48:02.486993+0000","last_undegraded":"2024-09-21T01:48:02.486993+0000","last_fullsized":"2024-09-21T01:48:02.486993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:46.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":251,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381826+0000","last_change":"2024-09-21T01:46:07.830311+0000","last_active":"2024-09-21T01:48:13.381826+0000","last_peered":"2024-09-21T01:48:13.381826+0000","last_clean":"2024-09-21T01:48:13.381826+0000","last_became_active":"2024-09-21T01:46:07.829845+0000","last_became_peered":"2024-09-21T01:46:07.829845+0000","last_unstale":"2024-09-21T01:48:13.381826+0000","last_undegraded":"2024-09-21T01:48:13.381826+0000","last_fullsized":"2024-09-21T01:48:13.381826+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:09.677570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.496142+0000","last_change":"2024-09-21T01:46:09.850539+0000","last_active":"2024-09-21T01:48:02.496142+0000","last_peered":"2024-09-21T01:48:02.496142+0000","last_clean":"2024-09-21T01:48:02.496142+0000","last_became_active":"2024-09-21T01:46:09.850135+0000","last_became_peered":"2024-09-21T01:46:09.850135+0000","last_unstale":"2024-09-21T01:48:02.496142+0000","last_undegraded":"2024-09-21T01:48:02.496142+0000","last_fullsized":"2024-09-21T01:48:02.496142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:02.623559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231549+0000","last_change":"2024-09-21T01:46:07.835895+0000","last_active":"2024-09-21T01:46:33.231549+0000","last_peered":"2024-09-21T01:46:33.231549+0000","last_clean":"2024-09-21T01:46:33.231549+0000","last_became_active":"2024-09-21T01:46:07.835425+0000","last_became_peered":"2024-09-21T01:46:07.835425+0000","last_unstale":"2024-09-21T01:46:33.231549+0000","last_undegraded":"2024-09-21T01:46:33.231549+0000","last_fullsized":"2024-09-21T01:46:33.231549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.684186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483673+0000","last_change":"2024-09-21T01:46:09.847930+0000","last_active":"2024-09-21T01:48:02.483673+0000","last_peered":"2024-09-21T01:48:02.483673+0000","last_clean":"2024-09-21T01:48:02.483673+0000","last_became_active":"2024-09-21T01:46:09.847552+0000","last_became_peered":"2024-09-21T01:46:09.847552+0000","last_unstale":"2024-09-21T01:48:02.483673+0000","last_undegraded":"2024-09-21T01:48:02.483673+0000","last_fullsized":"2024-09-21T01:48:02.483673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:09.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228099+0000","last_change":"2024-09-21T01:46:07.994297+0000","last_active":"2024-09-21T01:46:33.228099+0000","last_peered":"2024-09-21T01:46:33.228099+0000","last_clean":"2024-09-21T01:46:33.228099+0000","last_became_active":"2024-09-21T01:46:07.993867+0000","last_became_peered":"2024-09-21T01:46:07.993867+0000","last_unstale":"2024-09-21T01:46:33.228099+0000","last_undegraded":"2024-09-21T01:46:33.228099+0000","last_fullsized":"2024-09-21T01:46:33.228099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:21.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500539+0000","last_change":"2024-09-21T01:46:09.851416+0000","last_active":"2024-09-21T01:48:02.500539+0000","last_peered":"2024-09-21T01:48:02.500539+0000","last_clean":"2024-09-21T01:48:02.500539+0000","last_became_active":"2024-09-21T01:46:09.850990+0000","last_became_peered":"2024-09-21T01:46:09.850990+0000","last_unstale":"2024-09-21T01:48:02.500539+0000","last_undegraded":"2024-09-21T01:48:02.500539+0000","last_fullsized":"2024-09-21T01:48:02.500539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:35.583594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":290,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381821+0000","last_change":"2024-09-21T01:46:07.834156+0000","last_active":"2024-09-21T01:48:13.381821+0000","last_peered":"2024-09-21T01:48:13.381821+0000","last_clean":"2024-09-21T01:48:13.381821+0000","last_became_active":"2024-09-21T01:46:07.833700+0000","last_became_peered":"2024-09-21T01:46:07.833700+0000","last_unstale":"2024-09-21T01:48:13.381821+0000","last_undegraded":"2024-09-21T01:48:13.381821+0000","last_fullsized":"2024-09-21T01:48:13.381821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:28.710151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":179,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499321+0000","last_change":"2024-09-21T01:46:09.854964+0000","last_active":"2024-09-21T01:48:02.499321+0000","last_peered":"2024-09-21T01:48:02.499321+0000","last_clean":"2024-09-21T01:48:02.499321+0000","last_became_active":"2024-09-21T01:46:09.854596+0000","last_became_peered":"2024-09-21T01:46:09.854596+0000","last_unstale":"2024-09-21T01:48:02.499321+0000","last_undegraded":"2024-09-21T01:48:02.499321+0000","last_fullsized":"2024-09-21T01:48:02.499321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:03.905350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231892+0000","last_change":"2024-09-21T01:46:07.829943+0000","last_active":"2024-09-21T01:46:33.231892+0000","last_peered":"2024-09-21T01:46:33.231892+0000","last_clean":"2024-09-21T01:46:33.231892+0000","last_became_active":"2024-09-21T01:46:07.829193+0000","last_became_peered":"2024-09-21T01:46:07.829193+0000","last_unstale":"2024-09-21T01:46:33.231892+0000","last_undegraded":"2024-09-21T01:46:33.231892+0000","last_fullsized":"2024-09-21T01:46:33.231892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:19.240826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486655+0000","last_change":"2024-09-21T01:46:09.856182+0000","last_active":"2024-09-21T01:48:02.486655+0000","last_peered":"2024-09-21T01:48:02.486655+0000","last_clean":"2024-09-21T01:48:02.486655+0000","last_became_active":"2024-09-21T01:46:09.855697+0000","last_became_peered":"2024-09-21T01:46:09.855697+0000","last_unstale":"2024-09-21T01:48:02.486655+0000","last_undegraded":"2024-09-21T01:48:02.486655+0000","last_fullsized":"2024-09-21T01:48:02.486655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:17.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228347+0000","last_change":"2024-09-21T01:46:07.830937+0000","last_active":"2024-09-21T01:46:33.228347+0000","last_peered":"2024-09-21T01:46:33.228347+0000","last_clean":"2024-09-21T01:46:33.228347+0000","last_became_active":"2024-09-21T01:46:07.830287+0000","last_became_peered":"2024-09-21T01:46:07.830287+0000","last_unstale":"2024-09-21T01:46:33.228347+0000","last_undegraded":"2024-09-21T01:46:33.228347+0000","last_fullsized":"2024-09-21T01:46:33.228347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:50.098134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497852+0000","last_change":"2024-09-21T01:46:09.857068+0000","last_active":"2024-09-21T01:48:02.497852+0000","last_peered":"2024-09-21T01:48:02.497852+0000","last_clean":"2024-09-21T01:48:02.497852+0000","last_became_active":"2024-09-21T01:46:09.856589+0000","last_became_peered":"2024-09-21T01:46:09.856589+0000","last_unstale":"2024-09-21T01:48:02.497852+0000","last_undegraded":"2024-09-21T01:48:02.497852+0000","last_fullsized":"2024-09-21T01:48:02.497852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:33.916162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'5","reported_seq":145,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:15.385340+0000","last_change":"2024-09-21T01:46:07.830783+0000","last_active":"2024-09-21T01:48:15.385340+0000","last_peered":"2024-09-21T01:48:15.385340+0000","last_clean":"2024-09-21T01:48:15.385340+0000","last_became_active":"2024-09-21T01:46:07.830220+0000","last_became_peered":"2024-09-21T01:46:07.830220+0000","last_unstale":"2024-09-21T01:48:15.385340+0000","last_undegraded":"2024-09-21T01:48:15.385340+0000","last_fullsized":"2024-09-21T01:48:15.385340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.725610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499554+0000","last_change":"2024-09-21T01:46:09.856768+0000","last_active":"2024-09-21T01:48:02.499554+0000","last_peered":"2024-09-21T01:48:02.499554+0000","last_clean":"2024-09-21T01:48:02.499554+0000","last_became_active":"2024-09-21T01:46:09.856305+0000","last_became_peered":"2024-09-21T01:46:09.856305+0000","last_unstale":"2024-09-21T01:48:02.499554+0000","last_undegraded":"2024-09-21T01:48:02.499554+0000","last_fullsized":"2024-09-21T01:48:02.499554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:30.767721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228414+0000","last_change":"2024-09-21T01:46:07.994291+0000","last_active":"2024-09-21T01:46:33.228414+0000","last_peered":"2024-09-21T01:46:33.228414+0000","last_clean":"2024-09-21T01:46:33.228414+0000","last_became_active":"2024-09-21T01:46:07.993818+0000","last_became_peered":"2024-09-21T01:46:07.993818+0000","last_unstale":"2024-09-21T01:46:33.228414+0000","last_undegraded":"2024-09-21T01:46:33.228414+0000","last_fullsized":"2024-09-21T01:46:33.228414+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.860051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498081+0000","last_change":"2024-09-21T01:46:09.842560+0000","last_active":"2024-09-21T01:48:02.498081+0000","last_peered":"2024-09-21T01:48:02.498081+0000","last_clean":"2024-09-21T01:48:02.498081+0000","last_became_active":"2024-09-21T01:46:09.842165+0000","last_became_peered":"2024-09-21T01:46:09.842165+0000","last_unstale":"2024-09-21T01:48:02.498081+0000","last_undegraded":"2024-09-21T01:48:02.498081+0000","last_fullsized":"2024-09-21T01:48:02.498081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:48.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.229260+0000","last_change":"2024-09-21T01:46:07.837344+0000","last_active":"2024-09-21T01:46:33.229260+0000","last_peered":"2024-09-21T01:46:33.229260+0000","last_clean":"2024-09-21T01:46:33.229260+0000","last_became_active":"2024-09-21T01:46:07.836666+0000","last_became_peered":"2024-09-21T01:46:07.836666+0000","last_unstale":"2024-09-21T01:46:33.229260+0000","last_undegraded":"2024-09-21T01:46:33.229260+0000","last_fullsized":"2024-09-21T01:46:33.229260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.562724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.477597+0000","last_change":"2024-09-21T01:46:33.234073+0000","last_active":"2024-09-21T01:48:02.477597+0000","last_peered":"2024-09-21T01:48:02.477597+0000","last_clean":"2024-09-21T01:48:02.477597+0000","last_became_active":"2024-09-21T01:46:33.233613+0000","last_became_peered":"2024-09-21T01:46:33.233613+0000","last_unstale":"2024-09-21T01:48:02.477597+0000","last_undegraded":"2024-09-21T01:48:02.477597+0000","last_fullsized":"2024-09-21T01:48:02.477597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:42.323025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.243944+0000","last_change":"2024-09-21T01:46:33.243859+0000","last_active":"2024-09-21T01:46:33.243944+0000","last_peered":"2024-09-21T01:46:33.243944+0000","last_clean":"2024-09-21T01:46:33.243944+0000","last_became_active":"2024-09-21T01:46:33.238196+0000","last_became_peered":"2024-09-21T01:46:33.238196+0000","last_unstale":"2024-09-21T01:46:33.243944+0000","last_undegraded":"2024-09-21T01:46:33.243944+0000","last_fullsized":"2024-09-21T01:46:33.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:22.259381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.494315+0000","last_change":"2024-09-21T01:46:09.855090+0000","last_active":"2024-09-21T01:48:02.494315+0000","last_peered":"2024-09-21T01:48:02.494315+0000","last_clean":"2024-09-21T01:48:02.494315+0000","last_became_active":"2024-09-21T01:46:09.854676+0000","last_became_peered":"2024-09-21T01:46:09.854676+0000","last_unstale":"2024-09-21T01:48:02.494315+0000","last_undegraded":"2024-09-21T01:48:02.494315+0000","last_fullsized":"2024-09-21T01:48:02.494315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.568530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.228641+0000","last_change":"2024-09-21T01:46:07.832095+0000","last_active":"2024-09-21T01:46:33.228641+0000","last_peered":"2024-09-21T01:46:33.228641+0000","last_clean":"2024-09-21T01:46:33.228641+0000","last_became_active":"2024-09-21T01:46:07.831637+0000","last_became_peered":"2024-09-21T01:46:07.831637+0000","last_unstale":"2024-09-21T01:46:33.228641+0000","last_undegraded":"2024-09-21T01:46:33.228641+0000","last_fullsized":"2024-09-21T01:46:33.228641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.077615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.495872+0000","last_change":"2024-09-21T01:46:09.857426+0000","last_active":"2024-09-21T01:48:02.495872+0000","last_peered":"2024-09-21T01:48:02.495872+0000","last_clean":"2024-09-21T01:48:02.495872+0000","last_became_active":"2024-09-21T01:46:09.857013+0000","last_became_peered":"2024-09-21T01:46:09.857013+0000","last_unstale":"2024-09-21T01:48:02.495872+0000","last_undegraded":"2024-09-21T01:48:02.495872+0000","last_fullsized":"2024-09-21T01:48:02.495872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:17.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'5","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.384064+0000","last_change":"2024-09-21T01:46:07.994447+0000","last_active":"2024-09-21T01:48:14.384064+0000","last_peered":"2024-09-21T01:48:14.384064+0000","last_clean":"2024-09-21T01:48:14.384064+0000","last_became_active":"2024-09-21T01:46:07.993983+0000","last_became_peered":"2024-09-21T01:46:07.993983+0000","last_unstale":"2024-09-21T01:48:14.384064+0000","last_undegraded":"2024-09-21T01:48:14.384064+0000","last_fullsized":"2024-09-21T01:48:14.384064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:56.283493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.498216+0000","last_change":"2024-09-21T01:46:09.856244+0000","last_active":"2024-09-21T01:48:02.498216+0000","last_peered":"2024-09-21T01:48:02.498216+0000","last_clean":"2024-09-21T01:48:02.498216+0000","last_became_active":"2024-09-21T01:46:09.855875+0000","last_became_peered":"2024-09-21T01:46:09.855875+0000","last_unstale":"2024-09-21T01:48:02.498216+0000","last_undegraded":"2024-09-21T01:48:02.498216+0000","last_fullsized":"2024-09-21T01:48:02.498216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:50.982513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231673+0000","last_change":"2024-09-21T01:44:54.076947+0000","last_active":"2024-09-21T01:46:33.231673+0000","last_peered":"2024-09-21T01:46:33.231673+0000","last_clean":"2024-09-21T01:46:33.231673+0000","last_became_active":"2024-09-21T01:44:54.071131+0000","last_became_peered":"2024-09-21T01:44:54.071131+0000","last_unstale":"2024-09-21T01:46:33.231673+0000","last_undegraded":"2024-09-21T01:46:33.231673+0000","last_fullsized":"2024-09-21T01:46:33.231673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:41:30.539750+0000","last_clean_scrub_stamp":"2024-09-21T01:41:30.539750+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:17.958797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237139+0000","last_change":"2024-09-21T01:46:07.832339+0000","last_active":"2024-09-21T01:46:33.237139+0000","last_peered":"2024-09-21T01:46:33.237139+0000","last_clean":"2024-09-21T01:46:33.237139+0000","last_became_active":"2024-09-21T01:46:07.831832+0000","last_became_peered":"2024-09-21T01:46:07.831832+0000","last_unstale":"2024-09-21T01:46:33.237139+0000","last_undegraded":"2024-09-21T01:46:33.237139+0000","last_fullsized":"2024-09-21T01:46:33.237139+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:27.668554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.502137+0000","last_change":"2024-09-21T01:46:09.846502+0000","last_active":"2024-09-21T01:48:02.502137+0000","last_peered":"2024-09-21T01:48:02.502137+0000","last_clean":"2024-09-21T01:48:02.502137+0000","last_became_active":"2024-09-21T01:46:09.846208+0000","last_became_peered":"2024-09-21T01:46:09.846208+0000","last_unstale":"2024-09-21T01:48:02.502137+0000","last_undegraded":"2024-09-21T01:48:02.502137+0000","last_fullsized":"2024-09-21T01:48:02.502137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:38.642578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.231523+0000","last_change":"2024-09-21T01:46:07.831101+0000","last_active":"2024-09-21T01:46:33.231523+0000","last_peered":"2024-09-21T01:46:33.231523+0000","last_clean":"2024-09-21T01:46:33.231523+0000","last_became_active":"2024-09-21T01:46:07.830573+0000","last_became_peered":"2024-09-21T01:46:07.830573+0000","last_unstale":"2024-09-21T01:46:33.231523+0000","last_undegraded":"2024-09-21T01:46:33.231523+0000","last_fullsized":"2024-09-21T01:46:33.231523+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:17.873596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469566+0000","last_change":"2024-09-21T01:46:09.845007+0000","last_active":"2024-09-21T01:48:02.469566+0000","last_peered":"2024-09-21T01:48:02.469566+0000","last_clean":"2024-09-21T01:48:02.469566+0000","last_became_active":"2024-09-21T01:46:09.844611+0000","last_became_peered":"2024-09-21T01:46:09.844611+0000","last_unstale":"2024-09-21T01:48:02.469566+0000","last_undegraded":"2024-09-21T01:48:02.469566+0000","last_fullsized":"2024-09-21T01:48:02.469566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":141,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:12.380241+0000","last_change":"2024-09-21T01:46:07.992445+0000","last_active":"2024-09-21T01:48:12.380241+0000","last_peered":"2024-09-21T01:48:12.380241+0000","last_clean":"2024-09-21T01:48:12.380241+0000","last_became_active":"2024-09-21T01:46:07.992053+0000","last_became_peered":"2024-09-21T01:46:07.992053+0000","last_unstale":"2024-09-21T01:48:12.380241+0000","last_undegraded":"2024-09-21T01:48:12.380241+0000","last_fullsized":"2024-09-21T01:48:12.380241+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:46.750492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.474910+0000","last_change":"2024-09-21T01:46:33.237447+0000","last_active":"2024-09-21T01:48:02.474910+0000","last_peered":"2024-09-21T01:48:02.474910+0000","last_clean":"2024-09-21T01:48:02.474910+0000","last_became_active":"2024-09-21T01:46:33.237115+0000","last_became_peered":"2024-09-21T01:46:33.237115+0000","last_unstale":"2024-09-21T01:48:02.474910+0000","last_undegraded":"2024-09-21T01:48:02.474910+0000","last_fullsized":"2024-09-21T01:48:02.474910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:07.660003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"76'7","reported_seq":147,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:14.383550+0000","last_change":"2024-09-21T01:46:07.831915+0000","last_active":"2024-09-21T01:48:14.383550+0000","last_peered":"2024-09-21T01:48:14.383550+0000","last_clean":"2024-09-21T01:48:14.383550+0000","last_became_active":"2024-09-21T01:46:07.831490+0000","last_became_peered":"2024-09-21T01:46:07.831490+0000","last_unstale":"2024-09-21T01:48:14.383550+0000","last_undegraded":"2024-09-21T01:48:14.383550+0000","last_fullsized":"2024-09-21T01:48:14.383550+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:14.518501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.490525+0000","last_change":"2024-09-21T01:46:09.851258+0000","last_active":"2024-09-21T01:48:02.490525+0000","last_peered":"2024-09-21T01:48:02.490525+0000","last_clean":"2024-09-21T01:48:02.490525+0000","last_became_active":"2024-09-21T01:46:09.850798+0000","last_became_peered":"2024-09-21T01:46:09.850798+0000","last_unstale":"2024-09-21T01:48:02.490525+0000","last_undegraded":"2024-09-21T01:48:02.490525+0000","last_fullsized":"2024-09-21T01:48:02.490525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:30.174535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.225768+0000","last_change":"2024-09-21T01:46:07.992737+0000","last_active":"2024-09-21T01:46:33.225768+0000","last_peered":"2024-09-21T01:46:33.225768+0000","last_clean":"2024-09-21T01:46:33.225768+0000","last_became_active":"2024-09-21T01:46:07.992221+0000","last_became_peered":"2024-09-21T01:46:07.992221+0000","last_unstale":"2024-09-21T01:46:33.225768+0000","last_undegraded":"2024-09-21T01:46:33.225768+0000","last_fullsized":"2024-09-21T01:46:33.225768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.547659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.487242+0000","last_change":"2024-09-21T01:46:09.860632+0000","last_active":"2024-09-21T01:48:02.487242+0000","last_peered":"2024-09-21T01:48:02.487242+0000","last_clean":"2024-09-21T01:48:02.487242+0000","last_became_active":"2024-09-21T01:46:09.860195+0000","last_became_peered":"2024-09-21T01:46:09.860195+0000","last_unstale":"2024-09-21T01:48:02.487242+0000","last_undegraded":"2024-09-21T01:48:02.487242+0000","last_fullsized":"2024-09-21T01:48:02.487242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:48.174178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230587+0000","last_change":"2024-09-21T01:46:07.992846+0000","last_active":"2024-09-21T01:46:33.230587+0000","last_peered":"2024-09-21T01:46:33.230587+0000","last_clean":"2024-09-21T01:46:33.230587+0000","last_became_active":"2024-09-21T01:46:07.992312+0000","last_became_peered":"2024-09-21T01:46:07.992312+0000","last_unstale":"2024-09-21T01:46:33.230587+0000","last_undegraded":"2024-09-21T01:46:33.230587+0000","last_fullsized":"2024-09-21T01:46:33.230587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:53.782042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.469299+0000","last_change":"2024-09-21T01:46:09.848666+0000","last_active":"2024-09-21T01:48:02.469299+0000","last_peered":"2024-09-21T01:48:02.469299+0000","last_clean":"2024-09-21T01:48:02.469299+0000","last_became_active":"2024-09-21T01:46:09.848330+0000","last_became_peered":"2024-09-21T01:46:09.848330+0000","last_unstale":"2024-09-21T01:48:02.469299+0000","last_undegraded":"2024-09-21T01:48:02.469299+0000","last_fullsized":"2024-09-21T01:48:02.469299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:13.520376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'15","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:41.858878+0000","last_change":"2024-09-21T01:46:33.240231+0000","last_active":"2024-09-21T01:46:41.858878+0000","last_peered":"2024-09-21T01:46:41.858878+0000","last_clean":"2024-09-21T01:46:41.858878+0000","last_became_active":"2024-09-21T01:46:33.229787+0000","last_became_peered":"2024-09-21T01:46:33.229787+0000","last_unstale":"2024-09-21T01:46:41.858878+0000","last_undegraded":"2024-09-21T01:46:41.858878+0000","last_fullsized":"2024-09-21T01:46:41.858878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:26.418123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.484327+0000","last_change":"2024-09-21T01:46:09.857546+0000","last_active":"2024-09-21T01:48:02.484327+0000","last_peered":"2024-09-21T01:48:02.484327+0000","last_clean":"2024-09-21T01:48:02.484327+0000","last_became_active":"2024-09-21T01:46:09.856976+0000","last_became_peered":"2024-09-21T01:46:09.856976+0000","last_unstale":"2024-09-21T01:48:02.484327+0000","last_undegraded":"2024-09-21T01:48:02.484327+0000","last_fullsized":"2024-09-21T01:48:02.484327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:45.456425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.237630+0000","last_change":"2024-09-21T01:46:07.962646+0000","last_active":"2024-09-21T01:46:33.237630+0000","last_peered":"2024-09-21T01:46:33.237630+0000","last_clean":"2024-09-21T01:46:33.237630+0000","last_became_active":"2024-09-21T01:46:07.962033+0000","last_became_peered":"2024-09-21T01:46:07.962033+0000","last_unstale":"2024-09-21T01:46:33.237630+0000","last_undegraded":"2024-09-21T01:46:33.237630+0000","last_fullsized":"2024-09-21T01:46:33.237630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:12.077550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.485439+0000","last_change":"2024-09-21T01:46:09.846293+0000","last_active":"2024-09-21T01:48:02.485439+0000","last_peered":"2024-09-21T01:48:02.485439+0000","last_clean":"2024-09-21T01:48:02.485439+0000","last_became_active":"2024-09-21T01:46:09.845811+0000","last_became_peered":"2024-09-21T01:46:09.845811+0000","last_unstale":"2024-09-21T01:48:02.485439+0000","last_undegraded":"2024-09-21T01:48:02.485439+0000","last_fullsized":"2024-09-21T01:48:02.485439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:59:00.827011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.227685+0000","last_change":"2024-09-21T01:46:07.994541+0000","last_active":"2024-09-21T01:46:33.227685+0000","last_peered":"2024-09-21T01:46:33.227685+0000","last_clean":"2024-09-21T01:46:33.227685+0000","last_became_active":"2024-09-21T01:46:07.993968+0000","last_became_peered":"2024-09-21T01:46:07.993968+0000","last_unstale":"2024-09-21T01:46:33.227685+0000","last_undegraded":"2024-09-21T01:46:33.227685+0000","last_fullsized":"2024-09-21T01:46:33.227685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:10.779117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.441006+0000","last_change":"2024-09-21T01:46:09.847366+0000","last_active":"2024-09-21T01:48:02.441006+0000","last_peered":"2024-09-21T01:48:02.441006+0000","last_clean":"2024-09-21T01:48:02.441006+0000","last_became_active":"2024-09-21T01:46:09.846842+0000","last_became_peered":"2024-09-21T01:46:09.846842+0000","last_unstale":"2024-09-21T01:48:02.441006+0000","last_undegraded":"2024-09-21T01:48:02.441006+0000","last_fullsized":"2024-09-21T01:48:02.441006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:02.540279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.223551+0000","last_change":"2024-09-21T01:46:07.828365+0000","last_active":"2024-09-21T01:46:33.223551+0000","last_peered":"2024-09-21T01:46:33.223551+0000","last_clean":"2024-09-21T01:46:33.223551+0000","last_became_active":"2024-09-21T01:46:07.827807+0000","last_became_peered":"2024-09-21T01:46:07.827807+0000","last_unstale":"2024-09-21T01:46:33.223551+0000","last_undegraded":"2024-09-21T01:46:33.223551+0000","last_fullsized":"2024-09-21T01:46:33.223551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:58.840000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483903+0000","last_change":"2024-09-21T01:46:09.848977+0000","last_active":"2024-09-21T01:48:02.483903+0000","last_peered":"2024-09-21T01:48:02.483903+0000","last_clean":"2024-09-21T01:48:02.483903+0000","last_became_active":"2024-09-21T01:46:09.848543+0000","last_became_peered":"2024-09-21T01:46:09.848543+0000","last_unstale":"2024-09-21T01:48:02.483903+0000","last_undegraded":"2024-09-21T01:48:02.483903+0000","last_fullsized":"2024-09-21T01:48:02.483903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:50.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230691+0000","last_change":"2024-09-21T01:46:07.830941+0000","last_active":"2024-09-21T01:46:33.230691+0000","last_peered":"2024-09-21T01:46:33.230691+0000","last_clean":"2024-09-21T01:46:33.230691+0000","last_became_active":"2024-09-21T01:46:07.830539+0000","last_became_peered":"2024-09-21T01:46:07.830539+0000","last_unstale":"2024-09-21T01:46:33.230691+0000","last_undegraded":"2024-09-21T01:46:33.230691+0000","last_fullsized":"2024-09-21T01:46:33.230691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:25.178455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.486618+0000","last_change":"2024-09-21T01:46:09.858233+0000","last_active":"2024-09-21T01:48:02.486618+0000","last_peered":"2024-09-21T01:48:02.486618+0000","last_clean":"2024-09-21T01:48:02.486618+0000","last_became_active":"2024-09-21T01:46:09.857353+0000","last_became_peered":"2024-09-21T01:46:09.857353+0000","last_unstale":"2024-09-21T01:48:02.486618+0000","last_undegraded":"2024-09-21T01:48:02.486618+0000","last_fullsized":"2024-09-21T01:48:02.486618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:33.381626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.245917+0000","last_change":"2024-09-21T01:46:33.245807+0000","last_active":"2024-09-21T01:46:33.245917+0000","last_peered":"2024-09-21T01:46:33.245917+0000","last_clean":"2024-09-21T01:46:33.245917+0000","last_became_active":"2024-09-21T01:46:33.234195+0000","last_became_peered":"2024-09-21T01:46:33.234195+0000","last_unstale":"2024-09-21T01:46:33.245917+0000","last_undegraded":"2024-09-21T01:46:33.245917+0000","last_fullsized":"2024-09-21T01:46:33.245917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:51.317251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500620+0000","last_change":"2024-09-21T01:46:09.855676+0000","last_active":"2024-09-21T01:48:02.500620+0000","last_peered":"2024-09-21T01:48:02.500620+0000","last_clean":"2024-09-21T01:48:02.500620+0000","last_became_active":"2024-09-21T01:46:09.855069+0000","last_became_peered":"2024-09-21T01:46:09.855069+0000","last_unstale":"2024-09-21T01:48:02.500620+0000","last_undegraded":"2024-09-21T01:48:02.500620+0000","last_fullsized":"2024-09-21T01:48:02.500620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:26.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.499041+0000","last_change":"2024-09-21T01:46:09.847766+0000","last_active":"2024-09-21T01:48:02.499041+0000","last_peered":"2024-09-21T01:48:02.499041+0000","last_clean":"2024-09-21T01:48:02.499041+0000","last_became_active":"2024-09-21T01:46:09.847397+0000","last_became_peered":"2024-09-21T01:46:09.847397+0000","last_unstale":"2024-09-21T01:48:02.499041+0000","last_undegraded":"2024-09-21T01:48:02.499041+0000","last_fullsized":"2024-09-21T01:48:02.499041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:25.152885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.241981+0000","last_change":"2024-09-21T01:46:33.241375+0000","last_active":"2024-09-21T01:46:33.241981+0000","last_peered":"2024-09-21T01:46:33.241981+0000","last_clean":"2024-09-21T01:46:33.241981+0000","last_became_active":"2024-09-21T01:46:33.231637+0000","last_became_peered":"2024-09-21T01:46:33.231637+0000","last_unstale":"2024-09-21T01:46:33.241981+0000","last_undegraded":"2024-09-21T01:46:33.241981+0000","last_fullsized":"2024-09-21T01:46:33.241981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:59.723129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.497926+0000","last_change":"2024-09-21T01:46:09.845636+0000","last_active":"2024-09-21T01:48:02.497926+0000","last_peered":"2024-09-21T01:48:02.497926+0000","last_clean":"2024-09-21T01:48:02.497926+0000","last_became_active":"2024-09-21T01:46:09.844745+0000","last_became_peered":"2024-09-21T01:46:09.844745+0000","last_unstale":"2024-09-21T01:48:02.497926+0000","last_undegraded":"2024-09-21T01:48:02.497926+0000","last_fullsized":"2024-09-21T01:48:02.497926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:36.862528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.230392+0000","last_change":"2024-09-21T01:46:07.834192+0000","last_active":"2024-09-21T01:46:33.230392+0000","last_peered":"2024-09-21T01:46:33.230392+0000","last_clean":"2024-09-21T01:46:33.230392+0000","last_became_active":"2024-09-21T01:46:07.833743+0000","last_became_peered":"2024-09-21T01:46:07.833743+0000","last_unstale":"2024-09-21T01:46:33.230392+0000","last_undegraded":"2024-09-21T01:46:33.230392+0000","last_fullsized":"2024-09-21T01:46:33.230392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:27.390440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.500315+0000","last_change":"2024-09-21T01:46:09.852734+0000","last_active":"2024-09-21T01:48:02.500315+0000","last_peered":"2024-09-21T01:48:02.500315+0000","last_clean":"2024-09-21T01:48:02.500315+0000","last_became_active":"2024-09-21T01:46:09.852476+0000","last_became_peered":"2024-09-21T01:46:09.852476+0000","last_unstale":"2024-09-21T01:48:02.500315+0000","last_undegraded":"2024-09-21T01:48:02.500315+0000","last_fullsized":"2024-09-21T01:48:02.500315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.786795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:46:33.224354+0000","last_change":"2024-09-21T01:46:07.992470+0000","last_active":"2024-09-21T01:46:33.224354+0000","last_peered":"2024-09-21T01:46:33.224354+0000","last_clean":"2024-09-21T01:46:33.224354+0000","last_became_active":"2024-09-21T01:46:07.992078+0000","last_became_peered":"2024-09-21T01:46:07.992078+0000","last_unstale":"2024-09-21T01:46:33.224354+0000","last_undegraded":"2024-09-21T01:46:33.224354+0000","last_fullsized":"2024-09-21T01:46:33.224354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:14.431026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:02.483729+0000","last_change":"2024-09-21T01:46:09.856359+0000","last_active":"2024-09-21T01:48:02.483729+0000","last_peered":"2024-09-21T01:48:02.483729+0000","last_clean":"2024-09-21T01:48:02.483729+0000","last_became_active":"2024-09-21T01:46:09.856037+0000","last_became_peered":"2024-09-21T01:46:09.856037+0000","last_unstale":"2024-09-21T01:48:02.483729+0000","last_undegraded":"2024-09-21T01:48:02.483729+0000","last_fullsized":"2024-09-21T01:48:02.483729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:08.818323+0000","last_clean_scrub_stamp":"2024-09-21T01:46:08.818323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.534826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T01:48:13.381551+0000","last_change":"2024-09-21T01:46:07.830925+0000","last_active":"2024-09-21T01:48:13.381551+0000","last_peered":"2024-09-21T01:48:13.381551+0000","last_clean":"2024-09-21T01:48:13.381551+0000","last_became_active":"2024-09-21T01:46:07.830246+0000","last_became_peered":"2024-09-21T01:46:07.830246+0000","last_unstale":"2024-09-21T01:48:13.381551+0000","last_undegraded":"2024-09-21T01:48:13.381551+0000","last_fullsized":"2024-09-21T01:48:13.381551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:46:06.805539+0000","last_clean_scrub_stamp":"2024-09-21T01:46:06.805539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:46.098222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71105,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1164,"num_read_kb":657,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":582,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":219374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":831488,"data_stored":399386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1053984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":398734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":398530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1445888,"data_stored":991771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":398983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712544,"statfs":{"total":95995035648,"available":95961645056,"internally_reserved":0,"allocated":1458176,"data_stored":990917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":398312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":851968,"data_stored":398276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":464043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":464355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":399790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:48:19.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:48:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:19 smithi190 ceph-mon[34368]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:48:19.923 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:20.323 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:20 smithi029 ceph-mon[31228]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.323 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:20 smithi029 ceph-mon[31228]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.323 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:20 smithi029 ceph-mon[31228]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 -- 172.21.15.29:0/4024418672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b50102a60 msgr2=0x7f0b50102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 --2- 172.21.15.29:0/4024418672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b50102a60 0x7f0b50102e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009900 tx=0x7f0b4c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 -- 172.21.15.29:0/4024418672 shutdown_connections 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 --2- 172.21.15.29:0/4024418672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50108cf0 0x7f0b5010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 --2- 172.21.15.29:0/4024418672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b501083a0 0x7f0b501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 --2- 172.21.15.29:0/4024418672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b50102a60 0x7f0b50102e70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 -- 172.21.15.29:0/4024418672 >> 172.21.15.29:0/4024418672 conn(0x7f0b50079730 msgr2=0x7f0b50079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 -- 172.21.15.29:0/4024418672 shutdown_connections 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.415+0000 7f0b578af640 1 -- 172.21.15.29:0/4024418672 wait complete. 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.416+0000 7f0b578af640 1 Processor -- start 2024-09-21T01:48:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.416+0000 7f0b578af640 1 -- start start 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50102a60 0x7f0b5019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 0x7f0b5019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50113a80 con 0x7f0b501083a0 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50113900 con 0x7f0b50108cf0 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b578af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50113c00 con 0x7f0b50102a60 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44402/0 (socket says 172.21.15.29:44402) 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 -- 172.21.15.29:0/2227910732 learned_addr learned my addr 172.21.15.29:0/2227910732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b5564b640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50102a60 0x7f0b5019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b55e4c640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 0x7f0b5019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 -- 172.21.15.29:0/2227910732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50102a60 msgr2=0x7f0b5019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50102a60 0x7f0b5019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 -- 172.21.15.29:0/2227910732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 msgr2=0x7f0b5019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 0x7f0b5019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 -- 172.21.15.29:0/2227910732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b501a0320 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b55e4c640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 0x7f0b5019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.417+0000 7f0b54e4a640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c970 tx=0x7f0b4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.418+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40014070 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.418+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b40004440 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.418+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b501a0610 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.418+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40003e50 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.418+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b501a6e30 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.419+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b50102e70 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.420+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0b40017070 con 0x7f0b501083a0 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.421+0000 7f0b467fc640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 0x7f0b3c0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.421+0000 7f0b5564b640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 0x7f0b3c0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.421+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0b400e75a0 con 0x7f0b501083a0 2024-09-21T01:48:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.422+0000 7f0b5564b640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 0x7f0b3c0b8bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c031440 tx=0x7f0b4c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.424+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b4001e030 con 0x7f0b501083a0 2024-09-21T01:48:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:20 smithi066 ceph-mon[35001]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:20 smithi066 ceph-mon[35001]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:20 smithi066 ceph-mon[35001]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.651+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b5019b5a0 con 0x7f0b501083a0 2024-09-21T01:48:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.652+0000 7f0b467fc640 1 -- 172.21.15.29:0/2227910732 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11970 (secure 0 0 0) 0x7f0b400ade40 con 0x7f0b501083a0 2024-09-21T01:48:20.654 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:20.654 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":28,"btime":"2024-09-21T01:48:15:216396+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:48:14.736858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:48:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 28 2024-09-21T01:48:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 msgr2=0x7f0b3c0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 0x7f0b3c0b8bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c031440 tx=0x7f0b4c005e80 comp rx=0 tx=0).stop 2024-09-21T01:48:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 msgr2=0x7f0b5019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c970 tx=0x7f0b4000ce30 comp rx=0 tx=0).stop 2024-09-21T01:48:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 shutdown_connections 2024-09-21T01:48:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0b3c0b66e0 0x7f0b3c0b8bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b50108cf0 0x7f0b5019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b501083a0 0x7f0b5019aa10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 --2- 172.21.15.29:0/2227910732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b50102a60 0x7f0b5019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 >> 172.21.15.29:0/2227910732 conn(0x7f0b50079730 msgr2=0x7f0b501013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 shutdown_connections 2024-09-21T01:48:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:20.654+0000 7f0b578af640 1 -- 172.21.15.29:0/2227910732 wait complete. 2024-09-21T01:48:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:20 smithi190 ceph-mon[34368]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:20 smithi190 ceph-mon[34368]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:20 smithi190 ceph-mon[34368]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:48:21.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:48:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:21 smithi066 ceph-mon[35001]: pgmap v360: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:48:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:21 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2227910732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:21.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:21 smithi029 ceph-mon[31228]: pgmap v360: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:48:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:21 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2227910732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:21.635 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:21.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:21 smithi190 ceph-mon[34368]: pgmap v360: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:48:21.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:21 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2227910732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:48:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 -- 172.21.15.29:0/3410372687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf08109df0 msgr2=0x7faf08110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 --2- 172.21.15.29:0/3410372687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf08109df0 0x7faf08110620 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faf00009ca0 tx=0x7faf0002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 -- 172.21.15.29:0/3410372687 shutdown_connections 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 --2- 172.21.15.29:0/3410372687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf08109df0 0x7faf08110620 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 --2- 172.21.15.29:0/3410372687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf081094a0 0x7faf081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 --2- 172.21.15.29:0/3410372687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08103b60 0x7faf08103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 -- 172.21.15.29:0/3410372687 >> 172.21.15.29:0/3410372687 conn(0x7faf0807b850 msgr2=0x7faf0807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 -- 172.21.15.29:0/3410372687 shutdown_connections 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 -- 172.21.15.29:0/3410372687 wait complete. 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.121+0000 7faf0ee49640 1 Processor -- start 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 -- start start 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf08103b60 0x7faf0819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 0x7faf0819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113ab0 con 0x7faf081094a0 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113930 con 0x7faf08109df0 2024-09-21T01:48:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0ee49640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113c30 con 0x7faf08103b60 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49826/0 (socket says 172.21.15.29:49826) 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 -- 172.21.15.29:0/2979207709 learned_addr learned my addr 172.21.15.29:0/2979207709 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0cbe5640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf08103b60 0x7faf0819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0d3e6640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 0x7faf0819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 -- 172.21.15.29:0/2979207709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf08103b60 msgr2=0x7faf0819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf08103b60 0x7faf0819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 -- 172.21.15.29:0/2979207709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 msgr2=0x7faf0819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 0x7faf0819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf07fff640 1 -- 172.21.15.29:0/2979207709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081a03d0 con 0x7faf081094a0 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.122+0000 7faf0d3e6640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 0x7faf0819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf07fff640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9c0 tx=0x7faef400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef400ccc0 con 0x7faf081094a0 2024-09-21T01:48:22.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef40054d0 con 0x7faf081094a0 2024-09-21T01:48:22.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf081a06c0 con 0x7faf081094a0 2024-09-21T01:48:22.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4010430 con 0x7faf081094a0 2024-09-21T01:48:22.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.123+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf081a6e20 con 0x7faf081094a0 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.124+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf08103f70 con 0x7faf081094a0 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.125+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7faef4005020 con 0x7faf081094a0 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.126+0000 7faf05ffb640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 0x7faef80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.126+0000 7faf0cbe5640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 0x7faef80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.126+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faef40e7760 con 0x7faf081094a0 2024-09-21T01:48:22.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.127+0000 7faf0cbe5640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 0x7faef80b8a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faefc009bd0 tx=0x7faefc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:22.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.131+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faef40adfa0 con 0x7faf081094a0 2024-09-21T01:48:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.321+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf0810b650 con 0x7faf081094a0 2024-09-21T01:48:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.322+0000 7faf05ffb640 1 -- 172.21.15.29:0/2979207709 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7faef40b2380 con 0x7faf081094a0 2024-09-21T01:48:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:22.324 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:48:22.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 msgr2=0x7faef80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:22.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 0x7faef80b8a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faefc009bd0 tx=0x7faefc005f90 comp rx=0 tx=0).stop 2024-09-21T01:48:22.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 msgr2=0x7faf0819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9c0 tx=0x7faef400ee80 comp rx=0 tx=0).stop 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 shutdown_connections 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faef80b6510 0x7faef80b8a20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf08109df0 0x7faf0819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf081094a0 0x7faf0819aa00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 --2- 172.21.15.29:0/2979207709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf08103b60 0x7faf0819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:22.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 >> 172.21.15.29:0/2979207709 conn(0x7faf0807b850 msgr2=0x7faf081074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:22.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 shutdown_connections 2024-09-21T01:48:22.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:22.326+0000 7faf0ee49640 1 -- 172.21.15.29:0/2979207709 wait complete. 2024-09-21T01:48:23.047 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T01:48:23.047 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T01:48:23.056 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:48:23.056 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T01:48:23.057 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T01:48:23.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:48:23.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:48:23.059 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T01:48:23.067 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T01:48:23.067 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T01:48:23.068 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-21T01:48:23.107 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:23.127 INFO:teuthology.orchestra.run.smithi029.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-21T01:48:23.127 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:3300,172.21.15.66:3300,172.21.15.190:3300:/volumes/qa/sv_1/c64ca240-4c76-4594-a4c1-13b15f731d9d 340G 0 340G 0% /tmp/cephfs 2024-09-21T01:48:23.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-21T01:48:23.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-21T01:48:23.262 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:23 smithi029 ceph-mon[31228]: pgmap v361: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:48:23.262 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:23 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2979207709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:23.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-21T01:48:23.333 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:23 smithi066 ceph-mon[35001]: pgmap v361: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:48:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:23 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2979207709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:23.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:23 smithi190 ceph-mon[34368]: pgmap v361: 97 pgs: 97 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:48:23.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:23 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2979207709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:48:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.009+0000 7f1acd09c640 1 -- 172.21.15.29:0/812081825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 msgr2=0x7f1ac81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.009+0000 7f1acd09c640 1 --2- 172.21.15.29:0/812081825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac81098c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009900 tx=0x7f1abc02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 -- 172.21.15.29:0/812081825 shutdown_connections 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 --2- 172.21.15.29:0/812081825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8109e00 0x7f1ac8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 --2- 172.21.15.29:0/812081825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac81098c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 --2- 172.21.15.29:0/812081825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8103bc0 0x7f1ac8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 -- 172.21.15.29:0/812081825 >> 172.21.15.29:0/812081825 conn(0x7f1ac807b910 msgr2=0x7f1ac807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 -- 172.21.15.29:0/812081825 shutdown_connections 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 -- 172.21.15.29:0/812081825 wait complete. 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.010+0000 7f1acd09c640 1 Processor -- start 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 -- start start 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8103bc0 0x7f1ac819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 0x7f1ac819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8113ac0 con 0x7f1ac81094b0 2024-09-21T01:48:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8113940 con 0x7f1ac8109e00 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1acd09c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8113c40 con 0x7f1ac8103bc0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1ac659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1ac659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49838/0 (socket says 172.21.15.29:49838) 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1ac659c640 1 -- 172.21.15.29:0/1061337229 learned_addr learned my addr 172.21.15.29:0/1061337229 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8103bc0 0x7f1ac819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.011+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 0x7f1ac819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 -- 172.21.15.29:0/1061337229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8103bc0 msgr2=0x7f1ac819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8103bc0 0x7f1ac819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 -- 172.21.15.29:0/1061337229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 msgr2=0x7f1ac819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 0x7f1ac819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 -- 172.21.15.29:0/1061337229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a0320 con 0x7f1ac81094b0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 0x7f1ac819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1abc02a930 tx=0x7f1abc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1aaffff640 1 -- 172.21.15.29:0/1061337229 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc03f030 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1aaffff640 1 -- 172.21.15.29:0/1061337229 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc003aa0 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1aaffff640 1 -- 172.21.15.29:0/1061337229 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc035070 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a05b0 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.012+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a0a90 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.015+0000 7f1ad3610740 1 Processor -- start 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.015+0000 7f1aaffff640 1 -- 172.21.15.29:0/1061337229 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1abc042030 con 0x7f1ac81094b0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.015+0000 7f1ad3610740 1 -- start start 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cd4670 0x55f23509ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f235012e60 con 0x55f234ccc490 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f235002290 con 0x55f234cd4670 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ad3610740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f2350060a0 con 0x55f234cf5350 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49854/0 (socket says 172.21.15.29:49854) 2024-09-21T01:48:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/2816867048 learned_addr learned my addr 172.21.15.29:0/2816867048 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac759e640 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cd4670 0x55f23509ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/2816867048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 msgr2=0x55f23509c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/2816867048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cd4670 msgr2=0x55f23509ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cd4670 0x55f23509ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/2816867048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f234d60930 con 0x55f234ccc490 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1aaffff640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 0x7f1ab00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.016+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 0x7f1ab00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.017+0000 7f1aaffff640 1 -- 172.21.15.29:0/1061337229 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1abc1079b0 con 0x7f1ac81094b0 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.017+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800ad00 tx=0x7f1ab800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b42a6683c863a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.017+0000 7f1aad7fa640 1 -- 172.21.15.29:0/2816867048 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab801d070 con 0x55f234ccc490 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.017+0000 7f1aad7fa640 1 -- 172.21.15.29:0/2816867048 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab8018af0 con 0x55f234ccc490 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.017+0000 7f1aad7fa640 1 -- 172.21.15.29:0/2816867048 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8010b30 con 0x55f234ccc490 2024-09-21T01:48:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 0x7f1ab00b8a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005fd0 tx=0x7f1ac0009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 -- 172.21.15.29:0/2816867048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 msgr2=0x55f23509c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800ad00 tx=0x7f1ab800f790 comp rx=0 tx=0).stop 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 -- 172.21.15.29:0/2816867048 shutdown_connections 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cd4670 0x55f23509ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f23509c970 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2816867048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 -- 172.21.15.29:0/2816867048 >> 172.21.15.29:0/2816867048 conn(0x55f234ce9740 msgr2=0x55f23509a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 -- 172.21.15.29:0/2816867048 shutdown_connections 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.018+0000 7f1ad3610740 1 -- 172.21.15.29:0/2816867048 wait complete. 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.019+0000 7f1ad3610740 1 -- start start 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.019+0000 7f1ad3610740 1 Processor -- start 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.019+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 0x55f2350b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f234e63890 con 0x55f234ccc490 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f234f9cd40 con 0x55f234cf5350 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f2350aac10 con 0x55f234cd4670 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49862/0 (socket says 172.21.15.29:49862) 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/1875615412 learned_addr learned my addr 172.21.15.29:0/1875615412 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ad3610740 10 client.0 _ll_register_callbacks cb 0x55f235000c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 0x55f2350b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/1875615412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 msgr2=0x55f2350a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 msgr2=0x55f2350b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 0x55f2350b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 -- 172.21.15.29:0/1875615412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2350b44c0 con 0x55f234ccc490 2024-09-21T01:48:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac759e640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 0x55f2350b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8010470 tx=0x7f1ab80186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc39c3ff3acc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.020+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab801d040 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ab8010870 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab8018af0 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ab8010a70 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2350b6db0 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab8018af0 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1ab8020030 con 0x55f234ccc490 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 20 client.15462 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234cb3cd0 0x55f2350ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 4 client.15462 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.021+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f235014db0 con 0x55f234cb3cd0 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.022+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234cb3cd0 0x55f2350ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.022+0000 7f1aaaff5640 10 client.15462 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.022+0000 7f1ac659c640 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234cb3cd0 0x55f2350ba1c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9936738ca6e0f676 server_cookie=2575a3580697e929 in_seq=0 out_seq=0 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1aaaff5640 1 -- 172.21.15.29:0/1875615412 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f235014db0 con 0x55f234cb3cd0 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1aaaff5640 10 client.15462 handle_command_reply: tid=0 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 client.15462 shutdown 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 msgr2=0x55f2350a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8010470 tx=0x7f1ab80186f0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 shutdown_connections 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234cb3cd0 0x55f2350ba1c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234cf5350 0x55f2350b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 --2- 172.21.15.29:0/1875615412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234ccc490 0x55f2350a8000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 >> 172.21.15.29:0/1875615412 conn(0x55f234ce9740 msgr2=0x55f23509f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1aaaff5640 0 client.15462 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 shutdown_connections 2024-09-21T01:48:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 1 -- 172.21.15.29:0/1875615412 wait complete. 2024-09-21T01:48:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.023+0000 7f1ad3610740 20 client.15462 trim_cache size 0 max 16384 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.041+0000 7f1ad3610740 1 Processor -- start 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.041+0000 7f1ad3610740 1 -- start start 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.041+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.041+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ad3610740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81af310 con 0x55f234cd4670 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ad3610740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b0e50 con 0x55f234ccc490 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ad3610740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81af9e0 con 0x55f234cf5350 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49872/0 (socket says 172.21.15.29:49872) 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 -- 172.21.15.29:0/2977639515 learned_addr learned my addr 172.21.15.29:0/2977639515 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac759e640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 -- 172.21.15.29:0/2977639515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 msgr2=0x55f23509db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 -- 172.21.15.29:0/2977639515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 msgr2=0x55f2350a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 -- 172.21.15.29:0/2977639515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2350a8d30 con 0x55f234cd4670 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1abc031a70 tx=0x7f1abc10d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e27397f49cec82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.042+0000 7f1aacff9640 1 -- 172.21.15.29:0/2977639515 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc107e60 con 0x55f234cd4670 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.043+0000 7f1aacff9640 1 -- 172.21.15.29:0/2977639515 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc03b460 con 0x55f234cd4670 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.043+0000 7f1aacff9640 1 -- 172.21.15.29:0/2977639515 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc049040 con 0x55f234cd4670 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.043+0000 7f1ad3610740 1 -- 172.21.15.29:0/2977639515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 msgr2=0x55f2350a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.043+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1abc031a70 tx=0x7f1abc10d4f0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 -- 172.21.15.29:0/2977639515 shutdown_connections 2024-09-21T01:48:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cf5350 0x55f23509db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cd4670 0x55f2350a2550 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2977639515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 -- 172.21.15.29:0/2977639515 >> 172.21.15.29:0/2977639515 conn(0x55f234ce9740 msgr2=0x55f23509d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 -- 172.21.15.29:0/2977639515 shutdown_connections 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 -- 172.21.15.29:0/2977639515 wait complete. 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 -- start start 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.044+0000 7f1ad3610740 1 Processor -- start 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350bb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b0780 con 0x55f234cf5350 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b00b0 con 0x55f234ccc490 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ad2f0 con 0x55f234cd4670 2024-09-21T01:48:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ad3610740 10 client.0 _ll_register_callbacks cb 0x7f1ab801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49886/0 (socket says 172.21.15.29:49886) 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 -- 172.21.15.29:0/2292213091 learned_addr learned my addr 172.21.15.29:0/2292213091 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350bb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 -- 172.21.15.29:0/2292213091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 msgr2=0x55f2350bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 -- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 msgr2=0x55f2350bb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350bb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 -- 172.21.15.29:0/2292213091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f235123e80 con 0x55f234cf5350 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac659c640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350bb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1ac759e640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0011dd0 tx=0x7f1ac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b5da6eb558fd8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:24.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.045+0000 7f1aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac002d060 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac001d600 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f235010860 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ac001d840 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2351251a0 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f235010860 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac0027070 con 0x55f234cf5350 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 20 client.15468 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234c89080 0x55f235127990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 4 client.15468 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:48:24.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.046+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f23500dc90 con 0x55f234c89080 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.047+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234c89080 0x55f235127990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.047+0000 7f1ac6d9d640 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234c89080 0x55f235127990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f113b047c2e9963c server_cookie=b643db8968f8e2b in_seq=0 out_seq=0 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.047+0000 7f1aaaff5640 10 client.15468 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.049+0000 7f1aaaff5640 1 -- 172.21.15.29:0/2292213091 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f23500dc90 con 0x55f234c89080 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.049+0000 7f1aaaff5640 10 client.15468 handle_command_reply: tid=0 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.049+0000 7f1ad3610740 1 client.15468 shutdown 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 msgr2=0x55f235123940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0011dd0 tx=0x7f1ac0004a40 comp rx=0 tx=0).stop 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 shutdown_connections 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f234c89080 0x55f235127990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f234cf5350 0x55f235123940 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f234cd4670 0x55f2350bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 --2- 172.21.15.29:0/2292213091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f234ccc490 0x55f2350bb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 >> 172.21.15.29:0/2292213091 conn(0x55f234ce9740 msgr2=0x55f235098850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1aaaff5640 0 client.15468 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:24.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 shutdown_connections 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 1 -- 172.21.15.29:0/2292213091 wait complete. 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.050+0000 7f1ad3610740 20 client.15468 trim_cache size 0 max 16384 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3", 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:48:24.057 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 msgr2=0x7f1ab00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 0x7f1ab00b8a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005fd0 tx=0x7f1ac0009800 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 msgr2=0x7f1ac819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1abc02a930 tx=0x7f1abc004270 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 shutdown_connections 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab00b6540 0x7f1ab00b8a50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac8109e00 0x7f1ac819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ac81094b0 0x7f1ac819aa10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 --2- 172.21.15.29:0/1061337229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac8103bc0 0x7f1ac819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 >> 172.21.15.29:0/1061337229 conn(0x7f1ac807b910 msgr2=0x7f1ac8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 shutdown_connections 2024-09-21T01:48:24.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:24.051+0000 7f1acd09c640 1 -- 172.21.15.29:0/1061337229 wait complete. 2024-09-21T01:48:24.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:24 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:48:24.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:24 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:24 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:48:24.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:24 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:24 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:48:24.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:24 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:48:24.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:24 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:24 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:48:24.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:24 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:24 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:48:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:24 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:48:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:24 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:24 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:48:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:24 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:48:24.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:24 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:48:25.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:25 smithi066 ceph-mon[35001]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-21T01:48:25.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:25 smithi029 ceph-mon[31228]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-21T01:48:25.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-21T01:48:25.719 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-21T01:48:25.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-21T01:48:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:25 smithi190 ceph-mon[34368]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-21T01:48:25.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T01:48:25.824 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:48:25.902 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/: 2024-09-21T01:48:25.902 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 data 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data: 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:25.903 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T01:48:25.905 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-21T01:48:25.994 INFO:teuthology.orchestra.run.smithi029.stdout:[Service] 2024-09-21T01:48:25.994 INFO:teuthology.orchestra.run.smithi029.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-21T01:48:25.994 INFO:teuthology.orchestra.run.smithi029.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-21T01:48:25.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-21T01:48:26.089 INFO:teuthology.orchestra.run.smithi029.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-21T01:48:26.156 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:26.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 -- 172.21.15.29:0/170126190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 msgr2=0x7ffab0110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 --2- 172.21.15.29:0/170126190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0110340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009ca0 tx=0x7ffaa802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 -- 172.21.15.29:0/170126190 shutdown_connections 2024-09-21T01:48:26.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 --2- 172.21.15.29:0/170126190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0110340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 --2- 172.21.15.29:0/170126190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01052a0 0x7ffab010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 --2- 172.21.15.29:0/170126190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01048c0 0x7ffab0104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 -- 172.21.15.29:0/170126190 >> 172.21.15.29:0/170126190 conn(0x7ffab0079760 msgr2=0x7ffab0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 -- 172.21.15.29:0/170126190 shutdown_connections 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.698+0000 7ffab69d4640 1 -- 172.21.15.29:0/170126190 wait complete. 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.699+0000 7ffab69d4640 1 Processor -- start 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.699+0000 7ffab69d4640 1 -- start start 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01048c0 0x7ffab01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0113ab0 con 0x7ffab010de70 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0113930 con 0x7ffab01052a0 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab69d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0113c30 con 0x7ffab01048c0 2024-09-21T01:48:26.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab4f71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44550/0 (socket says 172.21.15.29:44550) 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab4f71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49904/0 (socket says 172.21.15.29:49904) 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 -- 172.21.15.29:0/161343691 learned_addr learned my addr 172.21.15.29:0/161343691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaaffff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01048c0 0x7ffab01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 -- 172.21.15.29:0/161343691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01048c0 msgr2=0x7ffab01027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01048c0 0x7ffab01027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 -- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 msgr2=0x7ffab0103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 -- 172.21.15.29:0/161343691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab01a86e0 con 0x7ffab01052a0 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffab4f71640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:26.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.700+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800d950 tx=0x7ffa9800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.701+0000 7ffaadffb640 1 -- 172.21.15.29:0/161343691 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98017070 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.701+0000 7ffaadffb640 1 -- 172.21.15.29:0/161343691 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98009c30 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.701+0000 7ffaadffb640 1 -- 172.21.15.29:0/161343691 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980047a0 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.701+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab01a89d0 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.701+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab01a8ec0 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.703+0000 7ffabcf48740 1 Processor -- start 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.703+0000 7ffaadffb640 1 -- 172.21.15.29:0/161343691 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ffa98004020 con 0x7ffab01052a0 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.703+0000 7ffabcf48740 1 -- start start 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 0x55675eb5a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e792390 0x55675eb5acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55675ead0e50 con 0x55675e7b3070 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55675eac0280 con 0x55675e792390 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffabcf48740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55675eac4090 con 0x55675e78a1b0 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e792390 0x55675eb5acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49914/0 (socket says 172.21.15.29:49914) 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 -- 172.21.15.29:0/3015772286 learned_addr learned my addr 172.21.15.29:0/3015772286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffaaffff640 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 0x55675eb5a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 -- 172.21.15.29:0/3015772286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 msgr2=0x55675eb5a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 0x55675eb5a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 -- 172.21.15.29:0/3015772286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e792390 msgr2=0x55675eb5acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e792390 0x55675eb5acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffab4f71640 1 -- 172.21.15.29:0/3015772286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675e81e650 con 0x55675e7b3070 2024-09-21T01:48:26.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffaadffb640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 0x7ffa9c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.704+0000 7ffaaffff640 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 0x55675eb5a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffaaffff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 0x7ffa9c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffaadffb640 1 -- 172.21.15.29:0/161343691 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffa980e6ee0 con 0x7ffab01052a0 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffab4f71640 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8033040 tx=0x7ffaa802f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48434446cf632099 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffaa67fc640 1 -- 172.21.15.29:0/3015772286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8031650 con 0x55675e7b3070 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffaa67fc640 1 -- 172.21.15.29:0/3015772286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa8036040 con 0x55675e7b3070 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.705+0000 7ffaa67fc640 1 -- 172.21.15.29:0/3015772286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80044e0 con 0x55675e7b3070 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- 172.21.15.29:0/3015772286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 msgr2=0x55675eb5a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffaaffff640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 0x7ffa9c0b8a20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55675eb58ef0 tx=0x7ffaa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8033040 tx=0x7ffaa802f3c0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- 172.21.15.29:0/3015772286 shutdown_connections 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e792390 0x55675eb5acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e78a1b0 0x55675eb5a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3015772286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675eb5a270 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- 172.21.15.29:0/3015772286 >> 172.21.15.29:0/3015772286 conn(0x55675e7a7460 msgr2=0x55675eb58160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- 172.21.15.29:0/3015772286 shutdown_connections 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- 172.21.15.29:0/3015772286 wait complete. 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.706+0000 7ffabcf48740 1 -- start start 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 Processor -- start 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 0x55675eb71b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55675e921700 con 0x55675e792390 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55675ea5b110 con 0x55675e78a1b0 2024-09-21T01:48:26.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55675eb68870 con 0x55675e7b3070 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffabcf48740 10 client.0 _ll_register_callbacks cb 0x55675ea74990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49928/0 (socket says 172.21.15.29:49928) 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.707+0000 7ffaa7fff640 1 -- 172.21.15.29:0/1885706648 learned_addr learned my addr 172.21.15.29:0/1885706648 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffab4f71640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 0x55675eb71b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 -- 172.21.15.29:0/1885706648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 msgr2=0x55675eb71b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 0x55675eb71b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 -- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 msgr2=0x55675eb65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb65960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 -- 172.21.15.29:0/1885706648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675ea88c40 con 0x55675e792390 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffab4f71640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 0x55675eb71b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffa98000c00 tx=0x7ffa980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ca5443b79e2df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffa8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980a8f20 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffa980ac4e0 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa8036040 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa980ac7f0 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffabcf48740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.708+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55675eb745d0 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa8036040 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa98098190 con 0x55675e792390 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 20 client.15480 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e7719f0 0x55675eb77a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 4 client.15480 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55675ead2da0 con 0x55675e7719f0 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.709+0000 7ffaaffff640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e7719f0 0x55675eb77a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.710+0000 7ffaaffff640 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e7719f0 0x55675eb77a40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424291c637d7bfcc server_cookie=494b24cba38e171a in_seq=0 out_seq=0 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.710+0000 7ffa8f7fe640 10 client.15480 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.711+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/1885706648 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55675ead2da0 con 0x55675e7719f0 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.711+0000 7ffa8f7fe640 10 client.15480 handle_command_reply: tid=0 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.711+0000 7ffabcf48740 1 client.15480 shutdown 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.711+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 msgr2=0x55675eb65ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.711+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffa98000c00 tx=0x7ffa980046c0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 shutdown_connections 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e7719f0 0x55675eb77a40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e7b3070 0x55675eb71b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffa8f7fe640 0 client.15480 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e792390 0x55675eb65ea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 --2- 172.21.15.29:0/1885706648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 >> 172.21.15.29:0/1885706648 conn(0x55675e7a7460 msgr2=0x55675eb5d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 shutdown_connections 2024-09-21T01:48:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 1 -- 172.21.15.29:0/1885706648 wait complete. 2024-09-21T01:48:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.712+0000 7ffabcf48740 20 client.15480 trim_cache size 0 max 16384 2024-09-21T01:48:26.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.731+0000 7ffabcf48740 1 Processor -- start 2024-09-21T01:48:26.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.731+0000 7ffabcf48740 1 -- start start 2024-09-21T01:48:26.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e78a1b0 0x55675eb5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e7b3070 0x55675eb5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01b1480 con 0x55675e78a1b0 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01b2fc0 con 0x55675e7b3070 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffabcf48740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01b1b50 con 0x55675e792390 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40856/0 (socket says 172.21.15.29:40856) 2024-09-21T01:48:26.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 -- 172.21.15.29:0/81859629 learned_addr learned my addr 172.21.15.29:0/81859629 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:26.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 -- 172.21.15.29:0/81859629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e7b3070 msgr2=0x55675eb5da80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e7b3070 0x55675eb5da80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 -- 172.21.15.29:0/81859629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e78a1b0 msgr2=0x55675eb5bc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e78a1b0 0x55675eb5bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.733+0000 7ffaa7fff640 1 -- 172.21.15.29:0/81859629 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675eb63e60 con 0x55675e792390 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.734+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa980aa2f0 tx=0x7ffa98005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3af0889a1fa2d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.734+0000 7ffaa5ffb640 1 -- 172.21.15.29:0/81859629 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980ac9d0 con 0x55675e792390 2024-09-21T01:48:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.734+0000 7ffaa5ffb640 1 -- 172.21.15.29:0/81859629 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa980acb90 con 0x55675e792390 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.734+0000 7ffaa5ffb640 1 -- 172.21.15.29:0/81859629 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980ace50 con 0x55675e792390 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 -- 172.21.15.29:0/81859629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 msgr2=0x55675eb71a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa980aa2f0 tx=0x7ffa98005bd0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 -- 172.21.15.29:0/81859629 shutdown_connections 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e7b3070 0x55675eb5da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb71a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 --2- 172.21.15.29:0/81859629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e78a1b0 0x55675eb5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 -- 172.21.15.29:0/81859629 >> 172.21.15.29:0/81859629 conn(0x55675e7a7460 msgr2=0x55675eb5d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 -- 172.21.15.29:0/81859629 shutdown_connections 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.735+0000 7ffabcf48740 1 -- 172.21.15.29:0/81859629 wait complete. 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.736+0000 7ffabcf48740 1 -- start start 2024-09-21T01:48:26.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.736+0000 7ffabcf48740 1 Processor -- start 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.736+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.736+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb6a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffabcf48740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675ebe13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffabcf48740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01b28f0 con 0x55675e7b3070 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffabcf48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01b2220 con 0x55675e78a1b0 2024-09-21T01:48:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffabcf48740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01af460 con 0x55675e792390 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffabcf48740 10 client.0 _ll_register_callbacks cb 0x7ffa9800bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44578/0 (socket says 172.21.15.29:44578) 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaaffff640 1 -- 172.21.15.29:0/3794252238 learned_addr learned my addr 172.21.15.29:0/3794252238 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffab4f71640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675ebe13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb6a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 -- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 msgr2=0x55675eb69e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 -- 172.21.15.29:0/3794252238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 msgr2=0x55675ebe13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675ebe13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 -- 172.21.15.29:0/3794252238 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675ebe1900 con 0x55675e792390 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaaffff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:26.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffab4f71640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675ebe13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffaa7fff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb6a3b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55675ea76190 tx=0x7ffa980b6aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a237ccdaf1f7ba3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.737+0000 7ffa8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980ace50 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffa980a89d0 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa980acb90 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa980ac9d0 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 10 client.25186 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55675ebe2a00 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa980acb90 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa980b2cf0 con 0x55675e792390 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 10 client.25186 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 20 client.25186 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e746da0 0x55675ebe4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 4 client.25186 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.738+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55675eb66250 con 0x55675e746da0 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.739+0000 7ffaaffff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e746da0 0x55675ebe4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.739+0000 7ffa8f7fe640 10 client.25186 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.739+0000 7ffaaffff640 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e746da0 0x55675ebe4f40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd58507d0cba9e41 server_cookie=f3b44c2c351c967b in_seq=0 out_seq=0 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffa8f7fe640 1 -- 172.21.15.29:0/3794252238 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55675eb66250 con 0x55675e746da0 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffa8f7fe640 10 client.25186 handle_command_reply: tid=0 2024-09-21T01:48:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 client.25186 shutdown 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 msgr2=0x55675eb6a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb6a3b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55675ea76190 tx=0x7ffa980b6aa0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 shutdown_connections 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55675e746da0 0x55675ebe4f40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55675e7b3070 0x55675ebe13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55675e792390 0x55675eb6a3b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffa8f7fe640 0 client.25186 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 --2- 172.21.15.29:0/3794252238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55675e78a1b0 0x55675eb69e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.740+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 >> 172.21.15.29:0/3794252238 conn(0x55675e7a7460 msgr2=0x55675eb5ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 shutdown_connections 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffabcf48740 1 -- 172.21.15.29:0/3794252238 wait complete. 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffabcf48740 20 client.25186 trim_cache size 0 max 16384 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:48:26.745 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3", 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:48:26.746 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 msgr2=0x7ffa9c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 0x7ffa9c0b8a20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55675eb58ef0 tx=0x7ffaa0005f90 comp rx=0 tx=0).stop 2024-09-21T01:48:26.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 msgr2=0x7ffab0102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800d950 tx=0x7ffa9800de10 comp rx=0 tx=0).stop 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.741+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 shutdown_connections 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffa9c0b6510 0x7ffa9c0b8a20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab010de70 0x7ffab0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab01052a0 0x7ffab0102d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 --2- 172.21.15.29:0/161343691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab01048c0 0x7ffab01027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 >> 172.21.15.29:0/161343691 conn(0x7ffab0079760 msgr2=0x7ffab01079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 shutdown_connections 2024-09-21T01:48:26.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:26.742+0000 7ffab69d4640 1 -- 172.21.15.29:0/161343691 wait complete. 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:26 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:26 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:26 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:27.825 INFO:tasks.cephfs.filesystem:scrub status for tag:8a12d9b0-8fa5-4a08-844f-7ed03505aeb3 - {'path': '/', 'tag': '8a12d9b0-8fa5-4a08-844f-7ed03505aeb3', 'options': 'recursive,force'} 2024-09-21T01:48:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:27 smithi190 ceph-mon[34368]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-21T01:48:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:27 smithi066 ceph-mon[35001]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-21T01:48:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:27 smithi029 ceph-mon[31228]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:48:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:48:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T01:48:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:48:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:48:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:48:31.046 INFO:teuthology.orchestra.run.smithi029.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-21T01:48:31.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T01:48:31.093 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/: 2024-09-21T01:48:31.093 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.093 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 data 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data: 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:total 35 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 . 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 base 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 01:48 global 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 log 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_commit_ts 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_dynshmem 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 01:48 pg_hba.conf 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 01:48 pg_ident.conf 2024-09-21T01:48:31.094 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 pg_logical 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 pg_multixact 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_notify 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_replslot 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_serial 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_snapshots 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_stat 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_stat_tmp 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 pg_subtrans 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_tblspc 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_twophase 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 pg_wal 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 pg_xact 2024-09-21T01:48:31.095 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 01:48 postgresql.auto.conf 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 01:48 postgresql.conf 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 . 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 1 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 13493 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 13494 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:total 7777 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 . 2024-09-21T01:48:31.096 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 .. 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 01:48 1259 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T01:48:31.097 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T01:48:31.098 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T01:48:31.099 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T01:48:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T01:48:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T01:48:31.102 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 01:48 2619 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2619_fsm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T01:48:31.103 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T01:48:31.104 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T01:48:31.105 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2696 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T01:48:31.107 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T01:48:31.113 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T01:48:31.114 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T01:48:31.117 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T01:48:31.118 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:total 7777 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 . 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 .. 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T01:48:31.119 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 01:48 1259 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T01:48:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T01:48:31.121 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T01:48:31.122 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T01:48:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T01:48:31.124 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 01:48 2619 2024-09-21T01:48:31.125 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2619_fsm 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T01:48:31.126 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T01:48:31.127 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2696 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T01:48:31.128 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T01:48:31.129 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T01:48:31.130 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T01:48:31.131 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T01:48:31.132 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T01:48:31.133 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T01:48:31.134 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T01:48:31.135 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T01:48:31.136 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T01:48:31.137 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T01:48:31.138 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:total 7777 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 . 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 .. 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T01:48:31.139 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 01:48 1259 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T01:48:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T01:48:31.141 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T01:48:31.142 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T01:48:31.143 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T01:48:31.144 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T01:48:31.145 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 01:48 2619 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2619_fsm 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T01:48:31.146 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T01:48:31.147 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T01:48:31.148 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2696 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T01:48:31.149 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T01:48:31.150 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T01:48:31.151 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T01:48:31.152 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T01:48:31.153 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T01:48:31.154 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T01:48:31.155 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T01:48:31.156 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T01:48:31.157 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T01:48:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T01:48:31.159 INFO:teuthology.orchestra.run.smithi029.stdout:total 537 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 01:48 . 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1213 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1213_fsm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1213_vm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1214 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1214_fsm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1214_vm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 1232 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 1233 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1260 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1260_fsm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1260_vm 2024-09-21T01:48:31.160 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1261 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1261_fsm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1261_vm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1262 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1262_fsm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1262_vm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2396 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2396_fsm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2396_vm 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2397 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2671 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2672 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2676 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2677 2024-09-21T01:48:31.161 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2694 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2695 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2697 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2698 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2846 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2847 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2964 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2965 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2966 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2967 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3592 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3593 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4060 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4061 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4175 2024-09-21T01:48:31.162 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4176 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4177 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4178 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4181 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4182 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4183 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4184 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4185 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4186 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6000 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6001 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6002 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6100 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6114 2024-09-21T01:48:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6115 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 pg_control 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T01:48:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:total 1 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 . 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 mappings 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 01:48 replorigin_checkpoint 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 snapshots 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.165 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 .. 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 .. 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 . 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 members 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 offsets 2024-09-21T01:48:31.166 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 .. 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 0000 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 .. 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 0000 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T01:48:31.167 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T01:48:31.168 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.169 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 0000 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.170 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:total 16384 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 . 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:48 000000010000000000000001 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 archive_status 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 .. 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T01:48:31.171 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T01:48:31.172 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 01:48 .. 2024-09-21T01:48:31.172 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 0000 2024-09-21T01:48:31.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-21T01:48:31.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:dropping old tables... 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-21T01:48:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:creating tables... 2024-09-21T01:48:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:generating data (client-side)... 2024-09-21T01:48:31.446 INFO:teuthology.orchestra.run.smithi029.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.07 s, remaining 2.05 s) 2024-09-21T01:48:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.25 s, remaining 3.77 s) 2024-09-21T01:48:31.728 INFO:teuthology.orchestra.run.smithi029.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.35 s, remaining 3.37 s) 2024-09-21T01:48:31.876 INFO:teuthology.orchestra.run.smithi029.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.49 s, remaining 3.42 s) 2024-09-21T01:48:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:31 smithi190 ceph-mon[34368]: pgmap v365: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-09-21T01:48:32.048 INFO:teuthology.orchestra.run.smithi029.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.67 s, remaining 3.61 s) 2024-09-21T01:48:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:31 smithi066 ceph-mon[35001]: pgmap v365: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-09-21T01:48:32.065 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:31 smithi029 ceph-mon[31228]: pgmap v365: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-09-21T01:48:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.84 s, remaining 3.64 s) 2024-09-21T01:48:32.293 INFO:teuthology.orchestra.run.smithi029.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.91 s, remaining 3.26 s) 2024-09-21T01:48:32.498 INFO:teuthology.orchestra.run.smithi029.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.12 s, remaining 3.35 s) 2024-09-21T01:48:32.670 INFO:teuthology.orchestra.run.smithi029.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.29 s, remaining 3.30 s) 2024-09-21T01:48:32.737 INFO:teuthology.orchestra.run.smithi029.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.36 s, remaining 2.99 s) 2024-09-21T01:48:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.54 s, remaining 2.94 s) 2024-09-21T01:48:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.61 s, remaining 2.68 s) 2024-09-21T01:48:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.79 s, remaining 2.62 s) 2024-09-21T01:48:33.372 INFO:teuthology.orchestra.run.smithi029.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.99 s, remaining 2.56 s) 2024-09-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.05 s, remaining 2.33 s) 2024-09-21T01:48:33.609 INFO:teuthology.orchestra.run.smithi029.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.23 s, remaining 2.23 s) 2024-09-21T01:48:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.39 s, remaining 2.11 s) 2024-09-21T01:48:33.855 INFO:teuthology.orchestra.run.smithi029.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.48 s, remaining 1.93 s) 2024-09-21T01:48:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:33 smithi190 ceph-mon[34368]: pgmap v366: 97 pgs: 97 active+clean; 31 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-09-21T01:48:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.66 s, remaining 1.82 s) 2024-09-21T01:48:34.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:33 smithi066 ceph-mon[35001]: pgmap v366: 97 pgs: 97 active+clean; 31 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-09-21T01:48:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:33 smithi029 ceph-mon[31228]: pgmap v366: 97 pgs: 97 active+clean; 31 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-09-21T01:48:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.85 s, remaining 1.71 s) 2024-09-21T01:48:34.291 INFO:teuthology.orchestra.run.smithi029.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.91 s, remaining 1.53 s) 2024-09-21T01:48:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.12 s, remaining 1.42 s) 2024-09-21T01:48:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.20 s, remaining 1.25 s) 2024-09-21T01:48:34.755 INFO:teuthology.orchestra.run.smithi029.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.38 s, remaining 1.13 s) 2024-09-21T01:48:34.965 INFO:teuthology.orchestra.run.smithi029.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.59 s, remaining 1.00 s) 2024-09-21T01:48:35.026 INFO:teuthology.orchestra.run.smithi029.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.65 s, remaining 0.84 s) 2024-09-21T01:48:35.201 INFO:teuthology.orchestra.run.smithi029.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.82 s, remaining 0.71 s) 2024-09-21T01:48:35.306 INFO:teuthology.orchestra.run.smithi029.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.93 s, remaining 0.56 s) 2024-09-21T01:48:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.08 s, remaining 0.42 s) 2024-09-21T01:48:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.26 s, remaining 0.28 s) 2024-09-21T01:48:35.736 INFO:teuthology.orchestra.run.smithi029.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.36 s, remaining 0.14 s) 2024-09-21T01:48:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.51 s, remaining 0.00 s) 2024-09-21T01:48:35.896 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:35 smithi029 ceph-mon[31228]: pgmap v367: 97 pgs: 97 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 8.1 MiB/s wr, 112 op/s 2024-09-21T01:48:35.896 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:48:35.950 INFO:teuthology.orchestra.run.smithi029.stderr:vacuuming... 2024-09-21T01:48:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:35 smithi190 ceph-mon[34368]: pgmap v367: 97 pgs: 97 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 8.1 MiB/s wr, 112 op/s 2024-09-21T01:48:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:48:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:35 smithi066 ceph-mon[35001]: pgmap v367: 97 pgs: 97 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 8.1 MiB/s wr, 112 op/s 2024-09-21T01:48:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:48:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 -- 172.21.15.29:0/863437146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780101a90 msgr2=0x7f8780101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 --2- 172.21.15.29:0/863437146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780101a90 0x7f8780101ea0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f877c009900 tx=0x7f877c02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 -- 172.21.15.29:0/863437146 shutdown_connections 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 --2- 172.21.15.29:0/863437146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780109480 0x7f878010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 --2- 172.21.15.29:0/863437146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780106b20 0x7f8780108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 --2- 172.21.15.29:0/863437146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780101a90 0x7f8780101ea0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 -- 172.21.15.29:0/863437146 >> 172.21.15.29:0/863437146 conn(0x7f87800fd690 msgr2=0x7f87800ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 -- 172.21.15.29:0/863437146 shutdown_connections 2024-09-21T01:48:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.216+0000 7f8788638640 1 -- 172.21.15.29:0/863437146 wait complete. 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 Processor -- start 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 -- start start 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 0x7f8780199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780109480 0x7f878019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f878010f2f0 con 0x7f8780106b20 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f878010f170 con 0x7f8780101a90 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8788638640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f878010f470 con 0x7f8780109480 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f87863d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 0x7f8780199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42000/0 (socket says 172.21.15.29:42000) 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.217+0000 7f8785bd3640 1 -- 172.21.15.29:0/2882860920 learned_addr learned my addr 172.21.15.29:0/2882860920 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f8786bd5640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780109480 0x7f878019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 -- 172.21.15.29:0/2882860920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780109480 msgr2=0x7f878019b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780109480 0x7f878019b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 -- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 msgr2=0x7f878019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 -- 172.21.15.29:0/2882860920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878019b980 con 0x7f8780101a90 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87863d4640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 0x7f8780199f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f877c006010 tx=0x7f877c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87777fe640 1 -- 172.21.15.29:0/2882860920 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c03f070 con 0x7f8780101a90 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87777fe640 1 -- 172.21.15.29:0/2882860920 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877c031910 con 0x7f8780101a90 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878019bc10 con 0x7f8780101a90 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f87777fe640 1 -- 172.21.15.29:0/2882860920 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035070 con 0x7f8780101a90 2024-09-21T01:48:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.218+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878019bf20 con 0x7f8780101a90 2024-09-21T01:48:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.222+0000 7f878f687740 1 Processor -- start 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.222+0000 7f878f687740 1 -- start start 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x5648816218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56488158f6f0 con 0x5648812751b0 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56488157eb20 con 0x56488124c2f0 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f878f687740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564881582930 con 0x5648812544d0 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f87777fe640 1 -- 172.21.15.29:0/2882860920 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f877c0037f0 con 0x7f8780101a90 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8786bd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38390/0 (socket says 172.21.15.29:38390) 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 -- 172.21.15.29:0/473249764 learned_addr learned my addr 172.21.15.29:0/473249764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8786bd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42002/0 (socket says 172.21.15.29:42002) 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f87863d4640 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x5648816218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 -- 172.21.15.29:0/473249764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 msgr2=0x5648816218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x5648816218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 -- 172.21.15.29:0/473249764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 msgr2=0x564881621260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f8785bd3640 1 -- 172.21.15.29:0/473249764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564881544530 con 0x5648812544d0 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.223+0000 7f87863d4640 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x5648816218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f8785bd3640 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8770011650 tx=0x7f8770009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486a3442477231f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f8774bfe640 1 -- 172.21.15.29:0/473249764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877001e070 con 0x5648812544d0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f87777fe640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 0x7f876c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f8774bfe640 1 -- 172.21.15.29:0/473249764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877000fb70 con 0x5648812544d0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f8774bfe640 1 -- 172.21.15.29:0/473249764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770019c00 con 0x5648812544d0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f8786bd5640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 0x7f876c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.224+0000 7f87777fe640 1 -- 172.21.15.29:0/2882860920 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f877c107870 con 0x7f8780101a90 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f8786bd5640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 0x7f876c0b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8778005fd0 tx=0x7f8778009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 -- 172.21.15.29:0/473249764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 msgr2=0x564881622420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8770011650 tx=0x7f8770009f90 comp rx=0 tx=0).stop 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 -- 172.21.15.29:0/473249764 shutdown_connections 2024-09-21T01:48:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881622420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x5648816218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 --2- 172.21.15.29:0/473249764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 -- 172.21.15.29:0/473249764 >> 172.21.15.29:0/473249764 conn(0x564881269300 msgr2=0x5648816188e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 -- 172.21.15.29:0/473249764 shutdown_connections 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.225+0000 7f878f687740 1 -- 172.21.15.29:0/473249764 wait complete. 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.226+0000 7f878f687740 1 -- start start 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.226+0000 7f878f687740 1 Processor -- start 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.226+0000 7f878f687740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.226+0000 7f878f687740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812544d0 0x56488162b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.226+0000 7f878f687740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812751b0 0x56488162bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f878f687740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5648814b6ab0 con 0x56488124c2f0 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f878f687740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564881524730 con 0x5648812544d0 2024-09-21T01:48:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f878f687740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564881622f40 con 0x5648812751b0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f878f687740 10 client.0 _ll_register_callbacks cb 0x5648815786b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42018/0 (socket says 172.21.15.29:42018) 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812544d0 0x56488162b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 -- 172.21.15.29:0/3495924768 learned_addr learned my addr 172.21.15.29:0/3495924768 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f8786bd5640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812751b0 0x56488162bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 -- 172.21.15.29:0/3495924768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812751b0 msgr2=0x56488162bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812751b0 0x56488162bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 -- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812544d0 msgr2=0x56488162b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812544d0 0x56488162b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 -- 172.21.15.29:0/3495924768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648812e0790 con 0x56488124c2f0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f87863d4640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cacd0 tx=0x7f877c03b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec57c4483ad35960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f8769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.227+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c048070 con 0x56488124c2f0 2024-09-21T01:48:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f877c003ce0 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877000fb70 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f877c0c91d0 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 10 client.15495 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564881359620 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877000fb70 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f877c0cece0 con 0x56488124c2f0 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 20 client.15495 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564881233d80 0x564881638020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 4 client.15495 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.228+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564881591640 con 0x564881233d80 2024-09-21T01:48:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.229+0000 7f8785bd3640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564881233d80 0x564881638020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.229+0000 7f8785bd3640 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564881233d80 0x564881638020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f332e39b575799c server_cookie=436ee045bad1b05b in_seq=0 out_seq=0 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.229+0000 7f8769ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f8769ffb640 1 -- 172.21.15.29:0/3495924768 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564881591640 con 0x564881233d80 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f8769ffb640 10 client.15495 handle_command_reply: tid=0 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 client.15495 shutdown 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 msgr2=0x56488162a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cacd0 tx=0x7f877c03b690 comp rx=0 tx=0).stop 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 shutdown_connections 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564881233d80 0x564881638020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812751b0 0x56488162bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812544d0 0x56488162b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 --2- 172.21.15.29:0/3495924768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56488124c2f0 0x56488162a9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 >> 172.21.15.29:0/3495924768 conn(0x564881269300 msgr2=0x564881616ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f8769ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 shutdown_connections 2024-09-21T01:48:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.230+0000 7f878f687740 1 -- 172.21.15.29:0/3495924768 wait complete. 2024-09-21T01:48:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.231+0000 7f878f687740 20 client.15495 trim_cache size 0 max 16384 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 Processor -- start 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 -- start start 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x564881617610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87801a8d40 con 0x5648812751b0 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87801aa880 con 0x56488124c2f0 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f878f687740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87801a9410 con 0x5648812544d0 2024-09-21T01:48:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8786bd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38418/0 (socket says 172.21.15.29:38418) 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 -- 172.21.15.29:0/2492379955 learned_addr learned my addr 172.21.15.29:0/2492379955 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f87863d4640 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x564881617610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 -- 172.21.15.29:0/2492379955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 msgr2=0x564881617610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x564881617610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 -- 172.21.15.29:0/2492379955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 msgr2=0x564881621200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.246+0000 7f8785bd3640 1 -- 172.21.15.29:0/2492379955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564881628b50 con 0x5648812544d0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.247+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f877001c040 tx=0x7f877000fda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a937a48e6702d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.247+0000 7f876bfff640 1 -- 172.21.15.29:0/2492379955 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770029070 con 0x5648812544d0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.247+0000 7f876bfff640 1 -- 172.21.15.29:0/2492379955 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877000fb70 con 0x5648812544d0 2024-09-21T01:48:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.247+0000 7f876bfff640 1 -- 172.21.15.29:0/2492379955 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877001e040 con 0x5648812544d0 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 -- 172.21.15.29:0/2492379955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 msgr2=0x564881617b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f877001c040 tx=0x7f877000fda0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 -- 172.21.15.29:0/2492379955 shutdown_connections 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812751b0 0x564881621200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648812544d0 0x564881617b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 --2- 172.21.15.29:0/2492379955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56488124c2f0 0x564881617610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 -- 172.21.15.29:0/2492379955 >> 172.21.15.29:0/2492379955 conn(0x564881269300 msgr2=0x5648816135c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 -- 172.21.15.29:0/2492379955 shutdown_connections 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.248+0000 7f878f687740 1 -- 172.21.15.29:0/2492379955 wait complete. 2024-09-21T01:48:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 -- start start 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 Processor -- start 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56488124c2f0 0x56488162f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 0x564881630b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87801aa1b0 con 0x5648812544d0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87801a9ae0 con 0x5648812751b0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87801a6d00 con 0x56488124c2f0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f878f687740 10 client.0 _ll_register_callbacks cb 0x7f877c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.249+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42034/0 (socket says 172.21.15.29:42034) 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 -- 172.21.15.29:0/2902845800 learned_addr learned my addr 172.21.15.29:0/2902845800 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f87863d4640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56488124c2f0 0x56488162f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8786bd5640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 0x564881630b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 -- 172.21.15.29:0/2902845800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56488124c2f0 msgr2=0x56488162f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56488124c2f0 0x56488162f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 -- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 msgr2=0x564881630b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 0x564881630b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 -- 172.21.15.29:0/2902845800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564881631280 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8786bd5640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 0x564881630b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8785bd3640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8770009250 tx=0x7f8770005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a34907af9211116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770031070 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f877000f880 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877000fb70 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.250+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f877003d050 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564881632570 con 0x5648812544d0 2024-09-21T01:48:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877000fb70 con 0x5648812544d0 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8770019530 con 0x5648812544d0 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 20 client.15501 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5648812091c0 0x5648816a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 4 client.15501 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56488158d0f0 con 0x5648812091c0 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.251+0000 7f87863d4640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5648812091c0 0x5648816a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.252+0000 7f87863d4640 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5648812091c0 0x5648816a49d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94c3689eff42355 server_cookie=7dbc9261d76c3f8c in_seq=0 out_seq=0 2024-09-21T01:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.252+0000 7f8769ffb640 10 client.15501 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f8769ffb640 1 -- 172.21.15.29:0/2902845800 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56488158d0f0 con 0x5648812091c0 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f8769ffb640 10 client.15501 handle_command_reply: tid=0 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 client.15501 shutdown 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 msgr2=0x56488162ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8770009250 tx=0x7f8770005ce0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 shutdown_connections 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5648812091c0 0x5648816a49d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648812751b0 0x564881630b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648812544d0 0x56488162ffb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f8769ffb640 0 client.15501 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 --2- 172.21.15.29:0/2902845800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56488124c2f0 0x56488162f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 >> 172.21.15.29:0/2902845800 conn(0x564881269300 msgr2=0x564881621200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 shutdown_connections 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.465+0000 7f878f687740 1 -- 172.21.15.29:0/2902845800 wait complete. 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f878f687740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T01:48:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:48:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:48:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:48:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:48:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 msgr2=0x7f876c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 0x7f876c0b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8778005fd0 tx=0x7f8778009500 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 msgr2=0x7f8780199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 0x7f8780199f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f877c006010 tx=0x7f877c004270 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 shutdown_connections 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f876c0b6590 0x7f876c0b8aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8780109480 0x7f878019b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8780106b20 0x7f878019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 --2- 172.21.15.29:0/2882860920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8780101a90 0x7f8780199f40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 >> 172.21.15.29:0/2882860920 conn(0x7f87800fd690 msgr2=0x7f87801079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 shutdown_connections 2024-09-21T01:48:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:36.466+0000 7f8788638640 1 -- 172.21.15.29:0/2882860920 wait complete. 2024-09-21T01:48:36.614 INFO:teuthology.orchestra.run.smithi029.stderr:creating primary keys... 2024-09-21T01:48:37.907 INFO:teuthology.orchestra.run.smithi029.stderr:done in 6.56 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.59 s, vacuum 0.66 s, primary keys 1.29 s). 2024-09-21T01:48:37.907 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:37 smithi029 ceph-mon[31228]: pgmap v368: 97 pgs: 97 active+clean; 276 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 42 MiB/s wr, 139 op/s 2024-09-21T01:48:37.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-21T01:48:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:starting vacuum...end. 2024-09-21T01:48:37.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:37 smithi190 ceph-mon[34368]: pgmap v368: 97 pgs: 97 active+clean; 276 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 42 MiB/s wr, 139 op/s 2024-09-21T01:48:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:37 smithi066 ceph-mon[35001]: pgmap v368: 97 pgs: 97 active+clean; 276 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 42 MiB/s wr, 139 op/s 2024-09-21T01:48:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:39 smithi190 ceph-mon[34368]: pgmap v369: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 43 MiB/s wr, 142 op/s 2024-09-21T01:48:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:39 smithi066 ceph-mon[35001]: pgmap v369: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 43 MiB/s wr, 142 op/s 2024-09-21T01:48:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:39 smithi029 ceph-mon[31228]: pgmap v369: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 43 MiB/s wr, 142 op/s 2024-09-21T01:48:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:41 smithi190 ceph-mon[34368]: pgmap v370: 97 pgs: 97 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 70 MiB/s wr, 360 op/s 2024-09-21T01:48:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:41 smithi066 ceph-mon[35001]: pgmap v370: 97 pgs: 97 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 70 MiB/s wr, 360 op/s 2024-09-21T01:48:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:41 smithi029 ceph-mon[31228]: pgmap v370: 97 pgs: 97 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 70 MiB/s wr, 360 op/s 2024-09-21T01:48:42.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 5.0 s, 5893.7 tps, lat 16.641 ms stddev 17.718 2024-09-21T01:48:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:43 smithi190 ceph-mon[34368]: pgmap v371: 97 pgs: 97 active+clean; 625 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 75 MiB/s wr, 385 op/s 2024-09-21T01:48:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:43 smithi066 ceph-mon[35001]: pgmap v371: 97 pgs: 97 active+clean; 625 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 75 MiB/s wr, 385 op/s 2024-09-21T01:48:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:43 smithi029 ceph-mon[31228]: pgmap v371: 97 pgs: 97 active+clean; 625 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 75 MiB/s wr, 385 op/s 2024-09-21T01:48:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:45 smithi190 ceph-mon[34368]: pgmap v372: 97 pgs: 97 active+clean; 750 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 75 MiB/s wr, 498 op/s 2024-09-21T01:48:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:45 smithi066 ceph-mon[35001]: pgmap v372: 97 pgs: 97 active+clean; 750 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 75 MiB/s wr, 498 op/s 2024-09-21T01:48:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:45 smithi029 ceph-mon[31228]: pgmap v372: 97 pgs: 97 active+clean; 750 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 75 MiB/s wr, 498 op/s 2024-09-21T01:48:47.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:47 smithi029 ceph-mon[31228]: pgmap v373: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 MiB/s wr, 732 op/s 2024-09-21T01:48:47.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 10.0 s, 6122.8 tps, lat 16.381 ms stddev 16.725 2024-09-21T01:48:47.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:47 smithi190 ceph-mon[34368]: pgmap v373: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 MiB/s wr, 732 op/s 2024-09-21T01:48:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:47 smithi066 ceph-mon[35001]: pgmap v373: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 MiB/s wr, 732 op/s 2024-09-21T01:48:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 -- 172.21.15.29:0/4137229399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810a3c0 msgr2=0x7f97b810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 --2- 172.21.15.29:0/4137229399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810a3c0 0x7f97b810dd70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f97a800d910 tx=0x7f97a802c380 comp rx=0 tx=0).stop 2024-09-21T01:48:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 -- 172.21.15.29:0/4137229399 shutdown_connections 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 --2- 172.21.15.29:0/4137229399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b8113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 --2- 172.21.15.29:0/4137229399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b810e2b0 0x7f97b81106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 --2- 172.21.15.29:0/4137229399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810a3c0 0x7f97b810dd70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 -- 172.21.15.29:0/4137229399 >> 172.21.15.29:0/4137229399 conn(0x7f97b8070c70 msgr2=0x7f97b8073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 -- 172.21.15.29:0/4137229399 shutdown_connections 2024-09-21T01:48:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.628+0000 7f97bd733640 1 -- 172.21.15.29:0/4137229399 wait complete. 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.629+0000 7f97bd733640 1 Processor -- start 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.629+0000 7f97bd733640 1 -- start start 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.629+0000 7f97bd733640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b810a3c0 0x7f97b81a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97bd733640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810e2b0 0x7f97b81a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97bd733640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97bd733640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8115960 con 0x7f97b810e2b0 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97bd733640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81157e0 con 0x7f97b8110c10 2024-09-21T01:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97bd733640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8115ae0 con 0x7f97b810a3c0 2024-09-21T01:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44634/0 (socket says 172.21.15.29:44634) 2024-09-21T01:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 -- 172.21.15.29:0/4158545229 learned_addr learned my addr 172.21.15.29:0/4158545229 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b67fc640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810e2b0 0x7f97b81a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 -- 172.21.15.29:0/4158545229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b810a3c0 msgr2=0x7f97b81a05a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b810a3c0 0x7f97b81a05a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 -- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810e2b0 msgr2=0x7f97b81a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810e2b0 0x7f97b81a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.630+0000 7f97b77fe640 1 -- 172.21.15.29:0/4158545229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b81a1fe0 con 0x7f97b8110c10 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f97b77fe640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97b000eb40 tx=0x7f97b000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f979ffff640 1 -- 172.21.15.29:0/4158545229 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0004b20 con 0x7f97b8110c10 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b81a22d0 con 0x7f97b8110c10 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f979ffff640 1 -- 172.21.15.29:0/4158545229 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b0004cc0 con 0x7f97b8110c10 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f979ffff640 1 -- 172.21.15.29:0/4158545229 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0010720 con 0x7f97b8110c10 2024-09-21T01:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.631+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b81a89c0 con 0x7f97b8110c10 2024-09-21T01:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.635+0000 7f97c469d740 1 Processor -- start 2024-09-21T01:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.635+0000 7f979ffff640 1 -- 172.21.15.29:0/4158545229 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f97b00108c0 con 0x7f97b8110c10 2024-09-21T01:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.635+0000 7f97c469d740 1 -- start start 2024-09-21T01:48:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.636+0000 7f97c469d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bdce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97c469d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8bdd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97c469d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97c469d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560de8b4f6f0 con 0x560de880c2f0 2024-09-21T01:48:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97c469d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560de8b3eb20 con 0x560de88351b0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97c469d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560de8b42930 con 0x560de88144d0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54452/0 (socket says 172.21.15.29:54452) 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 -- 172.21.15.29:0/1022971786 learned_addr learned my addr 172.21.15.29:0/1022971786 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bdce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8bdd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 -- 172.21.15.29:0/1022971786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 msgr2=0x560de8bdce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.637+0000 7f97b67fc640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bdce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f97b67fc640 1 -- 172.21.15.29:0/1022971786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 msgr2=0x560de8bdd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f97b67fc640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8bdd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f97b67fc640 1 -- 172.21.15.29:0/1022971786 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de8b04530 con 0x560de88144d0 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8bdd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f979ffff640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 0x7f97a00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f97b67fc640 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0127d0 tx=0x7f97ac009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d6c655f594a82c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f979ffff640 1 -- 172.21.15.29:0/4158545229 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f97b00e7bd0 con 0x7f97b8110c10 2024-09-21T01:48:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f979d7fa640 1 -- 172.21.15.29:0/1022971786 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac012d50 con 0x560de88144d0 2024-09-21T01:48:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f979d7fa640 1 -- 172.21.15.29:0/1022971786 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0107d0 con 0x560de88144d0 2024-09-21T01:48:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.638+0000 7f979d7fa640 1 -- 172.21.15.29:0/1022971786 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0044e0 con 0x560de88144d0 2024-09-21T01:48:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.639+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 0x7f97a00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- 172.21.15.29:0/1022971786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 msgr2=0x560de8bde010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0127d0 tx=0x7f97ac009750 comp rx=0 tx=0).stop 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 0x7f97a00b8ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004910 tx=0x7f97a8004810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- 172.21.15.29:0/1022971786 shutdown_connections 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bde010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8bdd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 --2- 172.21.15.29:0/1022971786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bdce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- 172.21.15.29:0/1022971786 >> 172.21.15.29:0/1022971786 conn(0x560de8829300 msgr2=0x560de8bd88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- 172.21.15.29:0/1022971786 shutdown_connections 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- 172.21.15.29:0/1022971786 wait complete. 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.640+0000 7f97c469d740 1 -- start start 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 Processor -- start 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8be66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 0x560de8be6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560de8a76ab0 con 0x560de880c2f0 2024-09-21T01:48:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560de8ae4730 con 0x560de88144d0 2024-09-21T01:48:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560de8bdeb30 con 0x560de88351b0 2024-09-21T01:48:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97c469d740 10 client.0 _ll_register_callbacks cb 0x560de8b386b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54464/0 (socket says 172.21.15.29:54464) 2024-09-21T01:48:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.641+0000 7f97b77fe640 1 -- 172.21.15.29:0/3104720063 learned_addr learned my addr 172.21.15.29:0/3104720063 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b67fc640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 0x560de8be6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 -- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 msgr2=0x560de8be6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 0x560de8be6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 -- 172.21.15.29:0/3104720063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 msgr2=0x560de8be66c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8be66c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 -- 172.21.15.29:0/3104720063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de88a0790 con 0x560de88351b0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.642+0000 7f97b77fe640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97b00adca0 tx=0x7f97b00ad790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a581f691597228 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.643+0000 7f97b67fc640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 0x560de8be6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.643+0000 7f979aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.643+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0010db0 con 0x560de88351b0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.643+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac0107d0 con 0x560de88351b0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.644+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f97b00b5b70 con 0x560de88351b0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.644+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97b00ae410 con 0x560de88351b0 2024-09-21T01:48:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.644+0000 7f97c469d740 10 client.25213 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.644+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560de8919620 con 0x560de88351b0 2024-09-21T01:48:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.644+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0107d0 con 0x560de88351b0 2024-09-21T01:48:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97b0093b30 con 0x560de88351b0 2024-09-21T01:48:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 10 client.25213 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 20 client.25213 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87f3d80 0x560de8bf4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 4 client.25213 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560de8b51640 con 0x560de87f3d80 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.645+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87f3d80 0x560de8bf4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.646+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87f3d80 0x560de8bf4390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedb198de96f024 server_cookie=7c0cf85cfed9295f in_seq=0 out_seq=0 2024-09-21T01:48:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.646+0000 7f979aff5640 10 client.25213 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f979aff5640 1 -- 172.21.15.29:0/3104720063 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560de8b51640 con 0x560de87f3d80 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f979aff5640 10 client.25213 handle_command_reply: tid=0 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f97c469d740 1 client.25213 shutdown 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 msgr2=0x560de8be7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97b00adca0 tx=0x7f97b00ad790 comp rx=0 tx=0).stop 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.647+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 shutdown_connections 2024-09-21T01:48:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87f3d80 0x560de8bf4390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88351b0 0x560de8be7880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f979aff5640 0 client.25213 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88144d0 0x560de8be6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 --2- 172.21.15.29:0/3104720063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8be66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 >> 172.21.15.29:0/3104720063 conn(0x560de8829300 msgr2=0x560de8bd94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 shutdown_connections 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 1 -- 172.21.15.29:0/3104720063 wait complete. 2024-09-21T01:48:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.648+0000 7f97c469d740 20 client.25213 trim_cache size 0 max 16384 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 Processor -- start 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 -- start start 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 0x560de8be1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 0x560de8bee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8be2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81b16b0 con 0x560de88144d0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81b31f0 con 0x560de88351b0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97c469d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81b1d80 con 0x560de880c2f0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97b6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 0x560de8be1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97b67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 0x560de8bee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97b6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 0x560de8be1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54478/0 (socket says 172.21.15.29:54478) 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97b6ffd640 1 -- 172.21.15.29:0/990494615 learned_addr learned my addr 172.21.15.29:0/990494615 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.675+0000 7f97b77fe640 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8be2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 -- 172.21.15.29:0/990494615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 msgr2=0x560de8be1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 0x560de8be1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 -- 172.21.15.29:0/990494615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 msgr2=0x560de8be2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8be2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 -- 172.21.15.29:0/990494615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de8be7540 con 0x560de88144d0 2024-09-21T01:48:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.676+0000 7f97b67fc640 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 0x560de8bee760 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0127a0 tx=0x7f97ac005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b4c6b309efafa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.677+0000 7f979cff9640 1 -- 172.21.15.29:0/990494615 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0052d0 con 0x560de88144d0 2024-09-21T01:48:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.677+0000 7f979cff9640 1 -- 172.21.15.29:0/990494615 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac005470 con 0x560de88144d0 2024-09-21T01:48:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.677+0000 7f979cff9640 1 -- 172.21.15.29:0/990494615 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac003a90 con 0x560de88144d0 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 -- 172.21.15.29:0/990494615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 msgr2=0x560de8bee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 0x560de8bee760 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0127a0 tx=0x7f97ac005fb0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 -- 172.21.15.29:0/990494615 shutdown_connections 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8be2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de88144d0 0x560de8bee760 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 --2- 172.21.15.29:0/990494615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de880c2f0 0x560de8be1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.678+0000 7f97c469d740 1 -- 172.21.15.29:0/990494615 >> 172.21.15.29:0/990494615 conn(0x560de8829300 msgr2=0x560de8be0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.679+0000 7f97c469d740 1 -- 172.21.15.29:0/990494615 shutdown_connections 2024-09-21T01:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.679+0000 7f97c469d740 1 -- 172.21.15.29:0/990494615 wait complete. 2024-09-21T01:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.679+0000 7f97c469d740 1 -- start start 2024-09-21T01:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.679+0000 7f97c469d740 1 Processor -- start 2024-09-21T01:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8beb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97b67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97b67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54480/0 (socket says 172.21.15.29:54480) 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97b67fc640 1 -- 172.21.15.29:0/2455577058 learned_addr learned my addr 172.21.15.29:0/2455577058 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81b2b20 con 0x560de880c2f0 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81b2450 con 0x560de88351b0 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81af690 con 0x560de88144d0 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97c469d740 10 client.0 _ll_register_callbacks cb 0x7f97b0004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97b77fe640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.680+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8beb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 -- 172.21.15.29:0/2455577058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 msgr2=0x560de8bebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 -- 172.21.15.29:0/2455577058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 msgr2=0x560de8beb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8beb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de8bece90 con 0x560de88351b0 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8beb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b67fc640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97b77fe640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bec730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97b00d5450 tx=0x7f97b00d1370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223e9c46f011e64d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f979aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:48:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b00b27f0 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac005470 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f97b0010bb0 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97b00d5270 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97c469d740 10 client.25184 fetch_fsmap learned FSMap version 28 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.681+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560de8bee1d0 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac005470 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97b0072140 con 0x560de88351b0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 10 client.25184 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 20 client.25184 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87c91c0 0x560de8c60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 4 client.25184 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.682+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560de8b4d0f0 con 0x560de87c91c0 2024-09-21T01:48:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.683+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87c91c0 0x560de8c60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.684+0000 7f979aff5640 10 client.25184 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.684+0000 7f97b6ffd640 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87c91c0 0x560de8c60da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395d27013fe3c913 server_cookie=97deef45fae4638f in_seq=0 out_seq=0 2024-09-21T01:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.709+0000 7f979aff5640 1 -- 172.21.15.29:0/2455577058 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560de8b4d0f0 con 0x560de87c91c0 2024-09-21T01:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f979aff5640 10 client.25184 handle_command_reply: tid=0 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 client.25184 shutdown 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 msgr2=0x560de8bec730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bec730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97b00d5450 tx=0x7f97b00d1370 comp rx=0 tx=0).stop 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 shutdown_connections 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560de87c91c0 0x560de8c60da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de88351b0 0x560de8bec730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560de88144d0 0x560de8bebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 --2- 172.21.15.29:0/2455577058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560de880c2f0 0x560de8beb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 >> 172.21.15.29:0/2455577058 conn(0x560de8829300 msgr2=0x560de8be18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f979aff5640 0 client.25184 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 shutdown_connections 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 1 -- 172.21.15.29:0/2455577058 wait complete. 2024-09-21T01:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.710+0000 7f97c469d740 20 client.25184 trim_cache size 0 max 16384 2024-09-21T01:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.711+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 msgr2=0x7f97a00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.711+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 0x7f97a00b8ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004910 tx=0x7f97a8004810 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.711+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 msgr2=0x7f97b81a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.711+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97b000eb40 tx=0x7f97b000c2d0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 shutdown_connections 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97a00b65d0 0x7f97a00b8ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b8110c10 0x7f97b81a17c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97b810e2b0 0x7f97b81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 --2- 172.21.15.29:0/4158545229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b810a3c0 0x7f97b81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 >> 172.21.15.29:0/4158545229 conn(0x7f97b8070c70 msgr2=0x7f97b810c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.713+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 shutdown_connections 2024-09-21T01:48:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:48:49.714+0000 7f97bd733640 1 -- 172.21.15.29:0/4158545229 wait complete. 2024-09-21T01:48:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:49 smithi190 ceph-mon[34368]: pgmap v374: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 703 op/s 2024-09-21T01:48:50.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:49 smithi066 ceph-mon[35001]: pgmap v374: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 703 op/s 2024-09-21T01:48:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:49 smithi029 ceph-mon[31228]: pgmap v374: 97 pgs: 97 active+clean; 915 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 703 op/s 2024-09-21T01:48:51.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:51 smithi190 ceph-mon[34368]: pgmap v375: 97 pgs: 97 active+clean; 931 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 43 MiB/s wr, 961 op/s 2024-09-21T01:48:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:51 smithi066 ceph-mon[35001]: pgmap v375: 97 pgs: 97 active+clean; 931 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 43 MiB/s wr, 961 op/s 2024-09-21T01:48:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:51 smithi029 ceph-mon[31228]: pgmap v375: 97 pgs: 97 active+clean; 931 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 43 MiB/s wr, 961 op/s 2024-09-21T01:48:52.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 15.0 s, 5630.3 tps, lat 17.763 ms stddev 19.086 2024-09-21T01:48:53.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:53 smithi190 ceph-mon[34368]: pgmap v376: 97 pgs: 97 active+clean; 946 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T01:48:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:53 smithi066 ceph-mon[35001]: pgmap v376: 97 pgs: 97 active+clean; 946 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T01:48:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:53 smithi029 ceph-mon[31228]: pgmap v376: 97 pgs: 97 active+clean; 946 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T01:48:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:55 smithi190 ceph-mon[34368]: pgmap v377: 97 pgs: 97 active+clean; 947 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 930 op/s 2024-09-21T01:48:56.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:55 smithi066 ceph-mon[35001]: pgmap v377: 97 pgs: 97 active+clean; 947 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 930 op/s 2024-09-21T01:48:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:55 smithi029 ceph-mon[31228]: pgmap v377: 97 pgs: 97 active+clean; 947 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 930 op/s 2024-09-21T01:48:56.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:56 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:57.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:56 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:57.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:56 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:48:57.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:48:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 20.0 s, 5850.1 tps, lat 17.040 ms stddev 17.409 2024-09-21T01:48:57.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:57 smithi190 ceph-mon[34368]: pgmap v378: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-21T01:48:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:57 smithi066 ceph-mon[35001]: pgmap v378: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-21T01:48:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:57 smithi029 ceph-mon[31228]: pgmap v378: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-21T01:48:58.263 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.893+0000 7fe8f93cb640 1 -- 172.21.15.29:0/371598770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 msgr2=0x7fe8f410ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.893+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/371598770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f410ff00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009900 tx=0x7fe8ec02a410 comp rx=0 tx=0).stop 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 -- 172.21.15.29:0/371598770 shutdown_connections 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/371598770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8f4110440 0x7fe8f4116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/371598770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f410ff00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/371598770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f410a1b0 0x7fe8f410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 -- 172.21.15.29:0/371598770 >> 172.21.15.29:0/371598770 conn(0x7fe8f40780e0 msgr2=0x7fe8f41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 -- 172.21.15.29:0/371598770 shutdown_connections 2024-09-21T01:48:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.894+0000 7fe8f93cb640 1 -- 172.21.15.29:0/371598770 wait complete. 2024-09-21T01:48:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.895+0000 7fe8f93cb640 1 Processor -- start 2024-09-21T01:48:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.895+0000 7fe8f93cb640 1 -- start start 2024-09-21T01:48:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8f410a1b0 0x7fe8f41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f41a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 0x7fe8f41a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f411a120 con 0x7fe8f410faf0 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4119fa0 con 0x7fe8f410a1b0 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f93cb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f411a2a0 con 0x7fe8f4110440 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f41a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f41a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43828/0 (socket says 172.21.15.29:43828) 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.896+0000 7fe8f27fc640 1 -- 172.21.15.29:0/3864388481 learned_addr learned my addr 172.21.15.29:0/3864388481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:58.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 0x7fe8f41a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3864388481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8f410a1b0 msgr2=0x7fe8f41a0b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8f410a1b0 0x7fe8f41a0b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 msgr2=0x7fe8f41a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f41a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.897+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3864388481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f41a6b00 con 0x7fe8f4110440 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 0x7fe8f41a63d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009fd0 tx=0x7fe8e400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8ebfff640 1 -- 172.21.15.29:0/3864388481 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400ce20 con 0x7fe8f4110440 2024-09-21T01:48:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8ebfff640 1 -- 172.21.15.29:0/3864388481 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e4005020 con 0x7fe8f4110440 2024-09-21T01:48:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8ebfff640 1 -- 172.21.15.29:0/3864388481 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4010690 con 0x7fe8f4110440 2024-09-21T01:48:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f41a6df0 con 0x7fe8f4110440 2024-09-21T01:48:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.898+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41ad550 con 0x7fe8f4110440 2024-09-21T01:48:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.904+0000 7fe8ebfff640 1 -- 172.21.15.29:0/3864388481 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe8e4010830 con 0x7fe8f4110440 2024-09-21T01:48:58.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.905+0000 7fe8ff93f740 1 Processor -- start 2024-09-21T01:48:58.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 -- start start 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 0x555b6f2ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 0x555b6f2ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 0x555b6f2accf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b6f222e50 con 0x555b6eee4390 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b6f212280 con 0x555b6ef05070 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.906+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b6f216090 con 0x555b6eedc1b0 2024-09-21T01:48:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 0x555b6f2ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 0x555b6f2ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50666/0 (socket says 172.21.15.29:50666) 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f27fc640 1 -- 172.21.15.29:0/3793629449 learned_addr learned my addr 172.21.15.29:0/3793629449 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 0x555b6f2accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 0x555b6f2ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f2ffd640 1 -- 172.21.15.29:0/3793629449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 msgr2=0x555b6f2ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.907+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 0x555b6f2ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8f2ffd640 1 -- 172.21.15.29:0/3793629449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 msgr2=0x555b6f2accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 0x555b6f2accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8f2ffd640 1 -- 172.21.15.29:0/3793629449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6ef70650 con 0x555b6eedc1b0 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8ebfff640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 0x7fe8e00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 0x555b6f2accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:48:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.908+0000 7fe8ebfff640 1 -- 172.21.15.29:0/3864388481 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e40e7790 con 0x7fe8f4110440 2024-09-21T01:48:58.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.909+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 0x555b6f2ac7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009ad0 tx=0x7fe8dc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be42ad539dfae6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.909+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 0x7fe8e00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.909+0000 7fe8e97fa640 1 -- 172.21.15.29:0/3793629449 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00ed10 con 0x555b6eedc1b0 2024-09-21T01:48:58.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.909+0000 7fe8e97fa640 1 -- 172.21.15.29:0/3793629449 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc005020 con 0x555b6eedc1b0 2024-09-21T01:48:58.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.909+0000 7fe8e97fa640 1 -- 172.21.15.29:0/3793629449 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc004a00 con 0x555b6eedc1b0 2024-09-21T01:48:58.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3793629449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 msgr2=0x555b6f2ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 0x555b6f2ac7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009ad0 tx=0x7fe8dc009f90 comp rx=0 tx=0).stop 2024-09-21T01:48:58.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 0x7fe8e00b8b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005f70 tx=0x7fe8ec005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3793629449 shutdown_connections 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eee4390 0x555b6f2accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eedc1b0 0x555b6f2ac7b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3793629449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6ef05070 0x555b6f2ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3793629449 >> 172.21.15.29:0/3793629449 conn(0x555b6eef9460 msgr2=0x555b6f2aa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3793629449 shutdown_connections 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.911+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3793629449 wait complete. 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.912+0000 7fe8ff93f740 1 -- start start 2024-09-21T01:48:58.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.912+0000 7fe8ff93f740 1 Processor -- start 2024-09-21T01:48:58.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b6f073700 con 0x555b6eedc1b0 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b6f1ad110 con 0x555b6eee4390 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b6f2baa50 con 0x555b6ef05070 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43856/0 (socket says 172.21.15.29:43856) 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8f2ffd640 1 -- 172.21.15.29:0/3840887953 learned_addr learned my addr 172.21.15.29:0/3840887953 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:58.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.913+0000 7fe8ff93f740 10 client.0 _ll_register_callbacks cb 0x555b6f1c6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:58.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 msgr2=0x555b6f2b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3840887953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 msgr2=0x555b6f2b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 -- 172.21.15.29:0/3840887953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6f1dac40 con 0x555b6ef05070 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8f37fe640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2c3da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c760 tx=0x7fe8e40ae330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7fa0f25644114a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.914+0000 7fe8da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40afd50 con 0x555b6ef05070 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc005020 con 0x555b6ef05070 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e40ada80 con 0x555b6ef05070 2024-09-21T01:48:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8e40b3300 con 0x555b6ef05070 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8ff93f740 10 client.25231 fetch_fsmap learned FSMap version 29 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.915+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b6f2c67a0 con 0x555b6ef05070 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc005020 con 0x555b6ef05070 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe8e40ad4f0 con 0x555b6ef05070 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 10 client.25231 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 20 client.25231 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6eec39f0 0x555b6f2ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 4 client.25231 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:48:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b6f224da0 con 0x555b6eec39f0 2024-09-21T01:48:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.916+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6eec39f0 0x555b6f2ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.917+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6eec39f0 0x555b6f2ca0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2153eacd9446fc server_cookie=d509d88e1eacae0f in_seq=0 out_seq=0 2024-09-21T01:48:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.917+0000 7fe8da7fc640 10 client.25231 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8da7fc640 1 -- 172.21.15.29:0/3840887953 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b6f224da0 con 0x555b6eec39f0 2024-09-21T01:48:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8da7fc640 10 client.25231 handle_command_reply: tid=0 2024-09-21T01:48:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 client.25231 shutdown 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 msgr2=0x555b6f2c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2c3da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c760 tx=0x7fe8e40ae330 comp rx=0 tx=0).stop 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 shutdown_connections 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6eec39f0 0x555b6f2ca0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2c3da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/3840887953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 >> 172.21.15.29:0/3840887953 conn(0x555b6eef9460 msgr2=0x555b6f2af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.919+0000 7fe8da7fc640 0 client.25231 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.920+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 shutdown_connections 2024-09-21T01:48:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.920+0000 7fe8ff93f740 1 -- 172.21.15.29:0/3840887953 wait complete. 2024-09-21T01:48:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.920+0000 7fe8ff93f740 20 client.25231 trim_cache size 0 max 16384 2024-09-21T01:48:58.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.956+0000 7fe8ff93f740 1 Processor -- start 2024-09-21T01:48:58.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.956+0000 7fe8ff93f740 1 -- start start 2024-09-21T01:48:58.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b5b00 con 0x555b6eedc1b0 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b7670 con 0x555b6eee4390 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.957+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b61d0 con 0x555b6ef05070 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50684/0 (socket says 172.21.15.29:50684) 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54290/0 (socket says 172.21.15.29:54290) 2024-09-21T01:48:58.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 -- 172.21.15.29:0/2093304088 learned_addr learned my addr 172.21.15.29:0/2093304088 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 -- 172.21.15.29:0/2093304088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 msgr2=0x555b6f2b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 -- 172.21.15.29:0/2093304088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 msgr2=0x555b6f2b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.958+0000 7fe8f27fc640 1 -- 172.21.15.29:0/2093304088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6f2b6890 con 0x555b6eee4390 2024-09-21T01:48:58.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.959+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec02a930 tx=0x7fe8ec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01d31d258f2e6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.959+0000 7fe8e8ff9640 1 -- 172.21.15.29:0/2093304088 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec003a80 con 0x555b6eee4390 2024-09-21T01:48:58.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.959+0000 7fe8e8ff9640 1 -- 172.21.15.29:0/2093304088 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ec003c20 con 0x555b6eee4390 2024-09-21T01:48:58.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.959+0000 7fe8e8ff9640 1 -- 172.21.15.29:0/2093304088 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec03d690 con 0x555b6eee4390 2024-09-21T01:48:58.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.961+0000 7fe8ff93f740 1 -- 172.21.15.29:0/2093304088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 msgr2=0x555b6f2c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.961+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec02a930 tx=0x7fe8ec009260 comp rx=0 tx=0).stop 2024-09-21T01:48:58.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 -- 172.21.15.29:0/2093304088 shutdown_connections 2024-09-21T01:48:58.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6ef05070 0x555b6f2b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eee4390 0x555b6f2c3ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/2093304088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6eedc1b0 0x555b6f2b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 -- 172.21.15.29:0/2093304088 >> 172.21.15.29:0/2093304088 conn(0x555b6eef9460 msgr2=0x555b6f2b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 -- 172.21.15.29:0/2093304088 shutdown_connections 2024-09-21T01:48:58.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.962+0000 7fe8ff93f740 1 -- 172.21.15.29:0/2093304088 wait complete. 2024-09-21T01:48:58.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.963+0000 7fe8ff93f740 1 -- start start 2024-09-21T01:48:58.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.963+0000 7fe8ff93f740 1 Processor -- start 2024-09-21T01:48:58.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eedc1b0 0x555b6f2bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 0x555b6f2bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 0x555b6f2bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b6f70 con 0x555b6ef05070 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b68a0 con 0x555b6eedc1b0 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b3ae0 con 0x555b6eee4390 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 0x555b6f2bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 0x555b6f2bc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43876/0 (socket says 172.21.15.29:43876) 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8f37fe640 1 -- 172.21.15.29:0/664360506 learned_addr learned my addr 172.21.15.29:0/664360506 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 0x555b6f2bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8ff93f740 10 client.0 _ll_register_callbacks cb 0x7fe8e4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.964+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eedc1b0 0x555b6f2bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 -- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eedc1b0 msgr2=0x555b6f2bbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eedc1b0 0x555b6f2bbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 -- 172.21.15.29:0/664360506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 msgr2=0x555b6f2bc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 0x555b6f2bc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 -- 172.21.15.29:0/664360506 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6f333c90 con 0x555b6eee4390 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8f27fc640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 0x555b6f2bc3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005ab0 tx=0x7fe8ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf150c3ba17bf20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:48:58.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.965+0000 7fe8da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:48:58.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.967+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec060060 con 0x555b6eee4390 2024-09-21T01:48:58.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.967+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ec003c20 con 0x555b6eee4390 2024-09-21T01:48:58.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.967+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe8ec0317b0 con 0x555b6eee4390 2024-09-21T01:48:58.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.967+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8ec0319b0 con 0x555b6eee4390 2024-09-21T01:48:58.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.968+0000 7fe8ff93f740 10 client.25237 fetch_fsmap learned FSMap version 29 2024-09-21T01:48:58.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.968+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b6f334f00 con 0x555b6eee4390 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.968+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ec03d900 con 0x555b6eee4390 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe8ec03c8b0 con 0x555b6eee4390 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 10 client.25237 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 20 client.25237 populate_metadata read hostname 'smithi029' 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6ee98da0 0x555b6f3375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:48:58.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 4 client.25237 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe8ec003c20 con 0x555b6ee98da0 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.969+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6ee98da0 0x555b6f3375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.970+0000 7fe8da7fc640 10 client.25237 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.970+0000 7fe8f2ffd640 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6ee98da0 0x555b6f3375c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db3432bb1338d5 server_cookie=84e96ec85f2cb1b0 in_seq=0 out_seq=0 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.970+0000 7fe8da7fc640 1 -- 172.21.15.29:0/664360506 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7fe8ec003c20 con 0x555b6ee98da0 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.970+0000 7fe8da7fc640 10 client.25237 handle_command_reply: tid=0 2024-09-21T01:48:58.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.971+0000 7fe8ff93f740 1 client.25237 shutdown 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.974+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 msgr2=0x555b6f2bc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.974+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 0x555b6f2bc3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005ab0 tx=0x7fe8ec000bc0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 shutdown_connections 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b6ee98da0 0x555b6f3375c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b6ef05070 0x555b6f2bc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6eee4390 0x555b6f2bc3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 --2- 172.21.15.29:0/664360506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b6eedc1b0 0x555b6f2bbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8da7fc640 0 client.25237 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 >> 172.21.15.29:0/664360506 conn(0x555b6eef9460 msgr2=0x555b6f2ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 shutdown_connections 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 1 -- 172.21.15.29:0/664360506 wait complete. 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.975+0000 7fe8ff93f740 20 client.25237 trim_cache size 0 max 16384 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:48:58.976 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:48:58.977 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.976+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 msgr2=0x7fe8e00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.976+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 0x7fe8e00b8b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005f70 tx=0x7fe8ec005a20 comp rx=0 tx=0).stop 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.976+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 msgr2=0x7fe8f41a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.976+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 0x7fe8f41a63d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009fd0 tx=0x7fe8e400ef10 comp rx=0 tx=0).stop 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 shutdown_connections 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe8e00b6610 0x7fe8e00b8b20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f4110440 0x7fe8f41a63d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f410faf0 0x7fe8f41a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 --2- 172.21.15.29:0/3864388481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8f410a1b0 0x7fe8f41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:48:58.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 >> 172.21.15.29:0/3864388481 conn(0x7fe8f40780e0 msgr2=0x7fe8f4108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:48:58.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 shutdown_connections 2024-09-21T01:48:58.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:48:58.977+0000 7fe8f93cb640 1 -- 172.21.15.29:0/3864388481 wait complete. 2024-09-21T01:48:59.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:48:59 smithi029 ceph-mon[31228]: pgmap v379: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-21T01:48:59.977 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:48:59.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:48:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:48:59 smithi190 ceph-mon[34368]: pgmap v379: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-21T01:49:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:48:59 smithi066 ceph-mon[35001]: pgmap v379: 97 pgs: 97 active+clean; 964 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-21T01:49:00.489 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.141+0000 7fb29d1f2640 1 -- 172.21.15.29:0/1897585550 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290035580 con 0x7fb2981083a0 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.142+0000 7fb2a0458640 1 -- 172.21.15.29:0/1897585550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2981083a0 msgr2=0x7fb2981087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.142+0000 7fb2a0458640 1 --2- 172.21.15.29:0/1897585550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2981083a0 0x7fb2981087b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb29002ed00 comp rx=0 tx=0).stop 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.143+0000 7fb2a0458640 1 -- 172.21.15.29:0/1897585550 shutdown_connections 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.143+0000 7fb2a0458640 1 --2- 172.21.15.29:0/1897585550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298108cf0 0x7fb29810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.143+0000 7fb2a0458640 1 --2- 172.21.15.29:0/1897585550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2981083a0 0x7fb2981087b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.143+0000 7fb2a0458640 1 --2- 172.21.15.29:0/1897585550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb298102a60 0x7fb298102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.143+0000 7fb2a0458640 1 -- 172.21.15.29:0/1897585550 >> 172.21.15.29:0/1897585550 conn(0x7fb298079730 msgr2=0x7fb298079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.145+0000 7fb2a0458640 1 -- 172.21.15.29:0/1897585550 shutdown_connections 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.145+0000 7fb2a0458640 1 -- 172.21.15.29:0/1897585550 wait complete. 2024-09-21T01:49:01.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 Processor -- start 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 -- start start 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 0x7fb29819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 0x7fb298198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb298108cf0 0x7fb2981987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.147+0000 7fb2a0458640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111890 con 0x7fb298108cf0 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb2a0458640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111710 con 0x7fb2981083a0 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb2a0458640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111a10 con 0x7fb298102a60 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29e1f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 0x7fb29819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29e1f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 0x7fb29819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54310/0 (socket says 172.21.15.29:54310) 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29e1f4640 1 -- 172.21.15.29:0/3363624042 learned_addr learned my addr 172.21.15.29:0/3363624042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 0x7fb298198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 -- 172.21.15.29:0/3363624042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 msgr2=0x7fb29819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:01.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 0x7fb29819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 -- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb298108cf0 msgr2=0x7fb2981987f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb298108cf0 0x7fb2981987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.148+0000 7fb29d9f3640 1 -- 172.21.15.29:0/3363624042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819db30 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.149+0000 7fb29d9f3640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 0x7fb298198190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb29003a040 tx=0x7fb29002ff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.150+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29003c070 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.150+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819ddc0 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.150+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29819e210 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.151+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2900105a0 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.151+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29000e6b0 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.152+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c0052c0 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.157+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb290047060 con 0x7fb2981083a0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.158+0000 7fb28f7fe640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 0x7fb2840b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.159+0000 7fb29e1f4640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 0x7fb2840b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.159+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb290107330 con 0x7fb2981083a0 2024-09-21T01:49:01.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.159+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb290107790 con 0x7fb2981083a0 2024-09-21T01:49:01.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.160+0000 7fb29e1f4640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 0x7fb2840b8b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb294009a40 tx=0x7fb2940091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.603+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb27c0055b0 con 0x7fb2981083a0 2024-09-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.604+0000 7fb28f7fe640 1 -- 172.21.15.29:0/3363624042 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7fb2900cdbf0 con 0x7fb2981083a0 2024-09-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":30,"btime":"2024-09-21T01:49:00:701789+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:49:00.229419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 30 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.608+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 msgr2=0x7fb2840b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.608+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 0x7fb2840b8b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb294009a40 tx=0x7fb2940091c0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.608+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 msgr2=0x7fb298198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.608+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 0x7fb298198190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb29003a040 tx=0x7fb29002ff30 comp rx=0 tx=0).stop 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 shutdown_connections 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2840b6660 0x7fb2840b8b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb298108cf0 0x7fb2981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2981083a0 0x7fb298198190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 --2- 172.21.15.29:0/3363624042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb298102a60 0x7fb29819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 >> 172.21.15.29:0/3363624042 conn(0x7fb298079730 msgr2=0x7fb298105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 shutdown_connections 2024-09-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:01.609+0000 7fb2a0458640 1 -- 172.21.15.29:0/3363624042 wait complete. 2024-09-21T01:49:01.985 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:01 smithi029 ceph-mon[31228]: pgmap v380: 97 pgs: 97 active+clean; 971 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T01:49:01.985 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:01 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:01.985 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:01 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3363624042' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:01 smithi066 ceph-mon[35001]: pgmap v380: 97 pgs: 97 active+clean; 971 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T01:49:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:01 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:01 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3363624042' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:01 smithi190 ceph-mon[34368]: pgmap v380: 97 pgs: 97 active+clean; 971 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T01:49:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:01 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:01 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3363624042' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:02.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:02 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 -- 172.21.15.29:0/3977280040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c095cd0 msgr2=0x7fb84c0960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3977280040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c095cd0 0x7fb84c0960e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb844009860 tx=0x7fb84402a340 comp rx=0 tx=0).stop 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 -- 172.21.15.29:0/3977280040 shutdown_connections 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3977280040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c096670 0x7fb84c09d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3977280040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c095cd0 0x7fb84c0960e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3977280040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c0950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 -- 172.21.15.29:0/3977280040 >> 172.21.15.29:0/3977280040 conn(0x7fb84c090590 msgr2=0x7fb84c092a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 -- 172.21.15.29:0/3977280040 shutdown_connections 2024-09-21T01:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.879+0000 7fb85aaee640 1 -- 172.21.15.29:0/3977280040 wait complete. 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.880+0000 7fb85aaee640 1 Processor -- start 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.880+0000 7fb85aaee640 1 -- start start 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c095cd0 0x7fb84c12ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c096670 0x7fb84c12fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c09e420 con 0x7fb84c095cd0 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c09e2a0 con 0x7fb84c094c50 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85aaee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c09e5a0 con 0x7fb84c096670 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:34802/0 (socket says 172.21.15.29:34802) 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 -- 172.21.15.29:0/3281401023 learned_addr learned my addr 172.21.15.29:0/3281401023 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb8592eb640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c095cd0 0x7fb84c12ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c096670 0x7fb84c12fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 -- 172.21.15.29:0/3281401023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c096670 msgr2=0x7fb84c12fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c096670 0x7fb84c12fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 -- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c095cd0 msgr2=0x7fb84c12ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.881+0000 7fb859aec640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c095cd0 0x7fb84c12ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.882+0000 7fb859aec640 1 -- 172.21.15.29:0/3281401023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c130250 con 0x7fb84c094c50 2024-09-21T01:49:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.882+0000 7fb859aec640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb85000e9f0 tx=0x7fb85000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.884+0000 7fb84affd640 1 -- 172.21.15.29:0/3281401023 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000cd80 con 0x7fb84c094c50 2024-09-21T01:49:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.884+0000 7fb84affd640 1 -- 172.21.15.29:0/3281401023 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500108e0 con 0x7fb84c094c50 2024-09-21T01:49:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.884+0000 7fb84affd640 1 -- 172.21.15.29:0/3281401023 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850018980 con 0x7fb84c094c50 2024-09-21T01:49:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.884+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c130540 con 0x7fb84c094c50 2024-09-21T01:49:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.884+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c0a1300 con 0x7fb84c094c50 2024-09-21T01:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.888+0000 7fb84affd640 1 -- 172.21.15.29:0/3281401023 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb850018b20 con 0x7fb84c094c50 2024-09-21T01:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.893+0000 7fb862cbc740 1 Processor -- start 2024-09-21T01:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.893+0000 7fb862cbc740 1 -- start start 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930ad16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930ad2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562930a436f0 con 0x5629307084d0 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562930a32b20 con 0x5629307002f0 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb862cbc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562930a36930 con 0x5629307291b0 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.894+0000 7fb85a2ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb84affd640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 0x7fb8400b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39298/0 (socket says 172.21.15.29:39298) 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 -- 172.21.15.29:0/44238974 learned_addr learned my addr 172.21.15.29:0/44238974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb859aec640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930ad16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb8592eb640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930ad2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 -- 172.21.15.29:0/44238974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 msgr2=0x562930ad16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930ad16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 -- 172.21.15.29:0/44238974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 msgr2=0x562930ad2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930ad2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 -- 172.21.15.29:0/44238974 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629309f8530 con 0x5629307291b0 2024-09-21T01:49:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb859aec640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930ad16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 0x7fb8400b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb84affd640 1 -- 172.21.15.29:0/3281401023 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb8500e74c0 con 0x7fb84c094c50 2024-09-21T01:49:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.895+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb854069330 tx=0x7fb854075d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8082d5141b8058 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.897+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 0x7fb8400b8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x562930ad29f0 tx=0x7fb854089000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.899+0000 7fb83ffff640 1 -- 172.21.15.29:0/44238974 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854093070 con 0x5629307291b0 2024-09-21T01:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.900+0000 7fb83ffff640 1 -- 172.21.15.29:0/44238974 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb854069810 con 0x5629307291b0 2024-09-21T01:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.900+0000 7fb83ffff640 1 -- 172.21.15.29:0/44238974 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85408b410 con 0x5629307291b0 2024-09-21T01:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 -- 172.21.15.29:0/44238974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 msgr2=0x562930ad1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb854069330 tx=0x7fb854075d90 comp rx=0 tx=0).stop 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 -- 172.21.15.29:0/44238974 shutdown_connections 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930ad2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930ad16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 --2- 172.21.15.29:0/44238974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad1040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 -- 172.21.15.29:0/44238974 >> 172.21.15.29:0/44238974 conn(0x56293071d300 msgr2=0x562930accad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 -- 172.21.15.29:0/44238974 shutdown_connections 2024-09-21T01:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.901+0000 7fb862cbc740 1 -- 172.21.15.29:0/44238974 wait complete. 2024-09-21T01:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.902+0000 7fb862cbc740 1 -- start start 2024-09-21T01:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.902+0000 7fb862cbc740 1 Processor -- start 2024-09-21T01:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 0x562930aeb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930aebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 0x562930aecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56293096aab0 con 0x5629307084d0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5629309d8730 con 0x5629307291b0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562930ad2d20 con 0x5629307002f0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb862cbc740 10 client.0 _ll_register_callbacks cb 0x562930a2c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb859aec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 0x562930aeb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930aebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb859aec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 0x562930aeb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39306/0 (socket says 172.21.15.29:39306) 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb859aec640 1 -- 172.21.15.29:0/1929287084 learned_addr learned my addr 172.21.15.29:0/1929287084 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 0x562930aecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 -- 172.21.15.29:0/1929287084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 msgr2=0x562930aeb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 0x562930aeb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 -- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 msgr2=0x562930aecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 0x562930aecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.903+0000 7fb8592eb640 1 -- 172.21.15.29:0/1929287084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562930794790 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.904+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 0x562930aecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.904+0000 7fb8592eb640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930aebf50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb844035040 tx=0x7fb844030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d47d6fab9c9026e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.904+0000 7fb83d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.904+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844042070 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.904+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb844030810 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.905+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562930a45640 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.905+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb84404c050 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.905+0000 7fb862cbc740 10 client.15543 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.905+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56293080d620 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562930a45640 con 0x5629307084d0 2024-09-21T01:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb844003bb0 con 0x5629307084d0 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 10 client.15543 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 20 client.15543 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306e7d80 0x562930af9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 4 client.15543 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562930a410f0 con 0x5629306e7d80 2024-09-21T01:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.906+0000 7fb859aec640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306e7d80 0x562930af9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.907+0000 7fb859aec640 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306e7d80 0x562930af9180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d614e501133d215f server_cookie=e835f0c939d2356d in_seq=0 out_seq=0 2024-09-21T01:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.907+0000 7fb83d7fa640 10 client.15543 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.908+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1929287084 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562930a410f0 con 0x5629306e7d80 2024-09-21T01:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.908+0000 7fb83d7fa640 10 client.15543 handle_command_reply: tid=0 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.908+0000 7fb862cbc740 1 client.15543 shutdown 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.908+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 msgr2=0x562930aebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.908+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930aebf50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb844035040 tx=0x7fb844030e50 comp rx=0 tx=0).stop 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 shutdown_connections 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306e7d80 0x562930af9180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307291b0 0x562930aecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307084d0 0x562930aebf50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1929287084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307002f0 0x562930aeb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 >> 172.21.15.29:0/1929287084 conn(0x56293071d300 msgr2=0x562930ad20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb83d7fa640 0 client.15543 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 shutdown_connections 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 1 -- 172.21.15.29:0/1929287084 wait complete. 2024-09-21T01:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.909+0000 7fb862cbc740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T01:49:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.939+0000 7fb862cbc740 1 Processor -- start 2024-09-21T01:49:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.939+0000 7fb862cbc740 1 -- start start 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307002f0 0x562930ad1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307084d0 0x562930aeaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800e550 con 0x5629307002f0 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb838010180 con 0x5629307084d0 2024-09-21T01:49:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb862cbc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800ec20 con 0x5629307291b0 2024-09-21T01:49:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb85a2ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb85a2ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39320/0 (socket says 172.21.15.29:39320) 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.940+0000 7fb85a2ed640 1 -- 172.21.15.29:0/432343207 learned_addr learned my addr 172.21.15.29:0/432343207 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 -- 172.21.15.29:0/432343207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307084d0 msgr2=0x562930aeaa90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307084d0 0x562930aeaa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 -- 172.21.15.29:0/432343207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307002f0 msgr2=0x562930ad1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307002f0 0x562930ad1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 -- 172.21.15.29:0/432343207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562930aece30 con 0x5629307291b0 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.941+0000 7fb85a2ed640 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb854066c70 tx=0x7fb854081000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994e902f84806507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.942+0000 7fb83f7fe640 1 -- 172.21.15.29:0/432343207 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854093030 con 0x5629307291b0 2024-09-21T01:49:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.942+0000 7fb83f7fe640 1 -- 172.21.15.29:0/432343207 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb854081e70 con 0x5629307291b0 2024-09-21T01:49:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.942+0000 7fb83f7fe640 1 -- 172.21.15.29:0/432343207 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8540989e0 con 0x5629307291b0 2024-09-21T01:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.943+0000 7fb862cbc740 1 -- 172.21.15.29:0/432343207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 msgr2=0x562930ad3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.943+0000 7fb862cbc740 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb854066c70 tx=0x7fb854081000 comp rx=0 tx=0).stop 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.945+0000 7fb862cbc740 1 -- 172.21.15.29:0/432343207 shutdown_connections 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.945+0000 7fb862cbc740 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307291b0 0x562930ad3120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.945+0000 7fb862cbc740 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307084d0 0x562930aeaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.945+0000 7fb862cbc740 1 --2- 172.21.15.29:0/432343207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307002f0 0x562930ad1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.945+0000 7fb862cbc740 1 -- 172.21.15.29:0/432343207 >> 172.21.15.29:0/432343207 conn(0x56293071d300 msgr2=0x562930aea1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.946+0000 7fb862cbc740 1 -- 172.21.15.29:0/432343207 shutdown_connections 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.946+0000 7fb862cbc740 1 -- 172.21.15.29:0/432343207 wait complete. 2024-09-21T01:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.946+0000 7fb862cbc740 1 -- start start 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.946+0000 7fb862cbc740 1 Processor -- start 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307084d0 0x562930af0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307291b0 0x562930af18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800f9f0 con 0x5629307291b0 2024-09-21T01:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800f320 con 0x5629307002f0 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800c530 con 0x5629307084d0 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.947+0000 7fb862cbc740 10 client.0 _ll_register_callbacks cb 0x7fb850005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:34824/0 (socket says 172.21.15.29:34824) 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 -- 172.21.15.29:0/1434552855 learned_addr learned my addr 172.21.15.29:0/1434552855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 -- 172.21.15.29:0/1434552855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307084d0 msgr2=0x562930af0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307084d0 0x562930af0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 -- 172.21.15.29:0/1434552855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307291b0 msgr2=0x562930af18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307291b0 0x562930af18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 -- 172.21.15.29:0/1434552855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562930af2020 con 0x5629307002f0 2024-09-21T01:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb859aec640 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c6a0 tx=0x7fb85000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987b8de84d5766a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.948+0000 7fb83d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850010630 con 0x5629307002f0 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb8500ac970 con 0x5629307002f0 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb854081e70 con 0x5629307002f0 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb8500b2140 con 0x5629307002f0 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb862cbc740 10 client.25217 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562930af3410 con 0x5629307002f0 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.949+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500b7080 con 0x5629307002f0 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb8500eaad0 con 0x5629307002f0 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 10 client.25217 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 10 client.25217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 20 client.25217 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306bd1c0 0x562930b65b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 4 client.25217 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.950+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb854081e70 con 0x5629306bd1c0 2024-09-21T01:49:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.953+0000 7fb8592eb640 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306bd1c0 0x562930b65b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.953+0000 7fb8592eb640 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306bd1c0 0x562930b65b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1d4e11896c3722 server_cookie=d697085050ed0560 in_seq=0 out_seq=0 2024-09-21T01:49:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.953+0000 7fb83d7fa640 10 client.25217 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:02.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 25.0 s, 4431.3 tps, lat 22.615 ms stddev 23.981 2024-09-21T01:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.968+0000 7fb83d7fa640 1 -- 172.21.15.29:0/1434552855 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb854081e70 con 0x5629306bd1c0 2024-09-21T01:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.968+0000 7fb83d7fa640 10 client.25217 handle_command_reply: tid=0 2024-09-21T01:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.968+0000 7fb862cbc740 1 client.25217 shutdown 2024-09-21T01:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 msgr2=0x562930af0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c6a0 tx=0x7fb85000cf20 comp rx=0 tx=0).stop 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 shutdown_connections 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5629306bd1c0 0x562930b65b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629307291b0 0x562930af18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb83d7fa640 0 client.25217 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629307084d0 0x562930af0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 --2- 172.21.15.29:0/1434552855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629307002f0 0x562930af0690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.970+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 >> 172.21.15.29:0/1434552855 conn(0x56293071d300 msgr2=0x562930aea1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.971+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 shutdown_connections 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.971+0000 7fb862cbc740 1 -- 172.21.15.29:0/1434552855 wait complete. 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.971+0000 7fb862cbc740 20 client.25217 trim_cache size 0 max 16384 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:49:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.974+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 msgr2=0x7fb8400b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.974+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 0x7fb8400b8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x562930ad29f0 tx=0x7fb854089000 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 msgr2=0x7fb84c12e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb85000e9f0 tx=0x7fb85000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 shutdown_connections 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb8400b6660 0x7fb8400b8b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c096670 0x7fb84c12fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb84c095cd0 0x7fb84c12ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 --2- 172.21.15.29:0/3281401023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb84c094c50 0x7fb84c12e810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 >> 172.21.15.29:0/3281401023 conn(0x7fb84c090590 msgr2=0x7fb84c092060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 shutdown_connections 2024-09-21T01:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:02.975+0000 7fb85aaee640 1 -- 172.21.15.29:0/3281401023 wait complete. 2024-09-21T01:49:03.009 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:49:03.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:02 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:03.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:02 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:49:03.471 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:03.915 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:03 smithi029 ceph-mon[31228]: pgmap v381: 97 pgs: 97 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T01:49:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:03 smithi066 ceph-mon[35001]: pgmap v381: 97 pgs: 97 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T01:49:04.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 -- 172.21.15.29:0/460274058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 msgr2=0x7fb774108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 --2- 172.21.15.29:0/460274058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb774108100 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009cd0 tx=0x7fb76c02a8f0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 -- 172.21.15.29:0/460274058 shutdown_connections 2024-09-21T01:49:04.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 --2- 172.21.15.29:0/460274058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb774108100 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 --2- 172.21.15.29:0/460274058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb774103130 0x7fb774107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 --2- 172.21.15.29:0/460274058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7741027e0 0x7fb774102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 -- 172.21.15.29:0/460274058 >> 172.21.15.29:0/460274058 conn(0x7fb774079750 msgr2=0x7fb774079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 -- 172.21.15.29:0/460274058 shutdown_connections 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.076+0000 7fb77a890640 1 -- 172.21.15.29:0/460274058 wait complete. 2024-09-21T01:49:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.077+0000 7fb77a890640 1 Processor -- start 2024-09-21T01:49:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 -- start start 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7741027e0 0x7fb7741a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 0x7fb7741a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb77411a120 con 0x7fb774107c20 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb774119fa0 con 0x7fb774103130 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.078+0000 7fb77a890640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb77411a2a0 con 0x7fb7741027e0 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56740/0 (socket says 172.21.15.29:56740) 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 -- 172.21.15.29:0/3584821008 learned_addr learned my addr 172.21.15.29:0/3584821008 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 0x7fb7741a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 -- 172.21.15.29:0/3584821008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7741027e0 msgr2=0x7fb7741a0870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7741027e0 0x7fb7741a0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 -- 172.21.15.29:0/3584821008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 msgr2=0x7fb7741a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 0x7fb7741a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb778e2d640 1 -- 172.21.15.29:0/3584821008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741a67e0 con 0x7fb774107c20 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.079+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 0x7fb7741a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.080+0000 7fb778e2d640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02ae10 tx=0x7fb76c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.081+0000 7fb771ffb640 1 -- 172.21.15.29:0/3584821008 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c004240 con 0x7fb774107c20 2024-09-21T01:49:04.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.081+0000 7fb771ffb640 1 -- 172.21.15.29:0/3584821008 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76c040070 con 0x7fb774107c20 2024-09-21T01:49:04.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.081+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741a6a70 con 0x7fb774107c20 2024-09-21T01:49:04.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.081+0000 7fb771ffb640 1 -- 172.21.15.29:0/3584821008 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c045c80 con 0x7fb774107c20 2024-09-21T01:49:04.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.082+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741ad230 con 0x7fb774107c20 2024-09-21T01:49:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.085+0000 7fb780e04740 1 Processor -- start 2024-09-21T01:49:04.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.087+0000 7fb771ffb640 1 -- 172.21.15.29:0/3584821008 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb76c045470 con 0x7fb774107c20 2024-09-21T01:49:04.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.088+0000 7fb771ffb640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 0x7fb7600b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.089+0000 7fb771ffb640 1 -- 172.21.15.29:0/3584821008 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb76c1083f0 con 0x7fb774107c20 2024-09-21T01:49:04.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.089+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 0x7fb7600b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.089+0000 7fb780e04740 1 -- start start 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.089+0000 7fb780e04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 0x560afa80a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb780e04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb780e04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa80b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb780e04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560afa780ea0 con 0x560afa442390 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb780e04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560afa7702d0 con 0x560afa463070 2024-09-21T01:49:04.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb780e04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560afa7740e0 con 0x560afa43a1b0 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb76b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb76b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39336/0 (socket says 172.21.15.29:39336) 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb76b7fe640 1 -- 172.21.15.29:0/1129414086 learned_addr learned my addr 172.21.15.29:0/1129414086 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.090+0000 7fb778e2d640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa80b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 -- 172.21.15.29:0/1129414086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 msgr2=0x560afa80a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb773fff640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 0x560afa80a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 0x560afa80a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 -- 172.21.15.29:0/1129414086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 msgr2=0x560afa80b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa80b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 -- 172.21.15.29:0/1129414086 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afa4ce650 con 0x560afa43a1b0 2024-09-21T01:49:04.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb778e2d640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa80b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:04.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb773fff640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 0x560afa80a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:04.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 0x7fb7600b8bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb75c012cf0 tx=0x7fb75c00bf00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.091+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005cf0 tx=0x7fb75c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94bc9e53038ca11a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.092+0000 7fb76a7fc640 1 -- 172.21.15.29:0/1129414086 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c025070 con 0x560afa43a1b0 2024-09-21T01:49:04.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.093+0000 7fb76a7fc640 1 -- 172.21.15.29:0/1129414086 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb75c012030 con 0x560afa43a1b0 2024-09-21T01:49:04.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.093+0000 7fb76a7fc640 1 -- 172.21.15.29:0/1129414086 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c010b50 con 0x560afa43a1b0 2024-09-21T01:49:04.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 -- 172.21.15.29:0/1129414086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 msgr2=0x560afa80abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005cf0 tx=0x7fb75c01c000 comp rx=0 tx=0).stop 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 -- 172.21.15.29:0/1129414086 shutdown_connections 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa80b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa80abe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 --2- 172.21.15.29:0/1129414086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa463070 0x560afa80a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 -- 172.21.15.29:0/1129414086 >> 172.21.15.29:0/1129414086 conn(0x560afa457460 msgr2=0x560afa808590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 -- 172.21.15.29:0/1129414086 shutdown_connections 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.094+0000 7fb780e04740 1 -- 172.21.15.29:0/1129414086 wait complete. 2024-09-21T01:49:04.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.095+0000 7fb780e04740 1 -- start start 2024-09-21T01:49:04.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.095+0000 7fb780e04740 1 Processor -- start 2024-09-21T01:49:04.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.096+0000 7fb780e04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.096+0000 7fb780e04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa442390 0x560afa816300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.096+0000 7fb780e04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 0x560afa822140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.096+0000 7fb773fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.096+0000 7fb773fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56748/0 (socket says 172.21.15.29:56748) 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb780e04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560afa5d1700 con 0x560afa43a1b0 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 0x560afa822140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 0x560afa822140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39350/0 (socket says 172.21.15.29:39350) 2024-09-21T01:49:04.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb773fff640 1 -- 172.21.15.29:0/3886960519 learned_addr learned my addr 172.21.15.29:0/3886960519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:04.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560afa70b160 con 0x560afa442390 2024-09-21T01:49:04.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560afa818e20 con 0x560afa463070 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.097+0000 7fb780e04740 10 client.0 _ll_register_callbacks cb 0x560afa7249e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 -- 172.21.15.29:0/3886960519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 msgr2=0x560afa822140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 0x560afa822140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 -- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa442390 msgr2=0x560afa816300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa442390 0x560afa816300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afa738c90 con 0x560afa43a1b0 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb773fff640 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb764010620 tx=0x7fb764010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a2726b0d828c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb7537fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:49:04.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.098+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75c012030 con 0x560afa43a1b0 2024-09-21T01:49:04.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.099+0000 7fb7537fe640 1 -- 172.21.15.29:0/3886960519 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764005220 con 0x560afa43a1b0 2024-09-21T01:49:04.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.099+0000 7fb7537fe640 1 -- 172.21.15.29:0/3886960519 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb7640053c0 con 0x560afa43a1b0 2024-09-21T01:49:04.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.099+0000 7fb7537fe640 1 -- 172.21.15.29:0/3886960519 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb764005600 con 0x560afa43a1b0 2024-09-21T01:49:04.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.100+0000 7fb780e04740 10 client.15555 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:04.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.100+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=31}) -- 0x560afa824ad0 con 0x560afa43a1b0 2024-09-21T01:49:04.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb7537fe640 1 -- 172.21.15.29:0/3886960519 <== mon.0 v2:172.21.15.29:3300/0 4 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb76400f040 con 0x560afa43a1b0 2024-09-21T01:49:04.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 10 client.15555 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:04.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 10 client.15555 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:49:04.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 20 client.15555 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:04.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa4219f0 0x560afa827e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 4 client.15555 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:04.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.101+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb75c012030 con 0x560afa4219f0 2024-09-21T01:49:04.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.102+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa4219f0 0x560afa827e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.104+0000 7fb7537fe640 10 client.15555 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:04.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.104+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa4219f0 0x560afa827e90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e7cba952389d73c server_cookie=94ee0572d999d08d in_seq=0 out_seq=0 2024-09-21T01:49:04.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.105+0000 7fb7537fe640 1 -- 172.21.15.29:0/3886960519 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb75c012030 con 0x560afa4219f0 2024-09-21T01:49:04.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.105+0000 7fb7537fe640 10 client.15555 handle_command_reply: tid=0 2024-09-21T01:49:04.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.106+0000 7fb780e04740 1 client.15555 shutdown 2024-09-21T01:49:04.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.106+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 msgr2=0x560afa815dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.106+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb764010620 tx=0x7fb764010a20 comp rx=0 tx=0).stop 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 shutdown_connections 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa4219f0 0x560afa827e90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa463070 0x560afa822140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa442390 0x560afa816300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 --2- 172.21.15.29:0/3886960519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa43a1b0 0x560afa815dc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 >> 172.21.15.29:0/3886960519 conn(0x560afa457460 msgr2=0x560afa80b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb7537fe640 0 client.15555 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 shutdown_connections 2024-09-21T01:49:04.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.108+0000 7fb780e04740 1 -- 172.21.15.29:0/3886960519 wait complete. 2024-09-21T01:49:04.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.110+0000 7fb780e04740 20 client.15555 trim_cache size 0 max 16384 2024-09-21T01:49:04.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.138+0000 7fb780e04740 1 Processor -- start 2024-09-21T01:49:04.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.138+0000 7fb780e04740 1 -- start start 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.139+0000 7fb780e04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa457460 0x560afa817ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.139+0000 7fb780e04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 0x560afa807bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.139+0000 7fb780e04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb780e04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b5920 con 0x560afa457460 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb780e04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b7550 con 0x560afa43a1b0 2024-09-21T01:49:04.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb780e04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b5ff0 con 0x560afa442390 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb76b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 0x560afa807bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39360/0 (socket says 172.21.15.29:39360) 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 -- 172.21.15.29:0/2113743981 learned_addr learned my addr 172.21.15.29:0/2113743981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb773fff640 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa457460 0x560afa817ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 -- 172.21.15.29:0/2113743981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 msgr2=0x560afa807bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 0x560afa807bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 -- 172.21.15.29:0/2113743981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa457460 msgr2=0x560afa817ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa457460 0x560afa817ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.140+0000 7fb778e2d640 1 -- 172.21.15.29:0/2113743981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afa80c990 con 0x560afa442390 2024-09-21T01:49:04.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.141+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 0x560afa807bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:04.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.142+0000 7fb778e2d640 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb76c105ea0 tx=0x7fb76c02a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b074838c21016ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.142+0000 7fb769ffb640 1 -- 172.21.15.29:0/2113743981 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0cec30 con 0x560afa442390 2024-09-21T01:49:04.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.142+0000 7fb769ffb640 1 -- 172.21.15.29:0/2113743981 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76c036050 con 0x560afa442390 2024-09-21T01:49:04.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.142+0000 7fb769ffb640 1 -- 172.21.15.29:0/2113743981 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0cd790 con 0x560afa442390 2024-09-21T01:49:04.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 -- 172.21.15.29:0/2113743981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 msgr2=0x560afa808100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb76c105ea0 tx=0x7fb76c02a8f0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 -- 172.21.15.29:0/2113743981 shutdown_connections 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa442390 0x560afa808100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa43a1b0 0x560afa807bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 --2- 172.21.15.29:0/2113743981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa457460 0x560afa817ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 -- 172.21.15.29:0/2113743981 >> 172.21.15.29:0/2113743981 conn(0x560afa816480 msgr2=0x560afa80bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.144+0000 7fb780e04740 1 -- 172.21.15.29:0/2113743981 shutdown_connections 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.145+0000 7fb780e04740 1 -- 172.21.15.29:0/2113743981 wait complete. 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.145+0000 7fb780e04740 1 -- start start 2024-09-21T01:49:04.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.146+0000 7fb780e04740 1 Processor -- start 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.146+0000 7fb780e04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.146+0000 7fb780e04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa81b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39370/0 (socket says 172.21.15.29:39370) 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 -- 172.21.15.29:0/2674184028 learned_addr learned my addr 172.21.15.29:0/2674184028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa457460 0x560afa88b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b6dc0 con 0x560afa442390 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b66f0 con 0x560afa457460 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b3900 con 0x560afa43a1b0 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb778e2d640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa457460 0x560afa88b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb780e04740 10 client.0 _ll_register_callbacks cb 0x7fb764009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:04.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 -- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa457460 msgr2=0x560afa88b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa457460 0x560afa88b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 -- 172.21.15.29:0/2674184028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 msgr2=0x560afa81b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:04.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa81b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.147+0000 7fb773fff640 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afa88bf70 con 0x560afa43a1b0 2024-09-21T01:49:04.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.148+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c036050 con 0x560afa43a1b0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.149+0000 7fb773fff640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb764010620 tx=0x7fb764004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31795d65e7e0e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.149+0000 7fb7537fe640 10 client.25273 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.150+0000 7fb7537fe640 1 -- 172.21.15.29:0/2674184028 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764058060 con 0x560afa43a1b0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.150+0000 7fb7537fe640 1 -- 172.21.15.29:0/2674184028 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb764012b30 con 0x560afa43a1b0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.150+0000 7fb7537fe640 1 -- 172.21.15.29:0/2674184028 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb76402b040 con 0x560afa43a1b0 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.150+0000 7fb780e04740 10 client.25273 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:04.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.151+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=31}) -- 0x560afa892ae0 con 0x560afa43a1b0 2024-09-21T01:49:04.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb7537fe640 1 -- 172.21.15.29:0/2674184028 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb764004c20 con 0x560afa43a1b0 2024-09-21T01:49:04.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 10 client.25273 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:04.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 10 client.25273 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:49:04.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 20 client.25273 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:04.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa463070 0x560afa8951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:04.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 4 client.25273 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:04.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.152+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560afa782df0 con 0x560afa463070 2024-09-21T01:49:04.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.153+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa463070 0x560afa8951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:04.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.156+0000 7fb76b7fe640 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa463070 0x560afa8951c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0f83c4ecea33f8 server_cookie=f1abf8ecdc1b058a in_seq=0 out_seq=0 2024-09-21T01:49:04.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.157+0000 7fb7537fe640 10 client.25273 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:04.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.157+0000 7fb7537fe640 1 -- 172.21.15.29:0/2674184028 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560afa782df0 con 0x560afa463070 2024-09-21T01:49:04.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.157+0000 7fb7537fe640 10 client.25273 handle_command_reply: tid=0 2024-09-21T01:49:04.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.157+0000 7fb780e04740 1 client.25273 shutdown 2024-09-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.161+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 msgr2=0x560afa81ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.161+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb764010620 tx=0x7fb764004730 comp rx=0 tx=0).stop 2024-09-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.161+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 shutdown_connections 2024-09-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x560afa463070 0x560afa8951c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560afa457460 0x560afa88b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560afa442390 0x560afa81b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 --2- 172.21.15.29:0/2674184028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa43a1b0 0x560afa81ab10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 >> 172.21.15.29:0/2674184028 conn(0x560afa816480 msgr2=0x560afa80bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb7537fe640 0 client.25273 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 shutdown_connections 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 1 -- 172.21.15.29:0/2674184028 wait complete. 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.162+0000 7fb780e04740 20 client.25273 trim_cache size 0 max 16384 2024-09-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:04.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.165+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 msgr2=0x7fb7600b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 0x7fb7600b8bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb75c012cf0 tx=0x7fb75c00bf00 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 msgr2=0x7fb7741a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02ae10 tx=0x7fb76c003ba0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 shutdown_connections 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb7600b66e0 0x7fb7600b8bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb774107c20 0x7fb7741a60b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb774103130 0x7fb7741a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 --2- 172.21.15.29:0/3584821008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7741027e0 0x7fb7741a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 >> 172.21.15.29:0/3584821008 conn(0x7fb774079750 msgr2=0x7fb774105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 shutdown_connections 2024-09-21T01:49:04.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:04.166+0000 7fb77a890640 1 -- 172.21.15.29:0/3584821008 wait complete. 2024-09-21T01:49:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:03 smithi190 ceph-mon[34368]: pgmap v381: 97 pgs: 97 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T01:49:05.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:49:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:05 smithi066 ceph-mon[35001]: pgmap v382: 97 pgs: 97 active+clean; 985 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-21T01:49:06.072 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:05 smithi029 ceph-mon[31228]: pgmap v382: 97 pgs: 97 active+clean; 985 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-21T01:49:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:05 smithi190 ceph-mon[34368]: pgmap v382: 97 pgs: 97 active+clean; 985 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 -- 172.21.15.29:0/3695481457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81030f0 msgr2=0x7f33f8107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 --2- 172.21.15.29:0/3695481457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81030f0 0x7f33f8107620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009930 tx=0x7f33e802a420 comp rx=0 tx=0).stop 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 -- 172.21.15.29:0/3695481457 shutdown_connections 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 --2- 172.21.15.29:0/3695481457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 0x7f33f81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 --2- 172.21.15.29:0/3695481457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81030f0 0x7f33f8107620 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 --2- 172.21.15.29:0/3695481457 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81027a0 0x7f33f8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.223+0000 7f33ff47b640 1 -- 172.21.15.29:0/3695481457 >> 172.21.15.29:0/3695481457 conn(0x7f33f8079730 msgr2=0x7f33f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.224+0000 7f33ff47b640 1 -- 172.21.15.29:0/3695481457 shutdown_connections 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.224+0000 7f33ff47b640 1 -- 172.21.15.29:0/3695481457 wait complete. 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.225+0000 7f33ff47b640 1 Processor -- start 2024-09-21T01:49:06.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 -- start start 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81030f0 0x7f33f81a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 0x7f33f81a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33f811a060 con 0x7f33f81027a0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8119ee0 con 0x7f33f81030f0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33ff47b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f811a1e0 con 0x7f33f8107be0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33fd217640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33fd217640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56788/0 (socket says 172.21.15.29:56788) 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33fd217640 1 -- 172.21.15.29:0/1916780931 learned_addr learned my addr 172.21.15.29:0/1916780931 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33fca16640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81030f0 0x7f33f81a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.226+0000 7f33fda18640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 0x7f33f81a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 -- 172.21.15.29:0/1916780931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 msgr2=0x7f33f81a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 0x7f33f81a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 -- 172.21.15.29:0/1916780931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81030f0 msgr2=0x7f33f81a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81030f0 0x7f33f81a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 -- 172.21.15.29:0/1916780931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f81a6a60 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.227+0000 7f33fd217640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d770 tx=0x7f33f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.228+0000 7f33ee7fc640 1 -- 172.21.15.29:0/1916780931 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4017070 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.228+0000 7f33ee7fc640 1 -- 172.21.15.29:0/1916780931 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f4009c30 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.228+0000 7f33ee7fc640 1 -- 172.21.15.29:0/1916780931 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4017070 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.228+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f81a6d50 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.228+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f81ad4b0 con 0x7f33f81027a0 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.232+0000 7f34059ef740 1 Processor -- start 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.232+0000 7f34059ef740 1 -- start start 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.232+0000 7f34059ef740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.232+0000 7f34059ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.232+0000 7f34059ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f34059ef740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5597a1169ea0 con 0x5597a0e4c070 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f34059ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5597a11592d0 con 0x5597a0e231b0 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f34059ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5597a115d0e0 con 0x5597a0e2b390 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33ee7fc640 1 -- 172.21.15.29:0/1916780931 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f33f4003a10 con 0x7f33f81027a0 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39400/0 (socket says 172.21.15.29:39400) 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 -- 172.21.15.29:0/582921729 learned_addr learned my addr 172.21.15.29:0/582921729 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fd217640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 -- 172.21.15.29:0/582921729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 msgr2=0x5597a11f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fda18640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 -- 172.21.15.29:0/582921729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 msgr2=0x5597a11f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fca16640 1 -- 172.21.15.29:0/582921729 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a0eb7650 con 0x5597a0e2b390 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.233+0000 7f33fd217640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.234+0000 7f33fda18640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.234+0000 7f33ee7fc640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 0x7f33e40b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.234+0000 7f33fda18640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 0x7f33e40b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.234+0000 7f33ee7fc640 1 -- 172.21.15.29:0/1916780931 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f33f4009a30 con 0x7f33f81027a0 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.235+0000 7f33fca16640 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009230 tx=0x7f33e80314b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222c6f611dd77b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.235+0000 7f33e37fe640 1 -- 172.21.15.29:0/582921729 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8040070 con 0x5597a0e2b390 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.235+0000 7f33e37fe640 1 -- 172.21.15.29:0/582921729 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e8004380 con 0x5597a0e2b390 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.235+0000 7f33e37fe640 1 -- 172.21.15.29:0/582921729 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035040 con 0x5597a0e2b390 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 -- 172.21.15.29:0/582921729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 msgr2=0x5597a11f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009230 tx=0x7f33e80314b0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 -- 172.21.15.29:0/582921729 shutdown_connections 2024-09-21T01:49:06.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e2b390 0x5597a11f3d40 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009230 tx=0x7f33e80314b0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 --2- 172.21.15.29:0/582921729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.236+0000 7f34059ef740 1 -- 172.21.15.29:0/582921729 >> 172.21.15.29:0/582921729 conn(0x5597a0e40460 msgr2=0x5597a11f11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.237+0000 7f34059ef740 1 -- 172.21.15.29:0/582921729 shutdown_connections 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.237+0000 7f33fda18640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 0x7f33e40b8e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005fd0 tx=0x7f33f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.237+0000 7f34059ef740 1 -- 172.21.15.29:0/582921729 wait complete. 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.237+0000 7f34059ef740 1 -- start start 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.237+0000 7f34059ef740 1 Processor -- start 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11fef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e0a9f0 0x5597a120ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5597a0fba700 con 0x5597a0e4c070 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5597a10f4160 con 0x5597a0e0a9f0 2024-09-21T01:49:06.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5597a1201a40 con 0x5597a0e231b0 2024-09-21T01:49:06.353 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:06.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f34059ef740 10 client.0 _ll_register_callbacks cb 0x5597a110d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:06.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f33fd217640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f33fd217640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39414/0 (socket says 172.21.15.29:39414) 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f33fd217640 1 -- 172.21.15.29:0/643123311 learned_addr learned my addr 172.21.15.29:0/643123311 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.238+0000 7f33fca16640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fda18640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e0a9f0 0x5597a120ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 -- 172.21.15.29:0/643123311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e0a9f0 msgr2=0x5597a120ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e0a9f0 0x5597a120ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 -- 172.21.15.29:0/643123311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 msgr2=0x5597a11fef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11fef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 -- 172.21.15.29:0/643123311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a1121c90 con 0x5597a0e231b0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33fd217640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33f40ac790 tx=0x7f33f40ac700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce92de8fde5b6a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4017030 con 0x5597a0e231b0 2024-09-21T01:49:06.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e8004380 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.239+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f33f40a8360 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.240+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f33f4017030 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.240+0000 7f34059ef740 10 client.25285 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.240+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5597a120d700 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e8004380 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f33f40e60d0 con 0x5597a0e231b0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 10 client.25285 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 10 client.25285 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 20 client.25285 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a0ddfda0 0x5597a1210b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 4 client.25285 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a116bdf0 con 0x5597a0ddfda0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.241+0000 7f33fca16640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a0ddfda0 0x5597a1210b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.242+0000 7f33fca16640 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a0ddfda0 0x5597a1210b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5cbe7c2648c4d64 server_cookie=d282257ff2d2bf6b in_seq=0 out_seq=0 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.242+0000 7f33e0ff9640 10 client.25285 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f33e0ff9640 1 -- 172.21.15.29:0/643123311 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597a116bdf0 con 0x5597a0ddfda0 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f33e0ff9640 10 client.25285 handle_command_reply: tid=0 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 client.25285 shutdown 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 msgr2=0x5597a11fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33f40ac790 tx=0x7f33f40ac700 comp rx=0 tx=0).stop 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 shutdown_connections 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e0a9f0 0x5597a120ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e4c070 0x5597a11fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a0ddfda0 0x5597a1210b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 --2- 172.21.15.29:0/643123311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a11fea00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 >> 172.21.15.29:0/643123311 conn(0x5597a0e40460 msgr2=0x5597a11f64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.243+0000 7f33e0ff9640 0 client.25285 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.244+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 shutdown_connections 2024-09-21T01:49:06.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.244+0000 7f34059ef740 1 -- 172.21.15.29:0/643123311 wait complete. 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.244+0000 7f34059ef740 20 client.25285 trim_cache size 0 max 16384 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.268+0000 7f34059ef740 1 Processor -- start 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.269+0000 7f34059ef740 1 -- start start 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a11f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b5b10 con 0x5597a0e0a9f0 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b7740 con 0x5597a0e231b0 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f34059ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b61e0 con 0x5597a0e4c070 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.270+0000 7f33fda18640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39424/0 (socket says 172.21.15.29:39424) 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 -- 172.21.15.29:0/612487519 learned_addr learned my addr 172.21.15.29:0/612487519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 -- 172.21.15.29:0/612487519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 msgr2=0x5597a11fbbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:06.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11fbbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 -- 172.21.15.29:0/612487519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 msgr2=0x5597a11f6ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a11f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.271+0000 7f33fda18640 1 -- 172.21.15.29:0/612487519 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a11f7010 con 0x5597a0e4c070 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.272+0000 7f33fda18640 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33f0003cd0 tx=0x7f33f0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fcbb981d0d27e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.274+0000 7f33e2ffd640 1 -- 172.21.15.29:0/612487519 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f00047c0 con 0x5597a0e4c070 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.274+0000 7f33e2ffd640 1 -- 172.21.15.29:0/612487519 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f0011950 con 0x5597a0e4c070 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.274+0000 7f33e2ffd640 1 -- 172.21.15.29:0/612487519 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f001e760 con 0x5597a0e4c070 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 -- 172.21.15.29:0/612487519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 msgr2=0x5597a11fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33f0003cd0 tx=0x7f33f0016000 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 -- 172.21.15.29:0/612487519 shutdown_connections 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e4c070 0x5597a11fc130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e231b0 0x5597a11fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 --2- 172.21.15.29:0/612487519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a11f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 -- 172.21.15.29:0/612487519 >> 172.21.15.29:0/612487519 conn(0x5597a0e40460 msgr2=0x5597a11f63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.276+0000 7f34059ef740 1 -- 172.21.15.29:0/612487519 shutdown_connections 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.277+0000 7f34059ef740 1 -- 172.21.15.29:0/612487519 wait complete. 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.277+0000 7f34059ef740 1 -- start start 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.278+0000 7f34059ef740 1 Processor -- start 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a1204280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fca16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a1204280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fca16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a1204280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39426/0 (socket says 172.21.15.29:39426) 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fd217640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fd217640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56830/0 (socket says 172.21.15.29:56830) 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b6fb0 con 0x5597a0e0a9f0 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b68e0 con 0x5597a0e4c070 2024-09-21T01:49:06.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b3a90 con 0x5597a0e231b0 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fda18640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f34059ef740 10 client.0 _ll_register_callbacks cb 0x7f33f4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.279+0000 7f33fda18640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:34944/0 (socket says 172.21.15.29:34944) 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 -- 172.21.15.29:0/3437690970 learned_addr learned my addr 172.21.15.29:0/3437690970 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 -- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 msgr2=0x5597a1204280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a1204280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 -- 172.21.15.29:0/3437690970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 msgr2=0x5597a1203d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fda18640 1 -- 172.21.15.29:0/3437690970 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a127b580 con 0x5597a0e4c070 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.280+0000 7f33fd217640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.281+0000 7f33fda18640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33f0004010 tx=0x7f33f0011530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb42d9fe970cf9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.281+0000 7f33e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.281+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035060 con 0x5597a0e4c070 2024-09-21T01:49:06.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.281+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f33f0011d10 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.281+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f0011950 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.282+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f33f0004a40 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.282+0000 7f34059ef740 10 client.25238 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.282+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5597a127c800 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.282+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f0004be0 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f33f0026070 con 0x5597a0e4c070 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 10 client.25238 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 10 client.25238 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 20 client.25238 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a127ceb0 0x5597a127f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 4 client.25238 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f33f0011950 con 0x5597a127ceb0 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.283+0000 7f33fd217640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a127ceb0 0x5597a127f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.284+0000 7f33fd217640 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a127ceb0 0x5597a127f2d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44cd4ac8f0b8f6 server_cookie=f2a9d8300ebcd8ec in_seq=0 out_seq=0 2024-09-21T01:49:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.284+0000 7f33e0ff9640 10 client.25238 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.284+0000 7f33e0ff9640 1 -- 172.21.15.29:0/3437690970 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f33f0011950 con 0x5597a127ceb0 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.284+0000 7f33e0ff9640 10 client.25238 handle_command_reply: tid=0 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.284+0000 7f34059ef740 1 client.25238 shutdown 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 msgr2=0x5597a12047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33f0004010 tx=0x7f33f0011530 comp rx=0 tx=0).stop 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 shutdown_connections 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a0e4c070 0x5597a12047c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5597a127ceb0 0x5597a127f2d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597a0e231b0 0x5597a1204280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 --2- 172.21.15.29:0/3437690970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597a0e0a9f0 0x5597a1203d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 >> 172.21.15.29:0/3437690970 conn(0x5597a0e40460 msgr2=0x5597a11f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f33e0ff9640 0 client.25238 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 shutdown_connections 2024-09-21T01:49:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 1 -- 172.21.15.29:0/3437690970 wait complete. 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.285+0000 7f34059ef740 20 client.25238 trim_cache size 0 max 16384 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 msgr2=0x7f33e40b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 0x7f33e40b8e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005fd0 tx=0x7f33f0009500 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 msgr2=0x7f33f81a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d770 tx=0x7f33f400dc30 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 shutdown_connections 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f33e40b6900 0x7f33e40b8e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f8107be0 0x7f33f81a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33f81030f0 0x7f33f81a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 --2- 172.21.15.29:0/1916780931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33f81027a0 0x7f33f81a0af0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 >> 172.21.15.29:0/1916780931 conn(0x7f33f8079730 msgr2=0x7f33f81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 shutdown_connections 2024-09-21T01:49:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:06.286+0000 7f33ff47b640 1 -- 172.21.15.29:0/1916780931 wait complete. 2024-09-21T01:49:07.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:49:07.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 30.0 s, 5065.0 tps, lat 19.717 ms stddev 21.223 2024-09-21T01:49:08.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:07 smithi066 ceph-mon[35001]: pgmap v383: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-21T01:49:08.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:07 smithi029 ceph-mon[31228]: pgmap v383: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-21T01:49:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:07 smithi190 ceph-mon[34368]: pgmap v383: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-21T01:49:08.279 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:08.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.903+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/2056216346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f806af90 msgr2=0x7fc1f810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.903+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/2056216346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f806af90 0x7fc1f810ac50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800aac0 tx=0x7fc1e802c870 comp rx=0 tx=0).stop 2024-09-21T01:49:08.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.904+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/2056216346 shutdown_connections 2024-09-21T01:49:08.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.904+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/2056216346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f81173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.904+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/2056216346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f806af90 0x7fc1f810ac50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.904+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/2056216346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f810f150 0x7fc1f810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.904+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/2056216346 >> 172.21.15.29:0/2056216346 conn(0x7fc1f80712f0 msgr2=0x7fc1f8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.906+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/2056216346 shutdown_connections 2024-09-21T01:49:08.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.906+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/2056216346 wait complete. 2024-09-21T01:49:08.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.907+0000 7fc1fdd8d640 1 Processor -- start 2024-09-21T01:49:08.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 -- start start 2024-09-21T01:49:08.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 0x7fc1f811c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 0x7fc1f8118310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f8118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f811e510 con 0x7fc1f806af90 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f811e390 con 0x7fc1f8114fb0 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1fdd8d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f811e690 con 0x7fc1f810f150 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1f6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 0x7fc1f8118310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 0x7fc1f811c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1f6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 0x7fc1f8118310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39446/0 (socket says 172.21.15.29:39446) 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.908+0000 7fc1f6ffd640 1 -- 172.21.15.29:0/189820364 learned_addr learned my addr 172.21.15.29:0/189820364 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:08.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f8118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f77fe640 1 -- 172.21.15.29:0/189820364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 msgr2=0x7fc1f8118310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 0x7fc1f8118310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f77fe640 1 -- 172.21.15.29:0/189820364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 msgr2=0x7fc1f8118a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f8118a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f77fe640 1 -- 172.21.15.29:0/189820364 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81b5490 con 0x7fc1f806af90 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.909+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f8118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:08.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.910+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 0x7fc1f811c030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec01e6e0 tx=0x7fc1ec01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.910+0000 7fc1f4ff9640 1 -- 172.21.15.29:0/189820364 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec028070 con 0x7fc1f806af90 2024-09-21T01:49:08.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.910+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a4c90 con 0x7fc1f806af90 2024-09-21T01:49:08.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.911+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a5220 con 0x7fc1f806af90 2024-09-21T01:49:08.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.911+0000 7fc1f4ff9640 1 -- 172.21.15.29:0/189820364 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ec009820 con 0x7fc1f806af90 2024-09-21T01:49:08.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.911+0000 7fc1f4ff9640 1 -- 172.21.15.29:0/189820364 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec031060 con 0x7fc1f806af90 2024-09-21T01:49:08.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 Processor -- start 2024-09-21T01:49:08.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 -- start start 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 0x55b66579d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b66579dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b665713ea0 con 0x55b6653d5390 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b6657032d0 con 0x55b6653f6070 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc204301740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6657070e0 con 0x55b6653cd1b0 2024-09-21T01:49:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.916+0000 7fc1f4ff9640 1 -- 172.21.15.29:0/189820364 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc1ec009cd0 con 0x7fc1f806af90 2024-09-21T01:49:08.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.917+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b66579dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.917+0000 7fc1f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.917+0000 7fc1f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56864/0 (socket says 172.21.15.29:56864) 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.917+0000 7fc1f6ffd640 1 -- 172.21.15.29:0/630177183 learned_addr learned my addr 172.21.15.29:0/630177183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.917+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 0x55b66579d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 -- 172.21.15.29:0/630177183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 msgr2=0x55b66579dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b66579dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 -- 172.21.15.29:0/630177183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 msgr2=0x55b66579d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 0x55b66579d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 -- 172.21.15.29:0/630177183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b665461650 con 0x55b6653d5390 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 0x55b66579d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f4ff9640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 0x7fc1e00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800aa90 tx=0x7fc1e8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c548695f155364 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 0x7fc1e00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.918+0000 7fc1f4ff9640 1 -- 172.21.15.29:0/189820364 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1ec0f7bf0 con 0x7fc1f806af90 2024-09-21T01:49:08.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.919+0000 7fc1ddffb640 1 -- 172.21.15.29:0/630177183 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8042030 con 0x55b6653d5390 2024-09-21T01:49:08.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.919+0000 7fc1ddffb640 1 -- 172.21.15.29:0/630177183 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8005040 con 0x55b6653d5390 2024-09-21T01:49:08.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.919+0000 7fc1ddffb640 1 -- 172.21.15.29:0/630177183 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8038040 con 0x55b6653d5390 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.920+0000 7fc204301740 1 -- 172.21.15.29:0/630177183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 msgr2=0x55b66579e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.920+0000 7fc204301740 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800aa90 tx=0x7fc1e8004040 comp rx=0 tx=0).stop 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.920+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 0x7fc1e00b8ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0005fd0 tx=0x7fc1f00096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 -- 172.21.15.29:0/630177183 shutdown_connections 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653d5390 0x55b66579e120 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b66579dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 --2- 172.21.15.29:0/630177183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653f6070 0x55b66579d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 -- 172.21.15.29:0/630177183 >> 172.21.15.29:0/630177183 conn(0x55b6653ea460 msgr2=0x55b66579b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 -- 172.21.15.29:0/630177183 shutdown_connections 2024-09-21T01:49:08.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.921+0000 7fc204301740 1 -- 172.21.15.29:0/630177183 wait complete. 2024-09-21T01:49:08.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.922+0000 7fc204301740 1 -- start start 2024-09-21T01:49:08.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.922+0000 7fc204301740 1 Processor -- start 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.922+0000 7fc204301740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653f6070 0x55b6657b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b665564700 con 0x55b6653f6070 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b66569e160 con 0x55b6653d5390 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6657abe20 con 0x55b6653cd1b0 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc204301740 10 client.0 _ll_register_callbacks cb 0x55b6656b79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39458/0 (socket says 172.21.15.29:39458) 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 -- 172.21.15.29:0/1765658709 learned_addr learned my addr 172.21.15.29:0/1765658709 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 -- 172.21.15.29:0/1765658709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 msgr2=0x55b6657a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 -- 172.21.15.29:0/1765658709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653f6070 msgr2=0x55b6657b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653f6070 0x55b6657b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.923+0000 7fc1f77fe640 1 -- 172.21.15.29:0/1765658709 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6656cbc90 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.924+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0ba3b0 tx=0x7fc1ec0bd550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359fe354518cae3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.924+0000 7fc1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0c5d90 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e8005040 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1ec0f5b60 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1ec0c7750 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc204301740 10 client.25300 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.925+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6657b7b70 con 0x55b6653cd1b0 2024-09-21T01:49:08.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.926+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8005040 con 0x55b6653cd1b0 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.926+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc1ec01f040 con 0x55b6653cd1b0 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 10 client.25300 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 10 client.25300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 20 client.25300 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653b49f0 0x55b6657bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 4 client.25300 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b665715df0 con 0x55b6653b49f0 2024-09-21T01:49:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.927+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653b49f0 0x55b6657bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.928+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653b49f0 0x55b6657bb140 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f131c3312f4848b4 server_cookie=a7146127399dc2d8 in_seq=0 out_seq=0 2024-09-21T01:49:08.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.928+0000 7fc1d6ffd640 10 client.25300 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:49:08.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.928+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1765658709 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b665715df0 con 0x55b6653b49f0 2024-09-21T01:49:08.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.928+0000 7fc1d6ffd640 10 client.25300 handle_command_reply: tid=0 2024-09-21T01:49:08.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.928+0000 7fc204301740 1 client.25300 shutdown 2024-09-21T01:49:08.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 msgr2=0x55b6657a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0ba3b0 tx=0x7fc1ec0bd550 comp rx=0 tx=0).stop 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 shutdown_connections 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653b49f0 0x55b6657bb140 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653f6070 0x55b6657b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 --2- 172.21.15.29:0/1765658709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653cd1b0 0x55b6657a8d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc1d6ffd640 0 client.25300 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:49:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.929+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 >> 172.21.15.29:0/1765658709 conn(0x55b6653ea460 msgr2=0x55b6657a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.930+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 shutdown_connections 2024-09-21T01:49:08.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.930+0000 7fc204301740 1 -- 172.21.15.29:0/1765658709 wait complete. 2024-09-21T01:49:08.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.930+0000 7fc204301740 20 client.25300 trim_cache size 0 max 16384 2024-09-21T01:49:08.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.958+0000 7fc204301740 1 Processor -- start 2024-09-21T01:49:08.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.958+0000 7fc204301740 1 -- start start 2024-09-21T01:49:08.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66579ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b5a20 con 0x55b6653cd1b0 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b7650 con 0x55b6653d5390 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc204301740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b60f0 con 0x55b6653ea460 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66579ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66579ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39464/0 (socket says 172.21.15.29:39464) 2024-09-21T01:49:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.959+0000 7fc1f77fe640 1 -- 172.21.15.29:0/358156222 learned_addr learned my addr 172.21.15.29:0/358156222 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:08.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.960+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.960+0000 7fc1f6ffd640 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 -- 172.21.15.29:0/358156222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 msgr2=0x55b66579ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66579ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 -- 172.21.15.29:0/358156222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 msgr2=0x55b6657b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 -- 172.21.15.29:0/358156222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6657ab2f0 con 0x55b6653d5390 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.961+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a5f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0011810 tx=0x7fc1f0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ac39b2ed52a619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.962+0000 7fc1dd7fa640 1 -- 172.21.15.29:0/358156222 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0011d60 con 0x55b6653d5390 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.962+0000 7fc1dd7fa640 1 -- 172.21.15.29:0/358156222 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f001c8a0 con 0x55b6653d5390 2024-09-21T01:49:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.962+0000 7fc1dd7fa640 1 -- 172.21.15.29:0/358156222 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0027800 con 0x55b6653d5390 2024-09-21T01:49:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 -- 172.21.15.29:0/358156222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 msgr2=0x55b6657a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a5f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0011810 tx=0x7fc1f0004a40 comp rx=0 tx=0).stop 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 -- 172.21.15.29:0/358156222 shutdown_connections 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657a5f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 --2- 172.21.15.29:0/358156222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66579ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 -- 172.21.15.29:0/358156222 >> 172.21.15.29:0/358156222 conn(0x55b665796780 msgr2=0x55b66579e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 -- 172.21.15.29:0/358156222 shutdown_connections 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.963+0000 7fc204301740 1 -- 172.21.15.29:0/358156222 wait complete. 2024-09-21T01:49:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.964+0000 7fc204301740 1 -- start start 2024-09-21T01:49:08.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.964+0000 7fc204301740 1 Processor -- start 2024-09-21T01:49:08.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.965+0000 7fc204301740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.965+0000 7fc204301740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.965+0000 7fc204301740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66581ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.965+0000 7fc204301740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b6ec0 con 0x55b6653cd1b0 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.965+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66581ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56894/0 (socket says 172.21.15.29:56894) 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f77fe640 1 -- 172.21.15.29:0/1069829678 learned_addr learned my addr 172.21.15.29:0/1069829678 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b67f0 con 0x55b6653d5390 2024-09-21T01:49:08.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ab7d0 con 0x55b6653ea460 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc204301740 10 client.0 _ll_register_callbacks cb 0x7fc1ec0057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 -- 172.21.15.29:0/1069829678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 msgr2=0x55b6657ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 -- 172.21.15.29:0/1069829678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 msgr2=0x55b6657adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.966+0000 7fc1f7fff640 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66581f2d0 con 0x55b6653ea460 2024-09-21T01:49:08.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.967+0000 7fc1f7fff640 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66581ec60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0010460 tx=0x7fc1f0011f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e169512f0d003df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:08.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.967+0000 7fc1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.967+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0036060 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.967+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1f001ca50 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.968+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f001c8a0 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.968+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1f001c700 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.968+0000 7fc204301740 10 client.25306 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.968+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b665826610 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.968+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f001cc50 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc1f0026070 con 0x55b6653ea460 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 10 client.25306 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 10 client.25306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:49:08.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 20 client.25306 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:08.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653f6070 0x55b665828c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:08.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 4 client.25306 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:08.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.969+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc1f001c8a0 con 0x55b6653f6070 2024-09-21T01:49:08.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.971+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653f6070 0x55b665828c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:08.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc1d6ffd640 10 client.25306 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:49:08.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc1f77fe640 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653f6070 0x55b665828c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9be882f60c3c2893 server_cookie=e58b4b0ea1afdb48 in_seq=0 out_seq=0 2024-09-21T01:49:08.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc1d6ffd640 1 -- 172.21.15.29:0/1069829678 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc1f001c8a0 con 0x55b6653f6070 2024-09-21T01:49:08.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc1d6ffd640 10 client.25306 handle_command_reply: tid=0 2024-09-21T01:49:08.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 client.25306 shutdown 2024-09-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 msgr2=0x55b66581ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66581ec60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0010460 tx=0x7fc1f0011f70 comp rx=0 tx=0).stop 2024-09-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 shutdown_connections 2024-09-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55b6653f6070 0x55b665828c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6653ea460 0x55b66581ec60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6653d5390 0x55b6657ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc1d6ffd640 0 client.25306 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 --2- 172.21.15.29:0/1069829678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6653cd1b0 0x55b6657adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.972+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 >> 172.21.15.29:0/1069829678 conn(0x55b665796780 msgr2=0x55b66579e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.973+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 shutdown_connections 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.973+0000 7fc204301740 1 -- 172.21.15.29:0/1069829678 wait complete. 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.973+0000 7fc204301740 20 client.25306 trim_cache size 0 max 16384 2024-09-21T01:49:08.974 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:08.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 msgr2=0x7fc1e00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 0x7fc1e00b8ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0005fd0 tx=0x7fc1f00096a0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 msgr2=0x7fc1f811c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 0x7fc1f811c030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec01e6e0 tx=0x7fc1ec01eba0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 shutdown_connections 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1e00b65d0 0x7fc1e00b8ae0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1f8114fb0 0x7fc1f8118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f810f150 0x7fc1f8118310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 --2- 172.21.15.29:0/189820364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1f806af90 0x7fc1f811c030 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.974+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 >> 172.21.15.29:0/189820364 conn(0x7fc1f80712f0 msgr2=0x7fc1f810d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.975+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 shutdown_connections 2024-09-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:08.975+0000 7fc1fdd8d640 1 -- 172.21.15.29:0/189820364 wait complete. 2024-09-21T01:49:09.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:49:10.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:09 smithi066 ceph-mon[35001]: pgmap v384: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-21T01:49:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:09 smithi029 ceph-mon[31228]: pgmap v384: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-21T01:49:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:09 smithi190 ceph-mon[34368]: pgmap v384: 97 pgs: 97 active+clean; 993 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-21T01:49:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:10.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 -- 172.21.15.29:0/2211231748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010be50 msgr2=0x7fcec0112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 --2- 172.21.15.29:0/2211231748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010be50 0x7fcec0112680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009be0 tx=0x7fceb802a6c0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 -- 172.21.15.29:0/2211231748 shutdown_connections 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 --2- 172.21.15.29:0/2211231748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010be50 0x7fcec0112680 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 --2- 172.21.15.29:0/2211231748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec010b500 0x7fcec010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 --2- 172.21.15.29:0/2211231748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0105bc0 0x7fcec0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.890+0000 7fcec6326640 1 -- 172.21.15.29:0/2211231748 >> 172.21.15.29:0/2211231748 conn(0x7fcec0101630 msgr2=0x7fcec0103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.891+0000 7fcec6326640 1 -- 172.21.15.29:0/2211231748 shutdown_connections 2024-09-21T01:49:10.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.891+0000 7fcec6326640 1 -- 172.21.15.29:0/2211231748 wait complete. 2024-09-21T01:49:10.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 Processor -- start 2024-09-21T01:49:10.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 -- start start 2024-09-21T01:49:10.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0105bc0 0x7fcec010fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec010be50 0x7fcec01105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0115b10 con 0x7fcec010b500 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0115990 con 0x7fcec0105bc0 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.892+0000 7fcec6326640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0115c90 con 0x7fcec010be50 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.893+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.893+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56920/0 (socket says 172.21.15.29:56920) 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.893+0000 7fcec4b23640 1 -- 172.21.15.29:0/1998601361 learned_addr learned my addr 172.21.15.29:0/1998601361 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:10.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 -- 172.21.15.29:0/1998601361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec010be50 msgr2=0x7fcec01105a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec010be50 0x7fcec01105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 -- 172.21.15.29:0/1998601361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0105bc0 msgr2=0x7fcec010fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0105bc0 0x7fcec010fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 -- 172.21.15.29:0/1998601361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec01ac9c0 con 0x7fcec010b500 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.894+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9f0 tx=0x7fcebc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.895+0000 7fceb67fc640 1 -- 172.21.15.29:0/1998601361 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00ccc0 con 0x7fcec010b500 2024-09-21T01:49:10.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.895+0000 7fceb67fc640 1 -- 172.21.15.29:0/1998601361 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc0108e0 con 0x7fcec010b500 2024-09-21T01:49:10.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.895+0000 7fceb67fc640 1 -- 172.21.15.29:0/1998601361 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc018860 con 0x7fcec010b500 2024-09-21T01:49:10.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.897+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec01accb0 con 0x7fcec010b500 2024-09-21T01:49:10.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.897+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01ad1c0 con 0x7fcec010b500 2024-09-21T01:49:10.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.900+0000 7fcecdafc740 1 Processor -- start 2024-09-21T01:49:10.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.900+0000 7fcecdafc740 1 -- start start 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982ee9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982ee9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557982e5fea0 con 0x557982b42070 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557982e4f2d0 con 0x557982b21390 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.901+0000 7fcecdafc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557982e530e0 con 0x557982b191b0 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.902+0000 7fcec5324640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.902+0000 7fcec5324640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39492/0 (socket says 172.21.15.29:39492) 2024-09-21T01:49:10.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.902+0000 7fcec5324640 1 -- 172.21.15.29:0/1364862785 learned_addr learned my addr 172.21.15.29:0/1364862785 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.902+0000 7fcec5b25640 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982ee9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 -- 172.21.15.29:0/1364862785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 msgr2=0x557982ee9ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982ee9ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 -- 172.21.15.29:0/1364862785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 msgr2=0x557982ee9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982ee9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fceb67fc640 1 -- 172.21.15.29:0/1998601361 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcebc010430 con 0x7fcec010b500 2024-09-21T01:49:10.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 -- 172.21.15.29:0/1364862785 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557982bad650 con 0x557982b191b0 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.903+0000 7fcec5324640 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004650 tx=0x7fceb000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d341034d557f0616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.904+0000 7fceab7fe640 1 -- 172.21.15.29:0/1364862785 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb000ed20 con 0x557982b191b0 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.904+0000 7fceab7fe640 1 -- 172.21.15.29:0/1364862785 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb00054d0 con 0x557982b191b0 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.904+0000 7fceb67fc640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 0x7fceac0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fceab7fe640 1 -- 172.21.15.29:0/1364862785 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0010430 con 0x557982b191b0 2024-09-21T01:49:10.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fceb67fc640 1 -- 172.21.15.29:0/1998601361 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcebc0e7960 con 0x7fcec010b500 2024-09-21T01:49:10.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 -- 172.21.15.29:0/1364862785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 msgr2=0x557982ee99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004650 tx=0x7fceb000e350 comp rx=0 tx=0).stop 2024-09-21T01:49:10.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 -- 172.21.15.29:0/1364862785 shutdown_connections 2024-09-21T01:49:10.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982ee9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982ee99a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1364862785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982ee9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 -- 172.21.15.29:0/1364862785 >> 172.21.15.29:0/1364862785 conn(0x557982b36460 msgr2=0x557982ee7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.905+0000 7fcecdafc740 1 -- 172.21.15.29:0/1364862785 shutdown_connections 2024-09-21T01:49:10.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.907+0000 7fcec5b25640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 0x7fceac0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.907+0000 7fcecdafc740 1 -- 172.21.15.29:0/1364862785 wait complete. 2024-09-21T01:49:10.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.907+0000 7fcec5b25640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 0x7fceac0b8aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fceb8004af0 tx=0x7fceb8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.907+0000 7fcecdafc740 1 -- start start 2024-09-21T01:49:10.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 Processor -- start 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982f06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557982cb0700 con 0x557982b42070 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557982dea160 con 0x557982b21390 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557982f08bf0 con 0x557982b191b0 2024-09-21T01:49:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.908+0000 7fcecdafc740 10 client.0 _ll_register_callbacks cb 0x557982e039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39502/0 (socket says 172.21.15.29:39502) 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 -- 172.21.15.29:0/3556941693 learned_addr learned my addr 172.21.15.29:0/3556941693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5b25640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 -- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 msgr2=0x557982f06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982f06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 -- 172.21.15.29:0/3556941693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 msgr2=0x557982f11fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f11fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.909+0000 7fcec5324640 1 -- 172.21.15.29:0/3556941693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557982e17c90 con 0x557982b191b0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcec5324640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004d40 tx=0x7fceb0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d56362438b8df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0025020 con 0x557982b191b0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fceb00047a0 con 0x557982b191b0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb00054d0 con 0x557982b191b0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fceb00097b0 con 0x557982b191b0 2024-09-21T01:49:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcecdafc740 10 client.25315 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.910+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557982f14960 con 0x557982b191b0 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb00054d0 con 0x557982b191b0 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fceb0010430 con 0x557982b191b0 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 10 client.25315 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 10 client.25315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 20 client.25315 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982b009f0 0x557982f18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 4 client.25315 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557982e61df0 con 0x557982b009f0 2024-09-21T01:49:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.911+0000 7fcec4b23640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982b009f0 0x557982f18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.912+0000 7fcec4b23640 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982b009f0 0x557982f18170 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbb4b281f997916e server_cookie=a60a6145cec96618 in_seq=0 out_seq=0 2024-09-21T01:49:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.912+0000 7fcea8ff9640 10 client.25315 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:49:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.913+0000 7fcea8ff9640 1 -- 172.21.15.29:0/3556941693 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557982e61df0 con 0x557982b009f0 2024-09-21T01:49:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.913+0000 7fcea8ff9640 10 client.25315 handle_command_reply: tid=0 2024-09-21T01:49:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.913+0000 7fcecdafc740 1 client.25315 shutdown 2024-09-21T01:49:10.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 msgr2=0x557982f05af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004d40 tx=0x7fceb0005e80 comp rx=0 tx=0).stop 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 shutdown_connections 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982b009f0 0x557982f18170 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f11fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b21390 0x557982f06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 --2- 172.21.15.29:0/3556941693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b191b0 0x557982f05af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 >> 172.21.15.29:0/3556941693 conn(0x557982b36460 msgr2=0x557982eeae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.915+0000 7fcea8ff9640 0 client.25315 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:49:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.916+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 shutdown_connections 2024-09-21T01:49:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.916+0000 7fcecdafc740 1 -- 172.21.15.29:0/3556941693 wait complete. 2024-09-21T01:49:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.916+0000 7fcecdafc740 20 client.25315 trim_cache size 0 max 16384 2024-09-21T01:49:10.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.944+0000 7fcecdafc740 1 Processor -- start 2024-09-21T01:49:10.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.944+0000 7fcecdafc740 1 -- start start 2024-09-21T01:49:10.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.945+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b191b0 0x557982f11ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.945+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.945+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b42070 0x557982f058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.945+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39514/0 (socket says 172.21.15.29:39514) 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcec5324640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b191b0 0x557982f11ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcec4b23640 1 -- 172.21.15.29:0/1361788381 learned_addr learned my addr 172.21.15.29:0/1361788381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01af510 con 0x557982b191b0 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01aee40 con 0x557982b42070 2024-09-21T01:49:10.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01ad8e0 con 0x557982b21390 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.946+0000 7fcec5b25640 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b42070 0x557982f058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 -- 172.21.15.29:0/1361788381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b42070 msgr2=0x557982f058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b42070 0x557982f058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 -- 172.21.15.29:0/1361788381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b191b0 msgr2=0x557982f11ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b191b0 0x557982f11ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 -- 172.21.15.29:0/1361788381 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557982f08230 con 0x557982b21390 2024-09-21T01:49:10.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0e7c40 tx=0x7fcebc00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39796b523f04fb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.947+0000 7fceaaffd640 1 -- 172.21.15.29:0/1361788381 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0b2070 con 0x557982b21390 2024-09-21T01:49:10.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.948+0000 7fceaaffd640 1 -- 172.21.15.29:0/1361788381 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc010720 con 0x557982b21390 2024-09-21T01:49:10.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.948+0000 7fceaaffd640 1 -- 172.21.15.29:0/1361788381 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc010a90 con 0x557982b21390 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 msgr2=0x557982eeee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0e7c40 tx=0x7fcebc00ce60 comp rx=0 tx=0).stop 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 shutdown_connections 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b42070 0x557982f058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982eeee40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1361788381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b191b0 0x557982f11ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 >> 172.21.15.29:0/1361788381 conn(0x557982b36460 msgr2=0x557982f00ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.949+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 shutdown_connections 2024-09-21T01:49:10.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.950+0000 7fcecdafc740 1 -- 172.21.15.29:0/1361788381 wait complete. 2024-09-21T01:49:10.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.950+0000 7fcecdafc740 1 -- start start 2024-09-21T01:49:10.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 Processor -- start 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b191b0 0x557982f092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f80b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01ae6b0 con 0x557982b42070 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01adfe0 con 0x557982b191b0 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a1620 con 0x557982b21390 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.951+0000 7fcecdafc740 10 client.0 _ll_register_callbacks cb 0x7fceb0009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39522/0 (socket says 172.21.15.29:39522) 2024-09-21T01:49:10.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 -- 172.21.15.29:0/1508493016 learned_addr learned my addr 172.21.15.29:0/1508493016 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:10.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec5324640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b191b0 0x557982f092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 -- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b191b0 msgr2=0x557982f092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b191b0 0x557982f092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 -- 172.21.15.29:0/1508493016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 msgr2=0x557982f80b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f80b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 -- 172.21.15.29:0/1508493016 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557982f0a040 con 0x557982b21390 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.952+0000 7fcec4b23640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ada30 tx=0x7fcebc0e7c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ecb97a05572c033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.953+0000 7fcea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.953+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0b2040 con 0x557982b21390 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.953+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcebc0a9d10 con 0x557982b21390 2024-09-21T01:49:10.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.953+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557982e5d8a0 con 0x557982b21390 2024-09-21T01:49:10.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.954+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcebc0ad5b0 con 0x557982b21390 2024-09-21T01:49:10.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.954+0000 7fcecdafc740 10 client.25321 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:10.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.954+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557982f82060 con 0x557982b21390 2024-09-21T01:49:10.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.954+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc0ae1a0 con 0x557982b21390 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fcebc0cfab0 con 0x557982b21390 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 10 client.25321 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 10 client.25321 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 20 client.25321 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982ad5da0 0x557982f84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 4 client.25321 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:10.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.955+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557982e5d8a0 con 0x557982ad5da0 2024-09-21T01:49:10.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcec5324640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982ad5da0 0x557982f84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:10.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcec5324640 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982ad5da0 0x557982f84780 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5583429ce33fd78 server_cookie=d12a654da81b57f1 in_seq=0 out_seq=0 2024-09-21T01:49:10.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcea8ff9640 10 client.25321 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:49:10.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcea8ff9640 1 -- 172.21.15.29:0/1508493016 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557982e5d8a0 con 0x557982ad5da0 2024-09-21T01:49:10.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcea8ff9640 10 client.25321 handle_command_reply: tid=0 2024-09-21T01:49:10.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.957+0000 7fcecdafc740 1 client.25321 shutdown 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 msgr2=0x557982f09830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ada30 tx=0x7fcebc0e7c70 comp rx=0 tx=0).stop 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 shutdown_connections 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x557982ad5da0 0x557982f84780 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557982b42070 0x557982f80b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcea8ff9640 0 client.25321 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557982b21390 0x557982f09830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 --2- 172.21.15.29:0/1508493016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557982b191b0 0x557982f092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 >> 172.21.15.29:0/1508493016 conn(0x557982b36460 msgr2=0x557982eeede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 shutdown_connections 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 1 -- 172.21.15.29:0/1508493016 wait complete. 2024-09-21T01:49:10.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.958+0000 7fcecdafc740 20 client.25321 trim_cache size 0 max 16384 2024-09-21T01:49:10.960 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 msgr2=0x7fceac0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 0x7fceac0b8aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fceb8004af0 tx=0x7fceb8004a40 comp rx=0 tx=0).stop 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 msgr2=0x7fcec0110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9f0 tx=0x7fcebc00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 shutdown_connections 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fceac0b6590 0x7fceac0b8aa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec010be50 0x7fcec01105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcec010b500 0x7fcec0110060 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 --2- 172.21.15.29:0/1998601361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0105bc0 0x7fcec010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:10.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 >> 172.21.15.29:0/1998601361 conn(0x7fcec0101630 msgr2=0x7fcec0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:10.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 shutdown_connections 2024-09-21T01:49:10.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:10.959+0000 7fcec6326640 1 -- 172.21.15.29:0/1998601361 wait complete. 2024-09-21T01:49:12.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:11 smithi066 ceph-mon[35001]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T01:49:12.057 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:11 smithi029 ceph-mon[31228]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T01:49:12.058 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:49:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:11 smithi190 ceph-mon[34368]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T01:49:12.486 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 35.0 s, 5259.4 tps, lat 19.033 ms stddev 19.687 2024-09-21T01:49:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.096+0000 7f704b59e640 1 -- 172.21.15.29:0/2750849311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c10dca0 msgr2=0x7f704c110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.096+0000 7f704b59e640 1 --2- 172.21.15.29:0/2750849311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c10dca0 0x7f704c110170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7044009ca0 tx=0x7f704402a8b0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.097+0000 7f704b59e640 1 -- 172.21.15.29:0/2750849311 shutdown_connections 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.097+0000 7f704b59e640 1 --2- 172.21.15.29:0/2750849311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c10dca0 0x7f704c110170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.097+0000 7f704b59e640 1 --2- 172.21.15.29:0/2750849311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.097+0000 7f704b59e640 1 --2- 172.21.15.29:0/2750849311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c1046f0 0x7f704c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.097+0000 7f704b59e640 1 -- 172.21.15.29:0/2750849311 >> 172.21.15.29:0/2750849311 conn(0x7f704c0fe380 msgr2=0x7f704c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.098+0000 7f704b59e640 1 -- 172.21.15.29:0/2750849311 shutdown_connections 2024-09-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.098+0000 7f704b59e640 1 -- 172.21.15.29:0/2750849311 wait complete. 2024-09-21T01:49:13.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.099+0000 7f704b59e640 1 Processor -- start 2024-09-21T01:49:13.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.099+0000 7f704b59e640 1 -- start start 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c1046f0 0x7f704c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c10dca0 0x7f704c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1138e0 con 0x7f704c1046f0 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f704c113760 con 0x7f704c105120 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704b59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f704c113a60 con 0x7f704c10dca0 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f7049d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f7049d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54518/0 (socket says 172.21.15.29:54518) 2024-09-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f7049d9b640 1 -- 172.21.15.29:0/1891667444 learned_addr learned my addr 172.21.15.29:0/1891667444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704ad9d640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c10dca0 0x7f704c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.100+0000 7f704a59c640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c1046f0 0x7f704c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.101+0000 7f7049d9b640 1 -- 172.21.15.29:0/1891667444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c10dca0 msgr2=0x7f704c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.101+0000 7f7049d9b640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c10dca0 0x7f704c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.101+0000 7f7049d9b640 1 -- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c1046f0 msgr2=0x7f704c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.101+0000 7f7049d9b640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c1046f0 0x7f704c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.101+0000 7f7049d9b640 1 -- 172.21.15.29:0/1891667444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a0240 con 0x7f704c105120 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f7049d9b640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f703c005fd0 tx=0x7f703c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f703b7fe640 1 -- 172.21.15.29:0/1891667444 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c017070 con 0x7f704c105120 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f703b7fe640 1 -- 172.21.15.29:0/1891667444 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c009c30 con 0x7f704c105120 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f703b7fe640 1 -- 172.21.15.29:0/1891667444 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c003d30 con 0x7f704c105120 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a0530 con 0x7f704c105120 2024-09-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.102+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a6c90 con 0x7f704c105120 2024-09-21T01:49:13.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.107+0000 7f7057560740 1 Processor -- start 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 -- start start 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cbc368ea0 con 0x564cbc02a390 2024-09-21T01:49:13.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.108+0000 7f7057560740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cbc3582d0 con 0x564cbc04b070 2024-09-21T01:49:13.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f7057560740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cbc35c0e0 con 0x564cbc0221b0 2024-09-21T01:49:13.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49970/0 (socket says 172.21.15.29:49970) 2024-09-21T01:49:13.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 -- 172.21.15.29:0/140834468 learned_addr learned my addr 172.21.15.29:0/140834468 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:13.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704ad9d640 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 -- 172.21.15.29:0/140834468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 msgr2=0x564cbc3f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f703b7fe640 1 -- 172.21.15.29:0/1891667444 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f703c003ed0 con 0x7f704c105120 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 -- 172.21.15.29:0/140834468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 msgr2=0x564cbc3f2ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.109+0000 7f704a59c640 1 -- 172.21.15.29:0/140834468 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc0b6650 con 0x564cbc0221b0 2024-09-21T01:49:13.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.110+0000 7f704a59c640 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704000bc90 tx=0x7f704000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b8c32653450dfc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.110+0000 7f7038ff9640 1 -- 172.21.15.29:0/140834468 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040015070 con 0x564cbc0221b0 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.110+0000 7f7038ff9640 1 -- 172.21.15.29:0/140834468 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704001a910 con 0x564cbc0221b0 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.110+0000 7f7038ff9640 1 -- 172.21.15.29:0/140834468 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400118e0 con 0x564cbc0221b0 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.112+0000 7f7057560740 1 -- 172.21.15.29:0/140834468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 msgr2=0x564cbc3f29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.112+0000 7f7057560740 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704000bc90 tx=0x7f704000f730 comp rx=0 tx=0).stop 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f703b7fe640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 0x7f70340b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 -- 172.21.15.29:0/140834468 shutdown_connections 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f29a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 --2- 172.21.15.29:0/140834468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 -- 172.21.15.29:0/140834468 >> 172.21.15.29:0/140834468 conn(0x564cbc03f460 msgr2=0x564cbc3f0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f703b7fe640 1 -- 172.21.15.29:0/1891667444 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f703c0e6ed0 con 0x7f704c105120 2024-09-21T01:49:13.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f704ad9d640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 0x7f70340b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.113+0000 7f7057560740 1 -- 172.21.15.29:0/140834468 shutdown_connections 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.114+0000 7f7057560740 1 -- 172.21.15.29:0/140834468 wait complete. 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.115+0000 7f7057560740 1 -- start start 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.115+0000 7f7057560740 1 Processor -- start 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc3fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc408b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564cbc1b9700 con 0x564cbc02a390 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564cbc2f3160 con 0x564cbc0221b0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564cbc3fcb80 con 0x564cbc04b070 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7057560740 10 client.0 _ll_register_callbacks cb 0x564cbc30c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 0x7f70340b8b20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70440094e0 tx=0x7f7044030f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49978/0 (socket says 172.21.15.29:49978) 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 -- 172.21.15.29:0/3798594063 learned_addr learned my addr 172.21.15.29:0/3798594063 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f7049d9b640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc408b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704a59c640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc3fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 -- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 msgr2=0x564cbc3fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc3fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 -- 172.21.15.29:0/3798594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 msgr2=0x564cbc408b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc408b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704ad9d640 1 -- 172.21.15.29:0/3798594063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbc320c90 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.116+0000 7f704a59c640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc3fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f704ad9d640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7044035000 tx=0x7f704402a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543da6f966454da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044003b80 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704001a910 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7044003d20 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7044004050 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7057560740 10 client.25333 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564cbc42ea50 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.117+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704001a910 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7044050020 con 0x564cbc04b070 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 10 client.25333 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:13.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:13.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 20 client.25333 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:13.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbc0099f0 0x564cbc431f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 4 client.25333 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:13.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.118+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cbc36adf0 con 0x564cbc0099f0 2024-09-21T01:49:13.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.119+0000 7f704a59c640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbc0099f0 0x564cbc431f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.122+0000 7f704a59c640 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbc0099f0 0x564cbc431f40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816bafb7840e01d7 server_cookie=c6e336694288d1d9 in_seq=0 out_seq=0 2024-09-21T01:49:13.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.122+0000 7f7031ffb640 10 client.25333 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.122+0000 7f7031ffb640 1 -- 172.21.15.29:0/3798594063 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564cbc36adf0 con 0x564cbc0099f0 2024-09-21T01:49:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.122+0000 7f7031ffb640 10 client.25333 handle_command_reply: tid=0 2024-09-21T01:49:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.122+0000 7f7057560740 1 client.25333 shutdown 2024-09-21T01:49:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 msgr2=0x564cbc3fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7044035000 tx=0x7f704402a8b0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 shutdown_connections 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbc0099f0 0x564cbc431f40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc04b070 0x564cbc3fe9c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc408b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 --2- 172.21.15.29:0/3798594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc3fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 >> 172.21.15.29:0/3798594063 conn(0x564cbc03f460 msgr2=0x564cbc3f3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.123+0000 7f7031ffb640 0 client.25333 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.124+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 shutdown_connections 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.124+0000 7f7057560740 1 -- 172.21.15.29:0/3798594063 wait complete. 2024-09-21T01:49:13.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.124+0000 7f7057560740 20 client.25333 trim_cache size 0 max 16384 2024-09-21T01:49:13.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.149+0000 7f7057560740 1 Processor -- start 2024-09-21T01:49:13.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.149+0000 7f7057560740 1 -- start start 2024-09-21T01:49:13.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f704c072ef0 con 0x564cbc02a390 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f704c074b20 con 0x564cbc04b070 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7057560740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0735c0 con 0x564cbc0221b0 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f7049d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f704a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f704ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f704ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54548/0 (socket says 172.21.15.29:54548) 2024-09-21T01:49:13.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.150+0000 7f704ad9d640 1 -- 172.21.15.29:0/3327733804 learned_addr learned my addr 172.21.15.29:0/3327733804 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 -- 172.21.15.29:0/3327733804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 msgr2=0x564cbc3f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 -- 172.21.15.29:0/3327733804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 msgr2=0x564cbc3f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 -- 172.21.15.29:0/3327733804 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc3f43a0 con 0x564cbc0221b0 2024-09-21T01:49:13.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f7049d9b640 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:13.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f704a59c640 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f7510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564cbc404a90 tx=0x7f7040011580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7f04c9102b4227 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.151+0000 7f7033fff640 1 -- 172.21.15.29:0/3327733804 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040031070 con 0x564cbc0221b0 2024-09-21T01:49:13.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.152+0000 7f7033fff640 1 -- 172.21.15.29:0/3327733804 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70400054f0 con 0x564cbc0221b0 2024-09-21T01:49:13.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.152+0000 7f7033fff640 1 -- 172.21.15.29:0/3327733804 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040015040 con 0x564cbc0221b0 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 -- 172.21.15.29:0/3327733804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 msgr2=0x564cbc3f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f7510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564cbc404a90 tx=0x7f7040011580 comp rx=0 tx=0).stop 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 -- 172.21.15.29:0/3327733804 shutdown_connections 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc04b070 0x564cbc3f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc02a390 0x564cbc3f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 --2- 172.21.15.29:0/3327733804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc0221b0 0x564cbc3f7510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.153+0000 7f7057560740 1 -- 172.21.15.29:0/3327733804 >> 172.21.15.29:0/3327733804 conn(0x564cbc03f460 msgr2=0x564cbc3eb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.154+0000 7f7057560740 1 -- 172.21.15.29:0/3327733804 shutdown_connections 2024-09-21T01:49:13.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.154+0000 7f7057560740 1 -- 172.21.15.29:0/3327733804 wait complete. 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.154+0000 7f7057560740 1 -- start start 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 Processor -- start 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc02a390 0x564cbc401520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc04b070 0x564cbc401a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f704c074390 con 0x564cbc04b070 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f704c073cc0 con 0x564cbc0221b0 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f7057560740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f704c070ef0 con 0x564cbc02a390 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.155+0000 7f704a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54554/0 (socket says 172.21.15.29:54554) 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 -- 172.21.15.29:0/3901212680 learned_addr learned my addr 172.21.15.29:0/3901212680 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f7057560740 10 client.0 _ll_register_callbacks cb 0x7f70440303a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704ad9d640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc04b070 0x564cbc401a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f7049d9b640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc02a390 0x564cbc401520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 -- 172.21.15.29:0/3901212680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc02a390 msgr2=0x564cbc401520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc02a390 0x564cbc401520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 -- 172.21.15.29:0/3901212680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc04b070 msgr2=0x564cbc401a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc04b070 0x564cbc401a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.156+0000 7f704a59c640 1 -- 172.21.15.29:0/3901212680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbc402170 con 0x564cbc0221b0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f704a59c640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70400058c0 tx=0x7f70400114b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dbe5101dc79cd84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f7031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040031070 con 0x564cbc0221b0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7040004040 con 0x564cbc0221b0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70400054f0 con 0x564cbc0221b0 2024-09-21T01:49:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.157+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7040015040 con 0x564cbc0221b0 2024-09-21T01:49:13.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.158+0000 7f7057560740 10 client.25280 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:13.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.158+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564cbc49c8d0 con 0x564cbc0221b0 2024-09-21T01:49:13.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040032040 con 0x564cbc0221b0 2024-09-21T01:49:13.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f704003e030 con 0x564cbc0221b0 2024-09-21T01:49:13.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 10 client.25280 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:13.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 20 client.25280 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbbfdeda0 0x564cbc49ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 4 client.25280 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f70400054f0 con 0x564cbbfdeda0 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.160+0000 7f7049d9b640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbbfdeda0 0x564cbc49ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.161+0000 7f7049d9b640 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbbfdeda0 0x564cbc49ede0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145dfd077ba65136 server_cookie=fe14d9741d514ff0 in_seq=0 out_seq=0 2024-09-21T01:49:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7031ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7031ffb640 1 -- 172.21.15.29:0/3901212680 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f70400054f0 con 0x564cbbfdeda0 2024-09-21T01:49:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7031ffb640 10 client.25280 handle_command_reply: tid=0 2024-09-21T01:49:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 client.25280 shutdown 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 msgr2=0x564cbc443e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70400058c0 tx=0x7f70400114b0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 shutdown_connections 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564cbbfdeda0 0x564cbc49ede0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564cbc04b070 0x564cbc401a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564cbc02a390 0x564cbc401520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 --2- 172.21.15.29:0/3901212680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564cbc0221b0 0x564cbc443e00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 >> 172.21.15.29:0/3901212680 conn(0x564cbc03f460 msgr2=0x564cbc3eb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7031ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.162+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 shutdown_connections 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.163+0000 7f7057560740 1 -- 172.21.15.29:0/3901212680 wait complete. 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.163+0000 7f7057560740 20 client.25280 trim_cache size 0 max 16384 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 msgr2=0x7f70340b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 0x7f70340b8b20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70440094e0 tx=0x7f7044030f70 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 msgr2=0x7f704c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f703c005fd0 tx=0x7f703c00dde0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 shutdown_connections 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f70340b6610 0x7f70340b8b20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f704c10dca0 0x7f704c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c105120 0x7f704c19a870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 --2- 172.21.15.29:0/1891667444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f704c1046f0 0x7f704c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 >> 172.21.15.29:0/1891667444 conn(0x7f704c0fe380 msgr2=0x7f704c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 shutdown_connections 2024-09-21T01:49:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:13.164+0000 7f704b59e640 1 -- 172.21.15.29:0/1891667444 wait complete. 2024-09-21T01:49:13.978 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:13 smithi029 ceph-mon[31228]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 841 op/s 2024-09-21T01:49:14.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:13 smithi066 ceph-mon[35001]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 841 op/s 2024-09-21T01:49:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:13 smithi190 ceph-mon[34368]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 841 op/s 2024-09-21T01:49:15.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:49:15.030 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:49:15.498 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:15.936 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:15 smithi029 ceph-mon[31228]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-21T01:49:16.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:15 smithi066 ceph-mon[35001]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 -- 172.21.15.29:0/761941798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b4108d30 msgr2=0x7fa4b410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/761941798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b4108d30 0x7fa4b410f560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009ca0 tx=0x7fa4ac02a8e0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 -- 172.21.15.29:0/761941798 shutdown_connections 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/761941798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b4108d30 0x7fa4b410f560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/761941798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b41083e0 0x7fa4b41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/761941798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 0x7fa4b4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 -- 172.21.15.29:0/761941798 >> 172.21.15.29:0/761941798 conn(0x7fa4b40797e0 msgr2=0x7fa4b4079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 -- 172.21.15.29:0/761941798 shutdown_connections 2024-09-21T01:49:16.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.105+0000 7fa4bc20f640 1 -- 172.21.15.29:0/761941798 wait complete. 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.106+0000 7fa4bc20f640 1 Processor -- start 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 -- start start 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 0x7fa4b410bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 0x7fa4b410c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 0x7fa4b410c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4110b20 con 0x7fa4b4108d30 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41109a0 con 0x7fa4b41083e0 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4bc20f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4110ca0 con 0x7fa4b4102aa0 2024-09-21T01:49:16.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4ba7ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 0x7fa4b410c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4ba7ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 0x7fa4b410c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59132/0 (socket says 172.21.15.29:59132) 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4ba7ac640 1 -- 172.21.15.29:0/1035979773 learned_addr learned my addr 172.21.15.29:0/1035979773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.107+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 0x7fa4b410bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.108+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 0x7fa4b410c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1035979773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 msgr2=0x7fa4b410bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 0x7fa4b410bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 msgr2=0x7fa4b410c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 0x7fa4b410c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1035979773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41acad0 con 0x7fa4b41083e0 2024-09-21T01:49:16.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.109+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 0x7fa4b410c0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800e9c0 tx=0x7fa4a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.110+0000 7fa4b2ffd640 1 -- 172.21.15.29:0/1035979773 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a800ccc0 con 0x7fa4b41083e0 2024-09-21T01:49:16.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.110+0000 7fa4b2ffd640 1 -- 172.21.15.29:0/1035979773 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a80108e0 con 0x7fa4b41083e0 2024-09-21T01:49:16.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.110+0000 7fa4b2ffd640 1 -- 172.21.15.29:0/1035979773 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80187f0 con 0x7fa4b41083e0 2024-09-21T01:49:16.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.110+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41acdc0 con 0x7fa4b41083e0 2024-09-21T01:49:16.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.111+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41ad220 con 0x7fa4b41083e0 2024-09-21T01:49:16.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.115+0000 7fa4c2783740 1 Processor -- start 2024-09-21T01:49:16.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.115+0000 7fa4c2783740 1 -- start start 2024-09-21T01:49:16.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde591b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde591b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde591beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bde5891e60 con 0x55bde554b490 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bde5881290 con 0x55bde5553670 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4c2783740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bde58850a0 con 0x55bde5574350 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4ba7ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde591b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4ba7ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde591b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50016/0 (socket says 172.21.15.29:50016) 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.116+0000 7fa4ba7ac640 1 -- 172.21.15.29:0/3512927670 learned_addr learned my addr 172.21.15.29:0/3512927670 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde591b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b2ffd640 1 -- 172.21.15.29:0/1035979773 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa4a8010430 con 0x7fa4b41083e0 2024-09-21T01:49:16.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b9fab640 1 -- 172.21.15.29:0/3512927670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 msgr2=0x55bde591b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde591b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b9fab640 1 -- 172.21.15.29:0/3512927670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 msgr2=0x55bde591beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.117+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde591beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde591beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b9fab640 1 -- 172.21.15.29:0/3512927670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde55df930 con 0x55bde554b490 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde591beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde591b970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400bbc0 tx=0x7fa4a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d655e00266777491 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b2ffd640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 0x7fa49c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4b2ffd640 1 -- 172.21.15.29:0/1035979773 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4a80e7590 con 0x7fa4b41083e0 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa49bfff640 1 -- 172.21.15.29:0/3512927670 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4004030 con 0x55bde554b490 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa49bfff640 1 -- 172.21.15.29:0/3512927670 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a40041d0 con 0x55bde554b490 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.118+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 0x7fa49c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.119+0000 7fa49bfff640 1 -- 172.21.15.29:0/3512927670 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40116a0 con 0x55bde554b490 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.120+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 0x7fa49c0b8c30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009a60 tx=0x7fa4ac02ade0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.120+0000 7fa4c2783740 1 -- 172.21.15.29:0/3512927670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 msgr2=0x55bde591b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.120+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde591b970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400bbc0 tx=0x7fa4a40107a0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 -- 172.21.15.29:0/3512927670 shutdown_connections 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde591beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde591b970 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3512927670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde591b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 -- 172.21.15.29:0/3512927670 >> 172.21.15.29:0/3512927670 conn(0x55bde5568740 msgr2=0x55bde5919320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 -- 172.21.15.29:0/3512927670 shutdown_connections 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.121+0000 7fa4c2783740 1 -- 172.21.15.29:0/3512927670 wait complete. 2024-09-21T01:49:16.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.122+0000 7fa4c2783740 1 -- start start 2024-09-21T01:49:16.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.122+0000 7fa4c2783740 1 Processor -- start 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.123+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 0x55bde5926ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.123+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde5927530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.123+0000 7fa4b9fab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 0x55bde5926ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.123+0000 7fa4b9fab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 0x55bde5926ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50020/0 (socket says 172.21.15.29:50020) 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4b9fab640 1 -- 172.21.15.29:0/3064430647 learned_addr learned my addr 172.21.15.29:0/3064430647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 0x55bde5932ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bde56e2890 con 0x55bde5574350 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde5927530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bde581bd40 con 0x55bde5553670 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bde5929cb0 con 0x55bde554b490 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 0x55bde5932ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.124+0000 7fa4c2783740 10 client.0 _ll_register_callbacks cb 0x55bde587fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:16.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 -- 172.21.15.29:0/3064430647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 msgr2=0x55bde5926ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 0x55bde5926ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 -- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 msgr2=0x55bde5932ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 0x55bde5932ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde59335e0 con 0x55bde5553670 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde5927530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80ad480 tx=0x7fa4a8010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7e6fb0b9b43c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4997fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4997fa640 1 -- 172.21.15.29:0/3064430647 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80e72e0 con 0x55bde5553670 2024-09-21T01:49:16.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4997fa640 1 -- 172.21.15.29:0/3064430647 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4a80a90d0 con 0x55bde5553670 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a40041d0 con 0x55bde5553670 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.125+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 0x55bde5932ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.126+0000 7fa4997fa640 1 -- 172.21.15.29:0/3064430647 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4a80b70f0 con 0x55bde5553670 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.126+0000 7fa4c2783740 10 client.25289 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.126+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55bde5935e20 con 0x55bde5553670 2024-09-21T01:49:16.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4997fa640 1 -- 172.21.15.29:0/3064430647 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa4a80a92d0 con 0x55bde5553670 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 10 client.25289 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 20 client.25289 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5532cd0 0x55bde5939730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 4 client.25289 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.127+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bde5893db0 con 0x55bde5532cd0 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.128+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5532cd0 0x55bde5939730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.128+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5532cd0 0x55bde5939730 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f8ac3cd00557cc server_cookie=3dc469bb5ce2455b in_seq=0 out_seq=0 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.128+0000 7fa4997fa640 10 client.25289 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.129+0000 7fa4997fa640 1 -- 172.21.15.29:0/3064430647 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bde5893db0 con 0x55bde5532cd0 2024-09-21T01:49:16.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.129+0000 7fa4997fa640 10 client.25289 handle_command_reply: tid=0 2024-09-21T01:49:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.129+0000 7fa4c2783740 1 client.25289 shutdown 2024-09-21T01:49:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 msgr2=0x55bde5927530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde5927530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80ad480 tx=0x7fa4a8010720 comp rx=0 tx=0).stop 2024-09-21T01:49:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 shutdown_connections 2024-09-21T01:49:16.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5532cd0 0x55bde5939730 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5574350 0x55bde5932ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5553670 0x55bde5927530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 --2- 172.21.15.29:0/3064430647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde554b490 0x55bde5926ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 >> 172.21.15.29:0/3064430647 conn(0x55bde5568740 msgr2=0x55bde591e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.130+0000 7fa4997fa640 0 client.25289 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.131+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 shutdown_connections 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.131+0000 7fa4c2783740 1 -- 172.21.15.29:0/3064430647 wait complete. 2024-09-21T01:49:16.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.131+0000 7fa4c2783740 20 client.25289 trim_cache size 0 max 16384 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 -- 172.21.15.29:0/2921508584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 msgr2=0x7f0c28103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 --2- 172.21.15.29:0/2921508584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c28103a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009ca0 tx=0x7f0c2002a8b0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 -- 172.21.15.29:0/2921508584 shutdown_connections 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 --2- 172.21.15.29:0/2921508584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c28103fc0 0x7f0c2806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 --2- 172.21.15.29:0/2921508584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c28103a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 --2- 172.21.15.29:0/2921508584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c281025a0 0x7f0c28102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.153+0000 7f0c2c907640 1 -- 172.21.15.29:0/2921508584 >> 172.21.15.29:0/2921508584 conn(0x7f0c280fdee0 msgr2=0x7f0c28100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.154+0000 7f0c2c907640 1 -- 172.21.15.29:0/2921508584 shutdown_connections 2024-09-21T01:49:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.154+0000 7f0c2c907640 1 -- 172.21.15.29:0/2921508584 wait complete. 2024-09-21T01:49:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 Processor -- start 2024-09-21T01:49:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 -- start start 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c281025a0 0x7f0c2819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c28103fc0 0x7f0c2819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281115c0 con 0x7f0c281025a0 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111440 con 0x7f0c28103fc0 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c2c907640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111740 con 0x7f0c28103620 2024-09-21T01:49:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50034/0 (socket says 172.21.15.29:50034) 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.155+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1234363862 learned_addr learned my addr 172.21.15.29:0/1234363862 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c28103fc0 0x7f0c2819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1234363862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c28103fc0 msgr2=0x7f0c2819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c28103fc0 0x7f0c2819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c281025a0 msgr2=0x7f0c2819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c281025a0 0x7f0c2819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.156+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1234363862 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2819dc50 con 0x7f0c28103620 2024-09-21T01:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.157+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002aea0 tx=0x7f0c20003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.158+0000 7f0c177fe640 1 -- 172.21.15.29:0/1234363862 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004240 con 0x7f0c28103620 2024-09-21T01:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.158+0000 7f0c177fe640 1 -- 172.21.15.29:0/1234363862 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20040070 con 0x7f0c28103620 2024-09-21T01:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.158+0000 7f0c177fe640 1 -- 172.21.15.29:0/1234363862 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20045ce0 con 0x7f0c28103620 2024-09-21T01:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.158+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2819dee0 con 0x7f0c28103620 2024-09-21T01:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.158+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2810cd50 con 0x7f0c28103620 2024-09-21T01:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.162+0000 7f0c33871740 1 Processor -- start 2024-09-21T01:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.162+0000 7f0c33871740 1 -- start start 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc3fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ba9dbb26f0 con 0x55ba9d8981b0 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ba9dba1b20 con 0x55ba9d8774d0 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c33871740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ba9dba5930 con 0x55ba9d86f2f0 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c26d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc3fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54618/0 (socket says 172.21.15.29:54618) 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 -- 172.21.15.29:0/3487236691 learned_addr learned my addr 172.21.15.29:0/3487236691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 -- 172.21.15.29:0/3487236691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 msgr2=0x55ba9dc404c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc404c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 -- 172.21.15.29:0/3487236691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 msgr2=0x55ba9dc3fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc3fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.163+0000 7f0c25d9b640 1 -- 172.21.15.29:0/3487236691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9db67530 con 0x55ba9d8774d0 2024-09-21T01:49:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.164+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000f80 tx=0x7f0c20009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea64cec356f59194 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.166+0000 7f0c177fe640 1 -- 172.21.15.29:0/1234363862 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0c20082020 con 0x7f0c28103620 2024-09-21T01:49:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.166+0000 7f0c14ff9640 1 -- 172.21.15.29:0/3487236691 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2010a3a0 con 0x55ba9d8774d0 2024-09-21T01:49:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.166+0000 7f0c14ff9640 1 -- 172.21.15.29:0/3487236691 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20036030 con 0x55ba9d8774d0 2024-09-21T01:49:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.166+0000 7f0c14ff9640 1 -- 172.21.15.29:0/3487236691 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2010bc90 con 0x55ba9d8774d0 2024-09-21T01:49:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.169+0000 7f0c33871740 1 -- 172.21.15.29:0/3487236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 msgr2=0x55ba9dc41010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.169+0000 7f0c33871740 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000f80 tx=0x7f0c20009a00 comp rx=0 tx=0).stop 2024-09-21T01:49:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.169+0000 7f0c177fe640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 0x7f0c100b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.170+0000 7f0c177fe640 1 -- 172.21.15.29:0/1234363862 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c20056070 con 0x7f0c28103620 2024-09-21T01:49:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.170+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 0x7f0c100b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 0x7f0c100b8b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ab0 tx=0x7f0c1c006f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.169+0000 7fa4c2783740 1 Processor -- start 2024-09-21T01:49:16.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.169+0000 7fa4c2783740 1 -- start start 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 0x55bde5920c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 0x55bde5918860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4b97aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 0x55bde5918860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4b9fab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 0x55bde5920c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4b97aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 0x55bde5918860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59194/0 (socket says 172.21.15.29:59194) 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1749589096 learned_addr learned my addr 172.21.15.29:0/1749589096 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde5918da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b5a00 con 0x55bde5553670 2024-09-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b7630 con 0x55bde554b490 2024-09-21T01:49:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 -- 172.21.15.29:0/3487236691 shutdown_connections 2024-09-21T01:49:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc41010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 --2- 172.21.15.29:0/3487236691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc3fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 -- 172.21.15.29:0/3487236691 >> 172.21.15.29:0/3487236691 conn(0x55ba9d88c300 msgr2=0x55ba9dc3b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.170+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b60d0 con 0x55bde5574350 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde5918da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 -- 172.21.15.29:0/1749589096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 msgr2=0x55bde5918da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde5918da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 -- 172.21.15.29:0/1749589096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 msgr2=0x55bde5918860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 0x55bde5918860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 -- 172.21.15.29:0/1749589096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde591dc30 con 0x55bde554b490 2024-09-21T01:49:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.171+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 0x55bde5920c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55bde5939e70 tx=0x7fa4a4003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfb7438ad293345 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.172+0000 7f0c33871740 1 -- 172.21.15.29:0/3487236691 shutdown_connections 2024-09-21T01:49:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.173+0000 7f0c33871740 1 -- 172.21.15.29:0/3487236691 wait complete. 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.173+0000 7f0c33871740 1 -- start start 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.173+0000 7f0c33871740 1 Processor -- start 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8774d0 0x55ba9dc49850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ba9dad9ab0 con 0x55ba9d8774d0 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ba9db47730 con 0x55ba9d8981b0 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ba9dc418f0 con 0x55ba9d86f2f0 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c33871740 10 client.0 _ll_register_callbacks cb 0x55ba9db9b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.173+0000 7fa49b7fe640 1 -- 172.21.15.29:0/1749589096 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4033070 con 0x55bde554b490 2024-09-21T01:49:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.174+0000 7fa49b7fe640 1 -- 172.21.15.29:0/1749589096 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4003f90 con 0x55bde554b490 2024-09-21T01:49:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.174+0000 7fa49b7fe640 1 -- 172.21.15.29:0/1749589096 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a402e440 con 0x55bde554b490 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50040/0 (socket says 172.21.15.29:50040) 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 -- 172.21.15.29:0/1365876258 learned_addr learned my addr 172.21.15.29:0/1365876258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 -- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 msgr2=0x55ba9dc4a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 -- 172.21.15.29:0/1365876258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8774d0 msgr2=0x55ba9dc49850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8774d0 0x55ba9dc49850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.174+0000 7f0c2659c640 1 -- 172.21.15.29:0/1365876258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9d903790 con 0x55ba9d86f2f0 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.175+0000 7f0c2659c640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800e600 tx=0x7f0c1800eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb138d82f53b6d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.175+0000 7f0c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.175+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18005220 con 0x55ba9d86f2f0 2024-09-21T01:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.175+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c180053c0 con 0x55ba9d86f2f0 2024-09-21T01:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.176+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20036030 con 0x55ba9d86f2f0 2024-09-21T01:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.176+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c18005600 con 0x55ba9d86f2f0 2024-09-21T01:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.177+0000 7f0c33871740 10 client.25363 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.177+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ba9d97c620 con 0x55ba9d86f2f0 2024-09-21T01:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20036030 con 0x55ba9d86f2f0 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 msgr2=0x55bde5920c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 0x55bde5920c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55bde5939e70 tx=0x7fa4a4003d20 comp rx=0 tx=0).stop 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 shutdown_connections 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5574350 0x55bde5918da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde5553670 0x55bde5918860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1749589096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde554b490 0x55bde5920c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 >> 172.21.15.29:0/1749589096 conn(0x55bde5568740 msgr2=0x55bde5920540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 shutdown_connections 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.176+0000 7fa4c2783740 1 -- 172.21.15.29:0/1749589096 wait complete. 2024-09-21T01:49:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.177+0000 7fa4c2783740 1 -- start start 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.178+0000 7fa4c2783740 1 Processor -- start 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.178+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde592b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5574350 0x55bde59a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4b9fab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde592b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4b9fab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde592b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59198/0 (socket says 172.21.15.29:59198) 2024-09-21T01:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0c18022470 con 0x55ba9d86f2f0 2024-09-21T01:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 10 client.25363 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4b97aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 20 client.25363 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9d856d80 0x55ba9dc56ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 4 client.25363 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba9dbb4640 con 0x55ba9d856d80 2024-09-21T01:49:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4b97aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50044/0 (socket says 172.21.15.29:50044) 2024-09-21T01:49:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4b9fab640 1 -- 172.21.15.29:0/1796137289 learned_addr learned my addr 172.21.15.29:0/1796137289 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b6ea0 con 0x55bde554b490 2024-09-21T01:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.179+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9d856d80 0x55ba9dc56ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b67d0 con 0x55bde5574350 2024-09-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41b39e0 con 0x55bde5553670 2024-09-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4c2783740 10 client.0 _ll_register_callbacks cb 0x7fa4a800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.179+0000 7fa4ba7ac640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5574350 0x55bde59a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.181+0000 7f0c0dffb640 10 client.25363 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.181+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9d856d80 0x55ba9dc56ab0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d481cfafdfe8d server_cookie=46f3b84bb9c337b3 in_seq=0 out_seq=0 2024-09-21T01:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.182+0000 7f0c0dffb640 1 -- 172.21.15.29:0/1365876258 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba9dbb4640 con 0x55ba9d856d80 2024-09-21T01:49:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.180+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5574350 msgr2=0x55bde59a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.180+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5574350 0x55bde59a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.180+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1796137289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 msgr2=0x55bde592b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.180+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde592b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.180+0000 7fa4b97aa640 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde592bf80 con 0x55bde5553670 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.181+0000 7fa4b97aa640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800cf10 tx=0x7fa4a800efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691e4de864a232c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.181+0000 7fa4997fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.181+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80ad0d0 con 0x55bde5553670 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.181+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4a80b3340 con 0x55bde5553670 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.181+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a4003f90 con 0x55bde5553670 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4a80ad550 con 0x55bde5553670 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 10 client.25366 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bde59a46c0 con 0x55bde5553670 2024-09-21T01:49:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.182+0000 7f0c0dffb640 10 client.25363 handle_command_reply: tid=0 2024-09-21T01:49:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.182+0000 7f0c33871740 1 client.25363 shutdown 2024-09-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a40041d0 con 0x55bde5553670 2024-09-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa4a80b6720 con 0x55bde5553670 2024-09-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 10 client.25366 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 20 client.25366 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.182+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5508080 0x55bde59a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.183+0000 7fa4c2783740 4 client.25366 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:49:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.183+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa4a4003f90 con 0x55bde5508080 2024-09-21T01:49:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 msgr2=0x55ba9dc49190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800e600 tx=0x7f0c1800eac0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 shutdown_connections 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9d856d80 0x55ba9dc56ab0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8774d0 0x55ba9dc49850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 --2- 172.21.15.29:0/1365876258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d86f2f0 0x55ba9dc49190 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800e600 tx=0x7f0c1800eac0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.183+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 >> 172.21.15.29:0/1365876258 conn(0x55ba9d88c300 msgr2=0x55ba9dc39ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.184+0000 7f0c0dffb640 0 client.25363 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.186+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 shutdown_connections 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.186+0000 7f0c33871740 1 -- 172.21.15.29:0/1365876258 wait complete. 2024-09-21T01:49:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.186+0000 7f0c33871740 20 client.25363 trim_cache size 0 max 16384 2024-09-21T01:49:16.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.183+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5508080 0x55bde59a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.189+0000 7fa4b9fab640 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5508080 0x55bde59a6ee0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8c4601e660ab2f server_cookie=608faf46703b2786 in_seq=0 out_seq=0 2024-09-21T01:49:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.189+0000 7fa4997fa640 10 client.25366 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.191+0000 7fa4997fa640 1 -- 172.21.15.29:0/1796137289 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa4a4003f90 con 0x55bde5508080 2024-09-21T01:49:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.191+0000 7fa4997fa640 10 client.25366 handle_command_reply: tid=0 2024-09-21T01:49:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.191+0000 7fa4c2783740 1 client.25366 shutdown 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 msgr2=0x55bde592b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800cf10 tx=0x7fa4a800efb0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 shutdown_connections 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bde5508080 0x55bde59a6ee0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bde5574350 0x55bde59a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde5553670 0x55bde592b730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4c2783740 1 --2- 172.21.15.29:0/1796137289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bde554b490 0x55bde592b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.194+0000 7fa4997fa640 0 client.25366 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.195+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 >> 172.21.15.29:0/1796137289 conn(0x55bde5568740 msgr2=0x55bde59280b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.196+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 shutdown_connections 2024-09-21T01:49:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.196+0000 7fa4c2783740 1 -- 172.21.15.29:0/1796137289 wait complete. 2024-09-21T01:49:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.197+0000 7fa4c2783740 20 client.25366 trim_cache size 0 max 16384 2024-09-21T01:49:16.198 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:49:16.198 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:49:16.198 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "fab85dd2-3e0e-4f9f-9c88-39d29141ba22", 2024-09-21T01:49:16.199 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:49:16.199 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:49:16.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 msgr2=0x7fa49c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 0x7fa49c0b8c30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009a60 tx=0x7fa4ac02ade0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 msgr2=0x7fa4b410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 0x7fa4b410c0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800e9c0 tx=0x7fa4a800ee80 comp rx=0 tx=0).stop 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 shutdown_connections 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa49c0b6720 0x7fa49c0b8c30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4b4108d30 0x7fa4b410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b41083e0 0x7fa4b410c0b0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800e9c0 tx=0x7fa4a800ee80 comp rx=0 tx=0).stop 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 --2- 172.21.15.29:0/1035979773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4b4102aa0 0x7fa4b410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.201+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 >> 172.21.15.29:0/1035979773 conn(0x7fa4b40797e0 msgr2=0x7fa4b41045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.202+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 shutdown_connections 2024-09-21T01:49:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:16.202+0000 7fa4bc20f640 1 -- 172.21.15.29:0/1035979773 wait complete. 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.225+0000 7f0c33871740 1 Processor -- start 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.225+0000 7f0c33871740 1 -- start start 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b15f0 con 0x55ba9d8981b0 2024-09-21T01:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b3220 con 0x55ba9d8774d0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c33871740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b1cc0 con 0x55ba9d82c1c0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c26d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c26d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50060/0 (socket says 172.21.15.29:50060) 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59214/0 (socket says 172.21.15.29:59214) 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c26d9d640 1 -- 172.21.15.29:0/1584457181 learned_addr learned my addr 172.21.15.29:0/1584457181 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1584457181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 msgr2=0x55ba9dc3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1584457181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 msgr2=0x55ba9dc56a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc56a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c25d9b640 1 -- 172.21.15.29:0/1584457181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9dc3d020 con 0x55ba9d8981b0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.226+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.227+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0c200306c0 tx=0x7f0c2003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecc174056e621fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.227+0000 7f0c0ffff640 1 -- 172.21.15.29:0/1584457181 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2007f040 con 0x55ba9d8981b0 2024-09-21T01:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.227+0000 7f0c0ffff640 1 -- 172.21.15.29:0/1584457181 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20036040 con 0x55ba9d8981b0 2024-09-21T01:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.227+0000 7f0c0ffff640 1 -- 172.21.15.29:0/1584457181 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200d9ca0 con 0x55ba9d8981b0 2024-09-21T01:49:16.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:15 smithi190 ceph-mon[34368]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- 172.21.15.29:0/1584457181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 msgr2=0x55ba9dc365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0c200306c0 tx=0x7f0c2003e070 comp rx=0 tx=0).stop 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- 172.21.15.29:0/1584457181 shutdown_connections 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d8981b0 0x55ba9dc365c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 --2- 172.21.15.29:0/1584457181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8774d0 0x55ba9dc56a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- 172.21.15.29:0/1584457181 >> 172.21.15.29:0/1584457181 conn(0x55ba9d88c300 msgr2=0x55ba9dc40c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- 172.21.15.29:0/1584457181 shutdown_connections 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- 172.21.15.29:0/1584457181 wait complete. 2024-09-21T01:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.228+0000 7f0c33871740 1 -- start start 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 Processor -- start 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc4e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 0x55ba9dc4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc4e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc4e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59230/0 (socket says 172.21.15.29:59230) 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c2659c640 1 -- 172.21.15.29:0/3048695019 learned_addr learned my addr 172.21.15.29:0/3048695019 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b2a90 con 0x55ba9d82c1c0 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b23c0 con 0x55ba9d8981b0 2024-09-21T01:49:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281af5d0 con 0x55ba9d8774d0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 0x55ba9dc4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c33871740 10 client.0 _ll_register_callbacks cb 0x7f0c180095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.229+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 -- 172.21.15.29:0/3048695019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 msgr2=0x55ba9dc4ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 0x55ba9dc4ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 -- 172.21.15.29:0/3048695019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 msgr2=0x55ba9dc4e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc4e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9dc4fd50 con 0x55ba9d8981b0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c25d9b640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 0x55ba9dc4ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c26d9d640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4f5f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009fd0 tx=0x7f0c1c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a38f0dd17b8099d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c02c470 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c1c02c610 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.230+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20036040 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c1c02c810 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c33871740 10 client.25313 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ba9dc510f0 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20036040 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0c1c014970 con 0x55ba9d8981b0 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c33871740 10 client.25313 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c33871740 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.231+0000 7f0c33871740 20 client.25313 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9dcc1870 0x55ba9dcc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c33871740 4 client.25313 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba9dbb00f0 con 0x55ba9dcc1870 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c2659c640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9dcc1870 0x55ba9dcc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c0dffb640 10 client.25313 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.232+0000 7f0c2659c640 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9dcc1870 0x55ba9dcc3c90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0005973cde39d61 server_cookie=432586c71cd215ff in_seq=0 out_seq=0 2024-09-21T01:49:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.290+0000 7f0c0dffb640 1 -- 172.21.15.29:0/3048695019 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba9dbb00f0 con 0x55ba9dcc1870 2024-09-21T01:49:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.290+0000 7f0c0dffb640 10 client.25313 handle_command_reply: tid=0 2024-09-21T01:49:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.290+0000 7f0c33871740 1 client.25313 shutdown 2024-09-21T01:49:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.291+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 msgr2=0x55ba9dc4f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.291+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4f5f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009fd0 tx=0x7f0c1c004710 comp rx=0 tx=0).stop 2024-09-21T01:49:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.291+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 shutdown_connections 2024-09-21T01:49:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.291+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ba9dcc1870 0x55ba9dcc3c90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba9d8981b0 0x55ba9dc4f5f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba9d8774d0 0x55ba9dc4ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 --2- 172.21.15.29:0/3048695019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba9d82c1c0 0x55ba9dc4e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 >> 172.21.15.29:0/3048695019 conn(0x55ba9d88c300 msgr2=0x55ba9dc40c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c0dffb640 0 client.25313 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 shutdown_connections 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 1 -- 172.21.15.29:0/3048695019 wait complete. 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.292+0000 7f0c33871740 20 client.25313 trim_cache size 0 max 16384 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:49:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:49:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:49:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.293+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 msgr2=0x7f0c100b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.293+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 0x7f0c100b8b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ab0 tx=0x7f0c1c006f90 comp rx=0 tx=0).stop 2024-09-21T01:49:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.293+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 msgr2=0x7f0c2819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.293+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002aea0 tx=0x7f0c20003ba0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 shutdown_connections 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c100b6650 0x7f0c100b8b60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c28103fc0 0x7f0c2819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c28103620 0x7f0c2819c880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 --2- 172.21.15.29:0/1234363862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c281025a0 0x7f0c2819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 >> 172.21.15.29:0/1234363862 conn(0x7f0c280fdee0 msgr2=0x7f0c28104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.294+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 shutdown_connections 2024-09-21T01:49:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:16.295+0000 7f0c2c907640 1 -- 172.21.15.29:0/1234363862 wait complete. 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:16 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:17.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:49:17.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:16 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:17.128 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:49:17.128 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:16 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:17.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 40.0 s, 4672.8 tps, lat 21.418 ms stddev 31.890 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: pgmap v388: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 897 op/s 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:49:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:17 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: pgmap v388: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 897 op/s 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:49:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:17 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: pgmap v388: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 897 op/s 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:17 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:19.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:18 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:19.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:18 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:19.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:18 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:49:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:19 smithi066 ceph-mon[35001]: pgmap v389: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-21T01:49:20.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:19 smithi029 ceph-mon[31228]: pgmap v389: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-21T01:49:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:19 smithi190 ceph-mon[34368]: pgmap v389: 97 pgs: 97 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-21T01:49:21.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:49:21.531 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:21.982 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:21 smithi029 ceph-mon[31228]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-09-21T01:49:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:21 smithi066 ceph-mon[35001]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-09-21T01:49:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.146+0000 7fb384ba4640 1 -- 172.21.15.29:0/2644487045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380107c20 msgr2=0x7fb380108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.146+0000 7fb384ba4640 1 --2- 172.21.15.29:0/2644487045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380107c20 0x7fb380108030 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb370009900 tx=0x7fb37002a410 comp rx=0 tx=0).stop 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 -- 172.21.15.29:0/2644487045 shutdown_connections 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 --2- 172.21.15.29:0/2644487045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380107c20 0x7fb380108030 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 --2- 172.21.15.29:0/2644487045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102460 0x7fb3801076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 --2- 172.21.15.29:0/2644487045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 -- 172.21.15.29:0/2644487045 >> 172.21.15.29:0/2644487045 conn(0x7fb380077800 msgr2=0x7fb380077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 -- 172.21.15.29:0/2644487045 shutdown_connections 2024-09-21T01:49:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.147+0000 7fb384ba4640 1 -- 172.21.15.29:0/2644487045 wait complete. 2024-09-21T01:49:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.148+0000 7fb384ba4640 1 Processor -- start 2024-09-21T01:49:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.148+0000 7fb384ba4640 1 -- start start 2024-09-21T01:49:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380102460 0x7fb38006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380107c20 0x7fb38006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010b8f0 con 0x7fb380102460 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010b770 con 0x7fb380101b10 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb384ba4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb380115f90 con 0x7fb380107c20 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53110/0 (socket says 172.21.15.29:53110) 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb37e59c640 1 -- 172.21.15.29:0/786317877 learned_addr learned my addr 172.21.15.29:0/786317877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380107c20 0x7fb38006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.149+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380102460 0x7fb38006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 -- 172.21.15.29:0/786317877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380107c20 msgr2=0x7fb38006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380107c20 0x7fb38006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 -- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380102460 msgr2=0x7fb38006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380102460 0x7fb38006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 -- 172.21.15.29:0/786317877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38006edf0 con 0x7fb380101b10 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb37e59c640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ebc0 tx=0x7fb37800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb36f7fe640 1 -- 172.21.15.29:0/786317877 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378010630 con 0x7fb380101b10 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb36f7fe640 1 -- 172.21.15.29:0/786317877 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb378010c80 con 0x7fb380101b10 2024-09-21T01:49:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.150+0000 7fb36f7fe640 1 -- 172.21.15.29:0/786317877 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378010630 con 0x7fb380101b10 2024-09-21T01:49:22.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.151+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb380073ab0 con 0x7fb380101b10 2024-09-21T01:49:22.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.151+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb380073fa0 con 0x7fb380101b10 2024-09-21T01:49:22.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.155+0000 7fb36f7fe640 1 -- 172.21.15.29:0/786317877 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb3780107d0 con 0x7fb380101b10 2024-09-21T01:49:22.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.157+0000 7fb36f7fe640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 0x7fb3680b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.157+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 0x7fb3680b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.157+0000 7fb36f7fe640 1 -- 172.21.15.29:0/786317877 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb378010430 con 0x7fb380101b10 2024-09-21T01:49:22.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.158+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 0x7fb3680b8e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb374006fd0 tx=0x7fb374008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.159+0000 7fb38b118740 1 Processor -- start 2024-09-21T01:49:22.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.160+0000 7fb38b118740 1 -- start start 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb38b118740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb38b118740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb38b118740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e171390 0x557c2e53a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56928/0 (socket says 172.21.15.29:56928) 2024-09-21T01:49:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb37dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.161+0000 7fb37dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53118/0 (socket says 172.21.15.29:53118) 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.162+0000 7fb37e59c640 1 -- 172.21.15.29:0/3058812086 learned_addr learned my addr 172.21.15.29:0/3058812086 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.162+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e171390 0x557c2e53a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.162+0000 7fb38b118740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557c2e4afe50 con 0x557c2e192070 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.162+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c2e49f280 con 0x557c2e1691b0 2024-09-21T01:49:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.162+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557c2e4a3090 con 0x557c2e171390 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 -- 172.21.15.29:0/3058812086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e171390 msgr2=0x557c2e53a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e171390 0x557c2e53a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 -- 172.21.15.29:0/3058812086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 msgr2=0x557c2e539650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 -- 172.21.15.29:0/3058812086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2e1fd650 con 0x557c2e1691b0 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37e59c640 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:22.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.163+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb380106580 tx=0x7fb374010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b77671e0a494b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.164+0000 7fb36cff9640 1 -- 172.21.15.29:0/3058812086 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374024070 con 0x557c2e1691b0 2024-09-21T01:49:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.164+0000 7fb36cff9640 1 -- 172.21.15.29:0/3058812086 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb374014d70 con 0x557c2e1691b0 2024-09-21T01:49:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.164+0000 7fb36cff9640 1 -- 172.21.15.29:0/3058812086 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37401fe30 con 0x557c2e1691b0 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.165+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 msgr2=0x557c2e539b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.165+0000 7fb38b118740 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb380106580 tx=0x7fb374010000 comp rx=0 tx=0).stop 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.165+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 shutdown_connections 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e171390 0x557c2e53a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e1691b0 0x557c2e539b90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 --2- 172.21.15.29:0/3058812086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e192070 0x557c2e539650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 >> 172.21.15.29:0/3058812086 conn(0x557c2e186460 msgr2=0x557c2e537540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 shutdown_connections 2024-09-21T01:49:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 -- 172.21.15.29:0/3058812086 wait complete. 2024-09-21T01:49:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.166+0000 7fb38b118740 1 -- start start 2024-09-21T01:49:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 Processor -- start 2024-09-21T01:49:22.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e1691b0 0x557c2e54de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 0x557c2e54e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557c2e300700 con 0x557c2e1691b0 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c2e43a110 con 0x557c2e171390 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557c2e547e30 con 0x557c2e192070 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb38b118740 10 client.0 _ll_register_callbacks cb 0x557c2e453990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb37ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb37ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35116/0 (socket says 172.21.15.29:35116) 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.167+0000 7fb37ed9d640 1 -- 172.21.15.29:0/3192995574 learned_addr learned my addr 172.21.15.29:0/3192995574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 0x557c2e54e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 -- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 msgr2=0x557c2e54e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 0x557c2e54e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 -- 172.21.15.29:0/3192995574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e1691b0 msgr2=0x557c2e54de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e1691b0 0x557c2e54de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 -- 172.21.15.29:0/3192995574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2e467c40 con 0x557c2e192070 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 0x557c2e54e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3700096c0 tx=0x7fb370005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afea1a0aa501a82b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.168+0000 7fb365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370035040 con 0x557c2e192070 2024-09-21T01:49:22.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb3700046c0 con 0x557c2e192070 2024-09-21T01:49:22.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb374014d70 con 0x557c2e192070 2024-09-21T01:49:22.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3700048c0 con 0x557c2e192070 2024-09-21T01:49:22.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb38b118740 10 client.25381 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:22.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.169+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c2e53d980 con 0x557c2e192070 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb374014d70 con 0x557c2e192070 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb3700314a0 con 0x557c2e192070 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 10 client.25381 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 20 client.25381 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e1509f0 0x557c2e540f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 4 client.25381 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c2e4b1da0 con 0x557c2e1509f0 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.170+0000 7fb37e59c640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e1509f0 0x557c2e540f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.171+0000 7fb365ffb640 10 client.25381 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.171+0000 7fb37e59c640 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e1509f0 0x557c2e540f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447204282116f85e server_cookie=9382e44b1eaac188 in_seq=0 out_seq=0 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb365ffb640 1 -- 172.21.15.29:0/3192995574 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c2e4b1da0 con 0x557c2e1509f0 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb365ffb640 10 client.25381 handle_command_reply: tid=0 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb38b118740 1 client.25381 shutdown 2024-09-21T01:49:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 msgr2=0x557c2e553430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3700096c0 tx=0x7fb370005250 comp rx=0 tx=0).stop 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 shutdown_connections 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.172+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e1509f0 0x557c2e540f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e192070 0x557c2e553430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e171390 0x557c2e54e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 --2- 172.21.15.29:0/3192995574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e1691b0 0x557c2e54de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 >> 172.21.15.29:0/3192995574 conn(0x557c2e186460 msgr2=0x557c2e53c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb365ffb640 0 client.25381 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 shutdown_connections 2024-09-21T01:49:22.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 1 -- 172.21.15.29:0/3192995574 wait complete. 2024-09-21T01:49:22.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.173+0000 7fb38b118740 20 client.25381 trim_cache size 0 max 16384 2024-09-21T01:49:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.212+0000 7fb38b118740 1 Processor -- start 2024-09-21T01:49:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 -- start start 2024-09-21T01:49:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e5532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e548d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e5532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e5532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35128/0 (socket says 172.21.15.29:35128) 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e5492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb37e59c640 1 -- 172.21.15.29:0/2036618557 learned_addr learned my addr 172.21.15.29:0/2036618557 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a4b00 con 0x557c2e171390 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a6730 con 0x557c2e192070 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.213+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a51d0 con 0x557c2e1691b0 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e548d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e5492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 -- 172.21.15.29:0/2036618557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 msgr2=0x557c2e5532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e5532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 -- 172.21.15.29:0/2036618557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 msgr2=0x557c2e5492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e5492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.214+0000 7fb37dd9b640 1 -- 172.21.15.29:0/2036618557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2e54ecd0 con 0x557c2e171390 2024-09-21T01:49:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.215+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e548d70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb37401fad0 tx=0x7fb37401fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b68c9a5cb86c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.215+0000 7fb367fff640 1 -- 172.21.15.29:0/2036618557 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374024040 con 0x557c2e171390 2024-09-21T01:49:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.216+0000 7fb367fff640 1 -- 172.21.15.29:0/2036618557 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3740148c0 con 0x557c2e171390 2024-09-21T01:49:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.216+0000 7fb367fff640 1 -- 172.21.15.29:0/2036618557 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3740045f0 con 0x557c2e171390 2024-09-21T01:49:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 msgr2=0x557c2e548d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e548d70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb37401fad0 tx=0x7fb37401fd00 comp rx=0 tx=0).stop 2024-09-21T01:49:22.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 shutdown_connections 2024-09-21T01:49:22.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e5492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e548d70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 --2- 172.21.15.29:0/2036618557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e5532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.217+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 >> 172.21.15.29:0/2036618557 conn(0x557c2e186460 msgr2=0x557c2e532280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.218+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 shutdown_connections 2024-09-21T01:49:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.218+0000 7fb38b118740 1 -- 172.21.15.29:0/2036618557 wait complete. 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.218+0000 7fb38b118740 1 -- start start 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.218+0000 7fb38b118740 1 Processor -- start 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e546e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e541aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a5fa0 con 0x557c2e171390 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a58d0 con 0x557c2e192070 2024-09-21T01:49:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb380115cf0 con 0x557c2e1691b0 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb38b118740 10 client.0 _ll_register_callbacks cb 0x7fb370009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb37dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb37dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56952/0 (socket says 172.21.15.29:56952) 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.219+0000 7fb37dd9b640 1 -- 172.21.15.29:0/4182387868 learned_addr learned my addr 172.21.15.29:0/4182387868 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37e59c640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e546e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37ed9d640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e541aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37dd9b640 1 -- 172.21.15.29:0/4182387868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 msgr2=0x557c2e546e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e546e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37dd9b640 1 -- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 msgr2=0x557c2e541aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e541aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.220+0000 7fb37dd9b640 1 -- 172.21.15.29:0/4182387868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2e542140 con 0x557c2e171390 2024-09-21T01:49:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.221+0000 7fb37dd9b640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb374008010 tx=0x7fb3740044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7351f82cc7b2828b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.221+0000 7fb365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.221+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374037070 con 0x557c2e171390 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.221+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb37401f810 con 0x557c2e171390 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.222+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3740148c0 con 0x557c2e171390 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.222+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb374041050 con 0x557c2e171390 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.222+0000 7fb38b118740 10 client.15654 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.222+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c2e5d23d0 con 0x557c2e171390 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb374003b40 con 0x557c2e171390 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb374013070 con 0x557c2e171390 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 10 client.15654 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 20 client.15654 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e125da0 0x557c2e5d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 4 client.15654 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:49:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.223+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb3740148c0 con 0x557c2e125da0 2024-09-21T01:49:22.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.224+0000 7fb37e59c640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e125da0 0x557c2e5d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:21 smithi190 ceph-mon[34368]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-09-21T01:49:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.228+0000 7fb37e59c640 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e125da0 0x557c2e5d48e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd2f7a229b3c586e server_cookie=8eeb9c15e0fcbe81 in_seq=0 out_seq=0 2024-09-21T01:49:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.228+0000 7fb365ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:22.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb365ffb640 1 -- 172.21.15.29:0/4182387868 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x7fb3740148c0 con 0x557c2e125da0 2024-09-21T01:49:22.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb365ffb640 10 client.15654 handle_command_reply: tid=0 2024-09-21T01:49:22.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 client.15654 shutdown 2024-09-21T01:49:22.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 msgr2=0x557c2e541560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb374008010 tx=0x7fb3740044a0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 shutdown_connections 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557c2e125da0 0x557c2e5d48e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c2e192070 0x557c2e541aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557c2e171390 0x557c2e541560 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb365ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 --2- 172.21.15.29:0/4182387868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c2e1691b0 0x557c2e546e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.229+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 >> 172.21.15.29:0/4182387868 conn(0x557c2e186460 msgr2=0x557c2e532280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.230+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 shutdown_connections 2024-09-21T01:49:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.230+0000 7fb38b118740 1 -- 172.21.15.29:0/4182387868 wait complete. 2024-09-21T01:49:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.230+0000 7fb38b118740 20 client.15654 trim_cache size 0 max 16384 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "fab85dd2-3e0e-4f9f-9c88-39d29141ba22", 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:49:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:49:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:49:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:49:22.234 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.232+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 msgr2=0x7fb3680b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.232+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 0x7fb3680b8e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb374006fd0 tx=0x7fb374008040 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.232+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 msgr2=0x7fb380074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.232+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ebc0 tx=0x7fb37800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 shutdown_connections 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3680b6900 0x7fb3680b8e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380107c20 0x7fb38006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb380102460 0x7fb38006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 --2- 172.21.15.29:0/786317877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb380101b10 0x7fb380074de0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 >> 172.21.15.29:0/786317877 conn(0x7fb380077800 msgr2=0x7fb3801008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 shutdown_connections 2024-09-21T01:49:22.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:22.233+0000 7fb384ba4640 1 -- 172.21.15.29:0/786317877 wait complete. 2024-09-21T01:49:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 45.0 s, 4229.2 tps, lat 23.661 ms stddev 46.206 2024-09-21T01:49:23.188 INFO:tasks.cephfs.filesystem:scrub status for tag:fab85dd2-3e0e-4f9f-9c88-39d29141ba22 - {'path': '/', 'tag': 'fab85dd2-3e0e-4f9f-9c88-39d29141ba22', 'options': 'recursive,force'} 2024-09-21T01:49:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:23 smithi029 ceph-mon[31228]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-21T01:49:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:23 smithi190 ceph-mon[34368]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-21T01:49:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:23 smithi066 ceph-mon[35001]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-21T01:49:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:25 smithi029 ceph-mon[31228]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 11 MiB/s wr, 992 op/s 2024-09-21T01:49:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:49:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:49:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:49:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:25 smithi190 ceph-mon[34368]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 11 MiB/s wr, 992 op/s 2024-09-21T01:49:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:49:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:49:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:49:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:25 smithi066 ceph-mon[35001]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 11 MiB/s wr, 992 op/s 2024-09-21T01:49:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:49:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:49:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:49:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:26 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:49:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:26 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:49:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:26 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:49:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:26 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:49:27.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:26 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:49:27.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:26 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:49:27.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 50.0 s, 4875.7 tps, lat 20.496 ms stddev 21.676 2024-09-21T01:49:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:27 smithi029 ceph-mon[31228]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T01:49:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:27 smithi190 ceph-mon[34368]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T01:49:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:27 smithi066 ceph-mon[35001]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T01:49:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.451+0000 7f342d206640 1 -- 172.21.15.29:0/1827388017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 msgr2=0x7f3428103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.451+0000 7f342d206640 1 --2- 172.21.15.29:0/1827388017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428103a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c02a440 comp rx=0 tx=0).stop 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 -- 172.21.15.29:0/1827388017 shutdown_connections 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 --2- 172.21.15.29:0/1827388017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 0x7f342810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 --2- 172.21.15.29:0/1827388017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428103a30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 --2- 172.21.15.29:0/1827388017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 -- 172.21.15.29:0/1827388017 >> 172.21.15.29:0/1827388017 conn(0x7f34280fdee0 msgr2=0x7f3428100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.452+0000 7f342d206640 1 -- 172.21.15.29:0/1827388017 shutdown_connections 2024-09-21T01:49:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.453+0000 7f342d206640 1 -- 172.21.15.29:0/1827388017 wait complete. 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.454+0000 7f342d206640 1 Processor -- start 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.454+0000 7f342d206640 1 -- start start 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.454+0000 7f342d206640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.454+0000 7f342d206640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342d206640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 0x7f3428198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342d206640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3428111570 con 0x7f3428103620 2024-09-21T01:49:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342d206640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34281113f0 con 0x7f3428103fc0 2024-09-21T01:49:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342d206640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f34281116f0 con 0x7f34281025a0 2024-09-21T01:49:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56970/0 (socket says 172.21.15.29:56970) 2024-09-21T01:49:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342659c640 1 -- 172.21.15.29:0/2348943152 learned_addr learned my addr 172.21.15.29:0/2348943152 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f342759e640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 0x7f3428198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 -- 172.21.15.29:0/2348943152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 msgr2=0x7f3428198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 0x7f3428198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 -- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 msgr2=0x7f3428198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.455+0000 7f3426d9d640 1 -- 172.21.15.29:0/2348943152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3428199760 con 0x7f34281025a0 2024-09-21T01:49:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428197d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9f0 tx=0x7f341800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f340ffff640 1 -- 172.21.15.29:0/2348943152 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800ccc0 con 0x7f34281025a0 2024-09-21T01:49:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3428199a50 con 0x7f34281025a0 2024-09-21T01:49:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a0140 con 0x7f34281025a0 2024-09-21T01:49:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f340ffff640 1 -- 172.21.15.29:0/2348943152 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34180108e0 con 0x7f34281025a0 2024-09-21T01:49:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.456+0000 7f340ffff640 1 -- 172.21.15.29:0/2348943152 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418018880 con 0x7f34281025a0 2024-09-21T01:49:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.461+0000 7f343424a740 1 Processor -- start 2024-09-21T01:49:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.461+0000 7f340ffff640 1 -- 172.21.15.29:0/2348943152 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3418018a20 con 0x7f34281025a0 2024-09-21T01:49:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.462+0000 7f343424a740 1 -- start start 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.462+0000 7f343424a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.462+0000 7f343424a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.462+0000 7f343424a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbabf4d0 0x5617dbe89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.462+0000 7f343424a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617dbdfa6f0 con 0x5617dbab72f0 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f343424a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617dbde9b20 con 0x5617dbae01b0 2024-09-21T01:49:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f343424a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617dbded930 con 0x5617dbabf4d0 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f342759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f342759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53178/0 (socket says 172.21.15.29:53178) 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f342759e640 1 -- 172.21.15.29:0/3806286949 learned_addr learned my addr 172.21.15.29:0/3806286949 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f342659c640 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbabf4d0 0x5617dbe89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.463+0000 7f3426d9d640 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 -- 172.21.15.29:0/3806286949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbabf4d0 msgr2=0x5617dbe89010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbabf4d0 0x5617dbe89010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 -- 172.21.15.29:0/3806286949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 msgr2=0x5617dbe884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 -- 172.21.15.29:0/3806286949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dbdaf530 con 0x5617dbae01b0 2024-09-21T01:49:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f340ffff640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 0x7f34100b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 0x7f34100b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f342759e640 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3420009f80 tx=0x7f3420011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ad6160fef62138 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f340ffff640 1 -- 172.21.15.29:0/2348943152 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f34180e7540 con 0x7f34281025a0 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.464+0000 7f340d7fa640 1 -- 172.21.15.29:0/3806286949 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34200129f0 con 0x5617dbae01b0 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.465+0000 7f340d7fa640 1 -- 172.21.15.29:0/3806286949 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3420012c10 con 0x5617dbae01b0 2024-09-21T01:49:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.465+0000 7f342759e640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 0x7f34100b8b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3420009fd0 tx=0x7f342000fa90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.465+0000 7f340d7fa640 1 -- 172.21.15.29:0/3806286949 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342000f760 con 0x5617dbae01b0 2024-09-21T01:49:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.466+0000 7f343424a740 1 -- 172.21.15.29:0/3806286949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 msgr2=0x5617dbe87e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.466+0000 7f343424a740 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3420009f80 tx=0x7f3420011870 comp rx=0 tx=0).stop 2024-09-21T01:49:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 -- 172.21.15.29:0/3806286949 shutdown_connections 2024-09-21T01:49:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbabf4d0 0x5617dbe89010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 --2- 172.21.15.29:0/3806286949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbae01b0 0x5617dbe87e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 -- 172.21.15.29:0/3806286949 >> 172.21.15.29:0/3806286949 conn(0x5617dbad4300 msgr2=0x5617dbe838e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 -- 172.21.15.29:0/3806286949 shutdown_connections 2024-09-21T01:49:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.467+0000 7f343424a740 1 -- 172.21.15.29:0/3806286949 wait complete. 2024-09-21T01:49:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.468+0000 7f343424a740 1 -- start start 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.468+0000 7f343424a740 1 Processor -- start 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.468+0000 7f343424a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.468+0000 7f343424a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 0x5617dbe91c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56980/0 (socket says 172.21.15.29:56980) 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f3426d9d640 1 -- 172.21.15.29:0/2526853282 learned_addr learned my addr 172.21.15.29:0/2526853282 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f342659c640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 0x5617dbe91c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617dbd21ab0 con 0x5617dbab72f0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617dbd8f730 con 0x5617dbabf4d0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617dbe89bd0 con 0x5617dbae01b0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f343424a740 10 client.0 _ll_register_callbacks cb 0x5617dbde36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f342759e640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f342659c640 1 -- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 msgr2=0x5617dbe927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.469+0000 7f342659c640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.470+0000 7f342659c640 1 -- 172.21.15.29:0/2526853282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 msgr2=0x5617dbe915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.470+0000 7f342659c640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.470+0000 7f342659c640 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dbb4b790 con 0x5617dbabf4d0 2024-09-21T01:49:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.470+0000 7f342759e640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.472+0000 7f342659c640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 0x5617dbe91c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f341c02aa30 tx=0x7f341c004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c1a7c17ad338c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.472+0000 7f340aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.472+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c035040 con 0x5617dbabf4d0 2024-09-21T01:49:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.472+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f341c003ac0 con 0x5617dbabf4d0 2024-09-21T01:49:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.473+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3420012c10 con 0x5617dbabf4d0 2024-09-21T01:49:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.474+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f341c003cc0 con 0x5617dbabf4d0 2024-09-21T01:49:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.474+0000 7f343424a740 10 client.25343 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.474+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5617dbbc4620 con 0x5617dbabf4d0 2024-09-21T01:49:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.476+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3420012c10 con 0x5617dbabf4d0 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.476+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f341c03c4e0 con 0x5617dbabf4d0 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 10 client.25343 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 20 client.25343 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dba9ed80 0x5617dbe9ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 4 client.25343 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.477+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617dbdfc640 con 0x5617dba9ed80 2024-09-21T01:49:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.478+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dba9ed80 0x5617dbe9ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.481+0000 7f3426d9d640 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dba9ed80 0x5617dbe9ee60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f3a5255df14800 server_cookie=44d0be8735d77535 in_seq=0 out_seq=0 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.482+0000 7f340aff5640 10 client.25343 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f340aff5640 1 -- 172.21.15.29:0/2526853282 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617dbdfc640 con 0x5617dba9ed80 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f340aff5640 10 client.25343 handle_command_reply: tid=0 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f343424a740 1 client.25343 shutdown 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 msgr2=0x5617dbe91c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 0x5617dbe91c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f341c02aa30 tx=0x7f341c004440 comp rx=0 tx=0).stop 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 shutdown_connections 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.485+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dba9ed80 0x5617dbe9ee60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.486+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.486+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbabf4d0 0x5617dbe91c70 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f341c02aa30 tx=0x7f341c004440 comp rx=0 tx=0).stop 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.486+0000 7f343424a740 1 --2- 172.21.15.29:0/2526853282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.486+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 >> 172.21.15.29:0/2526853282 conn(0x5617dbad4300 msgr2=0x5617dbe88ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.486+0000 7f340aff5640 0 client.25343 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.488+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 shutdown_connections 2024-09-21T01:49:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.488+0000 7f343424a740 1 -- 172.21.15.29:0/2526853282 wait complete. 2024-09-21T01:49:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.489+0000 7f343424a740 20 client.25343 trim_cache size 0 max 16384 2024-09-21T01:49:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.523+0000 7f343424a740 1 Processor -- start 2024-09-21T01:49:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 -- start start 2024-09-21T01:49:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe91af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe89d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f341400e4f0 con 0x5617dba741c0 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3414010120 con 0x5617dbab72f0 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f343424a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f341400ebc0 con 0x5617dbae01b0 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f342759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe89d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53194/0 (socket says 172.21.15.29:53194) 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f342759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe89d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56994/0 (socket says 172.21.15.29:56994) 2024-09-21T01:49:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f3426d9d640 1 -- 172.21.15.29:0/430376249 learned_addr learned my addr 172.21.15.29:0/430376249 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f3426d9d640 1 -- 172.21.15.29:0/430376249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 msgr2=0x5617dbe91af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.524+0000 7f3426d9d640 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe91af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.525+0000 7f3426d9d640 1 -- 172.21.15.29:0/430376249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 msgr2=0x5617dbe89d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.525+0000 7f3426d9d640 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe89d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.525+0000 7f3426d9d640 1 -- 172.21.15.29:0/430376249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dbe89a10 con 0x5617dbab72f0 2024-09-21T01:49:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.525+0000 7f3426d9d640 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34180e50a0 tx=0x7f341801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43db59795c32142d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.525+0000 7f340cff9640 1 -- 172.21.15.29:0/430376249 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180ad030 con 0x5617dbab72f0 2024-09-21T01:49:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.527+0000 7f340cff9640 1 -- 172.21.15.29:0/430376249 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34180ad1f0 con 0x5617dbab72f0 2024-09-21T01:49:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.527+0000 7f340cff9640 1 -- 172.21.15.29:0/430376249 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180ace50 con 0x5617dbab72f0 2024-09-21T01:49:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.528+0000 7f343424a740 1 -- 172.21.15.29:0/430376249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 msgr2=0x5617dbe915b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.528+0000 7f343424a740 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34180e50a0 tx=0x7f341801f000 comp rx=0 tx=0).stop 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 -- 172.21.15.29:0/430376249 shutdown_connections 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe89d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe91af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 --2- 172.21.15.29:0/430376249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe915b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 -- 172.21.15.29:0/430376249 >> 172.21.15.29:0/430376249 conn(0x5617dbad4300 msgr2=0x5617dbe7e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 -- 172.21.15.29:0/430376249 shutdown_connections 2024-09-21T01:49:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.529+0000 7f343424a740 1 -- 172.21.15.29:0/430376249 wait complete. 2024-09-21T01:49:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.530+0000 7f343424a740 1 -- start start 2024-09-21T01:49:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.530+0000 7f343424a740 1 Processor -- start 2024-09-21T01:49:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.531+0000 7f343424a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe96840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.531+0000 7f343424a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.531+0000 7f343424a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe97a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.531+0000 7f342659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53198/0 (socket says 172.21.15.29:53198) 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342659c640 1 -- 172.21.15.29:0/3821499138 learned_addr learned my addr 172.21.15.29:0/3821499138 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f343424a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f341400f990 con 0x5617dba741c0 2024-09-21T01:49:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe97a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f341400f2c0 con 0x5617dbab72f0 2024-09-21T01:49:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f341400c4d0 con 0x5617dbae01b0 2024-09-21T01:49:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f343424a740 10 client.0 _ll_register_callbacks cb 0x7f341c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 -- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 msgr2=0x5617dbe96f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 -- 172.21.15.29:0/3821499138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 msgr2=0x5617dbe96840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe96840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.532+0000 7f342759e640 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dbe981d0 con 0x5617dbae01b0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.533+0000 7f342659c640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.533+0000 7f342759e640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe97a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3420011d70 tx=0x7f3420017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7694dd7a74975e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.533+0000 7f340aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.533+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342002e040 con 0x5617dbae01b0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.533+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34180ad1f0 con 0x5617dbae01b0 2024-09-21T01:49:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.534+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3420022040 con 0x5617dbae01b0 2024-09-21T01:49:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.534+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f342000fe50 con 0x5617dbae01b0 2024-09-21T01:49:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.535+0000 7f343424a740 10 client.25399 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.535+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5617dbe99570 con 0x5617dbae01b0 2024-09-21T01:49:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34180ad1f0 con 0x5617dbae01b0 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f3420031070 con 0x5617dbae01b0 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 10 client.25399 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 20 client.25399 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dbf098e0 0x5617dbf0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 4 client.25399 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:49:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.536+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617dbdf80f0 con 0x5617dbf098e0 2024-09-21T01:49:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.537+0000 7f3426d9d640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dbf098e0 0x5617dbf0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.537+0000 7f340aff5640 10 client.25399 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.538+0000 7f3426d9d640 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dbf098e0 0x5617dbf0bd20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f405f981988244c server_cookie=bba8778d73e61a9 in_seq=0 out_seq=0 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f340aff5640 1 -- 172.21.15.29:0/3821499138 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617dbdf80f0 con 0x5617dbf098e0 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f340aff5640 10 client.25399 handle_command_reply: tid=0 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 client.25399 shutdown 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 msgr2=0x5617dbe97a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe97a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3420011d70 tx=0x7f3420017000 comp rx=0 tx=0).stop 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 shutdown_connections 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617dbf098e0 0x5617dbf0bd20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617dbae01b0 0x5617dbe97a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617dbab72f0 0x5617dbe96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 --2- 172.21.15.29:0/3821499138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617dba741c0 0x5617dbe96840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 >> 172.21.15.29:0/3821499138 conn(0x5617dbad4300 msgr2=0x5617dbe7e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f340aff5640 0 client.25399 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 shutdown_connections 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.562+0000 7f343424a740 1 -- 172.21.15.29:0/3821499138 wait complete. 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.563+0000 7f343424a740 20 client.25399 trim_cache size 0 max 16384 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:49:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 msgr2=0x7f34100b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 0x7f34100b8b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3420009fd0 tx=0x7f342000fa90 comp rx=0 tx=0).stop 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 msgr2=0x7f3428197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428197d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9f0 tx=0x7f341800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 shutdown_connections 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f34100b6650 0x7f34100b8b60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3428103fc0 0x7f3428198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3428103620 0x7f3428198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 --2- 172.21.15.29:0/2348943152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34281025a0 0x7f3428197d20 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9f0 tx=0x7f341800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:49:29.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.564+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 >> 172.21.15.29:0/2348943152 conn(0x7f34280fdee0 msgr2=0x7f342810eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.567+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 shutdown_connections 2024-09-21T01:49:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:29.567+0000 7f342d206640 1 -- 172.21.15.29:0/2348943152 wait complete. 2024-09-21T01:49:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 9.7 MiB/s wr, 901 op/s 2024-09-21T01:49:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:49:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:29 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 9.7 MiB/s wr, 901 op/s 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:29 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 9.7 MiB/s wr, 901 op/s 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:29 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:49:31.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:49:31.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:49:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:49:31.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:31.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:49:31.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:49:31.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:49:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:31 smithi029 ceph-mon[31228]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-09-21T01:49:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:31 smithi190 ceph-mon[34368]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-09-21T01:49:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:31 smithi066 ceph-mon[35001]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-09-21T01:49:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 55.0 s, 4981.4 tps, lat 20.051 ms stddev 21.405 2024-09-21T01:49:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:33 smithi029 ceph-mon[31228]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 812 op/s 2024-09-21T01:49:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:33 smithi190 ceph-mon[34368]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 812 op/s 2024-09-21T01:49:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:33 smithi066 ceph-mon[35001]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 812 op/s 2024-09-21T01:49:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:35 smithi029 ceph-mon[31228]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-21T01:49:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:35 smithi190 ceph-mon[34368]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-21T01:49:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:35 smithi066 ceph-mon[35001]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-21T01:49:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 60.0 s, 5269.1 tps, lat 18.972 ms stddev 20.526 2024-09-21T01:49:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:37 smithi029 ceph-mon[31228]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 999 op/s 2024-09-21T01:49:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:37 smithi190 ceph-mon[34368]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 999 op/s 2024-09-21T01:49:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:37 smithi066 ceph-mon[35001]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 999 op/s 2024-09-21T01:49:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:39 smithi029 ceph-mon[31228]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.6 MiB/s wr, 765 op/s 2024-09-21T01:49:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:39 smithi190 ceph-mon[34368]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.6 MiB/s wr, 765 op/s 2024-09-21T01:49:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:39 smithi066 ceph-mon[35001]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.6 MiB/s wr, 765 op/s 2024-09-21T01:49:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:41 smithi029 ceph-mon[31228]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-21T01:49:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:41 smithi190 ceph-mon[34368]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-21T01:49:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:41 smithi066 ceph-mon[35001]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 -- 172.21.15.29:0/1168194923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 msgr2=0x7fc930104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 --2- 172.21.15.29:0/1168194923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930104b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009860 tx=0x7fc92c02a340 comp rx=0 tx=0).stop 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 -- 172.21.15.29:0/1168194923 shutdown_connections 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 --2- 172.21.15.29:0/1168194923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301050e0 0x7fc93010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 --2- 172.21.15.29:0/1168194923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930104b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 --2- 172.21.15.29:0/1168194923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301036c0 0x7fc930103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.733+0000 7fc93836e640 1 -- 172.21.15.29:0/1168194923 >> 172.21.15.29:0/1168194923 conn(0x7fc9300ff020 msgr2=0x7fc9301014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.734+0000 7fc93836e640 1 -- 172.21.15.29:0/1168194923 shutdown_connections 2024-09-21T01:49:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.734+0000 7fc93836e640 1 -- 172.21.15.29:0/1168194923 wait complete. 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.735+0000 7fc93836e640 1 Processor -- start 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.735+0000 7fc93836e640 1 -- start start 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.735+0000 7fc93836e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 0x7fc930071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.735+0000 7fc93836e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93836e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301050e0 0x7fc930072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93836e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc930076040 con 0x7fc9301036c0 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93836e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc930075ec0 con 0x7fc9301050e0 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93836e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300761c0 con 0x7fc930104740 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc935909640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc935909640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52010/0 (socket says 172.21.15.29:52010) 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc935909640 1 -- 172.21.15.29:0/3908629124 learned_addr learned my addr 172.21.15.29:0/3908629124 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93610a640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 0x7fc930071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc93690b640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301050e0 0x7fc930072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc935909640 1 -- 172.21.15.29:0/3908629124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301050e0 msgr2=0x7fc930072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.736+0000 7fc935909640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301050e0 0x7fc930072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.737+0000 7fc935909640 1 -- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 msgr2=0x7fc930071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.737+0000 7fc935909640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 0x7fc930071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.737+0000 7fc935909640 1 -- 172.21.15.29:0/3908629124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc930072fb0 con 0x7fc930104740 2024-09-21T01:49:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.737+0000 7fc93610a640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 0x7fc930071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.738+0000 7fc935909640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009980 tx=0x7fc92c032d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.738+0000 7fc9237fe640 1 -- 172.21.15.29:0/3908629124 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c042070 con 0x7fc930104740 2024-09-21T01:49:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.738+0000 7fc9237fe640 1 -- 172.21.15.29:0/3908629124 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c030e10 con 0x7fc930104740 2024-09-21T01:49:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.738+0000 7fc9237fe640 1 -- 172.21.15.29:0/3908629124 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c037040 con 0x7fc930104740 2024-09-21T01:49:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.738+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc930073240 con 0x7fc930104740 2024-09-21T01:49:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.739+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc930073550 con 0x7fc930104740 2024-09-21T01:49:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 Processor -- start 2024-09-21T01:49:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 -- start start 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d63d6f0 con 0x55aa4d3231b0 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d62cb20 con 0x55aa4d3024d0 2024-09-21T01:49:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.744+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d630930 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93690b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93690b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34996/0 (socket says 172.21.15.29:34996) 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93690b640 1 -- 172.21.15.29:0/3130262993 learned_addr learned my addr 172.21.15.29:0/3130262993 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 -- 172.21.15.29:0/3130262993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 msgr2=0x55aa4d6cc010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6cc010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 -- 172.21.15.29:0/3130262993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 msgr2=0x55aa4d6cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 -- 172.21.15.29:0/3130262993 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4d5f2530 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.745+0000 7fc93610a640 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6cb4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc9240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a061eab2779099c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.746+0000 7fc920bfe640 1 -- 172.21.15.29:0/3130262993 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924004030 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.746+0000 7fc920bfe640 1 -- 172.21.15.29:0/3130262993 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9240054d0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.746+0000 7fc920bfe640 1 -- 172.21.15.29:0/3130262993 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924004ce0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.746+0000 7fc9237fe640 1 -- 172.21.15.29:0/3908629124 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc92c045070 con 0x7fc930104740 2024-09-21T01:49:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3130262993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 msgr2=0x55aa4d6cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6cb4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc9240118d0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3130262993 shutdown_connections 2024-09-21T01:49:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6cb4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3130262993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.747+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3130262993 >> 172.21.15.29:0/3130262993 conn(0x55aa4d317300 msgr2=0x55aa4d6c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.748+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3130262993 shutdown_connections 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.748+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3130262993 wait complete. 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.749+0000 7fc93f3bd740 1 -- start start 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.749+0000 7fc93f3bd740 1 Processor -- start 2024-09-21T01:49:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.749+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.749+0000 7fc9237fe640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 0x7fc91c0c1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d564ab0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d5d2730 con 0x55aa4d3024d0 2024-09-21T01:49:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55aa4d6ccb30 con 0x55aa4d3231b0 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93690b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93690b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52022/0 (socket says 172.21.15.29:52022) 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93690b640 1 -- 172.21.15.29:0/3870795328 learned_addr learned my addr 172.21.15.29:0/3870795328 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc9237fe640 1 -- 172.21.15.29:0/3908629124 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc92c106a60 con 0x7fc930104740 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.750+0000 7fc93f3bd740 10 client.0 _ll_register_callbacks cb 0x55aa4d6266b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93610a640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 0x7fc91c0c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc935909640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93690b640 1 -- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 msgr2=0x55aa4d6d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93690b640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93690b640 1 -- 172.21.15.29:0/3870795328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d2fa2f0 msgr2=0x55aa4d6d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93690b640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc93690b640 1 -- 172.21.15.29:0/3870795328 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4d38e790 con 0x55aa4d3231b0 2024-09-21T01:49:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.751+0000 7fc935909640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:49:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.752+0000 7fc93610a640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 0x7fc91c0c1350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9240057c0 tx=0x7fc924000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.752+0000 7fc93690b640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc928009f10 tx=0x7fc92800d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b85ba87a31d4a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.753+0000 7fc919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.753+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92800b570 con 0x55aa4d3231b0 2024-09-21T01:49:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.754+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc92800b710 con 0x55aa4d3231b0 2024-09-21T01:49:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.756+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9240054d0 con 0x55aa4d3231b0 2024-09-21T01:49:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.758+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc92800b910 con 0x55aa4d3231b0 2024-09-21T01:49:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.761+0000 7fc93f3bd740 10 client.25411 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.761+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aa4d407620 con 0x55aa4d3231b0 2024-09-21T01:49:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.764+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9240054d0 con 0x55aa4d3231b0 2024-09-21T01:49:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.764+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc92800b040 con 0x55aa4d3231b0 2024-09-21T01:49:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.765+0000 7fc93f3bd740 10 client.25411 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.765+0000 7fc93f3bd740 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.766+0000 7fc93f3bd740 20 client.25411 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.766+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2e1d80 0x55aa4d6e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.766+0000 7fc93f3bd740 4 client.25411 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.766+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa4d63f640 con 0x55aa4d2e1d80 2024-09-21T01:49:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.769+0000 7fc935909640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2e1d80 0x55aa4d6e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.773+0000 7fc935909640 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2e1d80 0x55aa4d6e1f50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a19aaaaadc23973 server_cookie=f4d9b2152d124802 in_seq=0 out_seq=0 2024-09-21T01:49:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.773+0000 7fc919ffb640 10 client.25411 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.775+0000 7fc919ffb640 1 -- 172.21.15.29:0/3870795328 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa4d63f640 con 0x55aa4d2e1d80 2024-09-21T01:49:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.775+0000 7fc919ffb640 10 client.25411 handle_command_reply: tid=0 2024-09-21T01:49:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.775+0000 7fc93f3bd740 1 client.25411 shutdown 2024-09-21T01:49:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 msgr2=0x55aa4d6d58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc928009f10 tx=0x7fc92800d7b0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 shutdown_connections 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2e1d80 0x55aa4d6e1f50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d3231b0 0x55aa4d6d58c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/3870795328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.776+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 >> 172.21.15.29:0/3870795328 conn(0x55aa4d317300 msgr2=0x55aa4d6c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.777+0000 7fc919ffb640 0 client.25411 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.777+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 shutdown_connections 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.777+0000 7fc93f3bd740 1 -- 172.21.15.29:0/3870795328 wait complete. 2024-09-21T01:49:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.777+0000 7fc93f3bd740 20 client.25411 trim_cache size 0 max 16384 2024-09-21T01:49:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.806+0000 7fc93f3bd740 1 Processor -- start 2024-09-21T01:49:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.806+0000 7fc93f3bd740 1 -- start start 2024-09-21T01:49:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.806+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.806+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6c6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52032/0 (socket says 172.21.15.29:52032) 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301a0600 con 0x55aa4d3231b0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301a2230 con 0x55aa4d3024d0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93f3bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301a0cd0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 -- 172.21.15.29:0/1878584693 learned_addr learned my addr 172.21.15.29:0/1878584693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93690b640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc935909640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 -- 172.21.15.29:0/1878584693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 msgr2=0x55aa4d6c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 -- 172.21.15.29:0/1878584693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 msgr2=0x55aa4d6c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93610a640 1 -- 172.21.15.29:0/1878584693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4d6c20a0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.807+0000 7fc93690b640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.808+0000 7fc93610a640 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc924009f30 tx=0x7fc924022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc94fa86bf8a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.809+0000 7fc91bfff640 1 -- 172.21.15.29:0/1878584693 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9240255d0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.809+0000 7fc91bfff640 1 -- 172.21.15.29:0/1878584693 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc924035070 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.809+0000 7fc91bfff640 1 -- 172.21.15.29:0/1878584693 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92402fa90 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.810+0000 7fc93f3bd740 1 -- 172.21.15.29:0/1878584693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 msgr2=0x55aa4d6d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.810+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc924009f30 tx=0x7fc924022000 comp rx=0 tx=0).stop 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 -- 172.21.15.29:0/1878584693 shutdown_connections 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3231b0 0x55aa4d6c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3024d0 0x55aa4d6c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/1878584693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d3ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 -- 172.21.15.29:0/1878584693 >> 172.21.15.29:0/1878584693 conn(0x55aa4d317300 msgr2=0x55aa4d6c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 -- 172.21.15.29:0/1878584693 shutdown_connections 2024-09-21T01:49:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 -- 172.21.15.29:0/1878584693 wait complete. 2024-09-21T01:49:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 -- start start 2024-09-21T01:49:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.811+0000 7fc93f3bd740 1 Processor -- start 2024-09-21T01:49:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3024d0 0x55aa4d6da030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93610a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93610a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52044/0 (socket says 172.21.15.29:52044) 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3231b0 0x55aa4d6daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93610a640 1 -- 172.21.15.29:0/2057518298 learned_addr learned my addr 172.21.15.29:0/2057518298 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301a1aa0 con 0x55aa4d3024d0 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301a13d0 con 0x55aa4d3231b0 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc930112690 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.812+0000 7fc93f3bd740 10 client.0 _ll_register_callbacks cb 0x7fc928009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 -- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3231b0 msgr2=0x55aa4d6daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3231b0 0x55aa4d6daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 -- 172.21.15.29:0/2057518298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3024d0 msgr2=0x55aa4d6da030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3024d0 0x55aa4d6da030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4d6db300 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93610a640 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc924011910 tx=0x7fc924005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddfdda6d922ae1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924040020 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc9240255d0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.813+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc924035070 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9240257d0 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 10 client.25417 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aa4d6dc600 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92404c070 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc924019920 con 0x55aa4d2fa2f0 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 10 client.25417 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 10 client.25417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 20 client.25417 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.814+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2b71c0 0x55aa4d74eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.815+0000 7fc93f3bd740 4 client.25417 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:49:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.815+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa4d63b0f0 con 0x55aa4d2b71c0 2024-09-21T01:49:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.815+0000 7fc935909640 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2b71c0 0x55aa4d74eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.816+0000 7fc935909640 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2b71c0 0x55aa4d74eb10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42d72fdf6fe9124 server_cookie=3a81bc574ce1f71b in_seq=0 out_seq=0 2024-09-21T01:49:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.816+0000 7fc919ffb640 10 client.25417 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.831+0000 7fc919ffb640 1 -- 172.21.15.29:0/2057518298 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa4d63b0f0 con 0x55aa4d2b71c0 2024-09-21T01:49:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.831+0000 7fc919ffb640 10 client.25417 handle_command_reply: tid=0 2024-09-21T01:49:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.831+0000 7fc93f3bd740 1 client.25417 shutdown 2024-09-21T01:49:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.831+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 msgr2=0x55aa4d6d9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc924011910 tx=0x7fc924005cc0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 shutdown_connections 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55aa4d2b71c0 0x55aa4d74eb10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa4d3231b0 0x55aa4d6daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55aa4d3024d0 0x55aa4d6da030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 --2- 172.21.15.29:0/2057518298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa4d2fa2f0 0x55aa4d6d9970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 >> 172.21.15.29:0/2057518298 conn(0x55aa4d317300 msgr2=0x55aa4d6c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc919ffb640 0 client.25417 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 shutdown_connections 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 1 -- 172.21.15.29:0/2057518298 wait complete. 2024-09-21T01:49:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.832+0000 7fc93f3bd740 20 client.25417 trim_cache size 0 max 16384 2024-09-21T01:49:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:49:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:49:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:49:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 msgr2=0x7fc91c0c1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 0x7fc91c0c1350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9240057c0 tx=0x7fc924000bc0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 msgr2=0x7fc930071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009980 tx=0x7fc92c032d40 comp rx=0 tx=0).stop 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 shutdown_connections 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc91c0bee40 0x7fc91c0c1350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9301050e0 0x7fc930072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc930104740 0x7fc930071be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 --2- 172.21.15.29:0/3908629124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9301036c0 0x7fc930071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.833+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 >> 172.21.15.29:0/3908629124 conn(0x7fc9300ff020 msgr2=0x7fc9300ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.835+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 shutdown_connections 2024-09-21T01:49:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:42.835+0000 7fc93836e640 1 -- 172.21.15.29:0/3908629124 wait complete. 2024-09-21T01:49:42.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 65.0 s, 4799.2 tps, lat 20.833 ms stddev 21.181 2024-09-21T01:49:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:43 smithi029 ceph-mon[31228]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-09-21T01:49:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:43 smithi190 ceph-mon[34368]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-09-21T01:49:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:43 smithi066 ceph-mon[35001]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-09-21T01:49:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:45 smithi190 ceph-mon[34368]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-09-21T01:49:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:45 smithi066 ceph-mon[35001]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-09-21T01:49:46.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:45 smithi029 ceph-mon[31228]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-09-21T01:49:47.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 70.0 s, 4950.4 tps, lat 20.218 ms stddev 21.273 2024-09-21T01:49:48.116 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:47 smithi029 ceph-mon[31228]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 953 op/s 2024-09-21T01:49:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:47 smithi190 ceph-mon[34368]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 953 op/s 2024-09-21T01:49:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:47 smithi066 ceph-mon[35001]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 953 op/s 2024-09-21T01:49:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:49 smithi190 ceph-mon[34368]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 698 op/s 2024-09-21T01:49:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:49 smithi066 ceph-mon[35001]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 698 op/s 2024-09-21T01:49:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:49 smithi029 ceph-mon[31228]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 698 op/s 2024-09-21T01:49:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:51 smithi190 ceph-mon[34368]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-21T01:49:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:51 smithi066 ceph-mon[35001]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-21T01:49:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:51 smithi029 ceph-mon[31228]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-21T01:49:52.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 75.0 s, 5384.2 tps, lat 18.553 ms stddev 19.755 2024-09-21T01:49:53.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:49:53.626 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:53 smithi029 ceph-mon[31228]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 920 op/s 2024-09-21T01:49:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:53 smithi190 ceph-mon[34368]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 920 op/s 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.276+0000 7f5677fff640 1 -- 172.21.15.29:0/2403431205 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c030c00 con 0x7f5684109e20 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 -- 172.21.15.29:0/2403431205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684109e20 msgr2=0x7f5684110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 --2- 172.21.15.29:0/2403431205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684109e20 0x7f5684110650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ca0 tx=0x7f567c02a8b0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 -- 172.21.15.29:0/2403431205 shutdown_connections 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 --2- 172.21.15.29:0/2403431205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684109e20 0x7f5684110650 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 --2- 172.21.15.29:0/2403431205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56841094d0 0x7f56841098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 --2- 172.21.15.29:0/2403431205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5684103b90 0x7f5684103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 -- 172.21.15.29:0/2403431205 >> 172.21.15.29:0/2403431205 conn(0x7f56840ff600 msgr2=0x7f5684101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 -- 172.21.15.29:0/2403431205 shutdown_connections 2024-09-21T01:49:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.277+0000 7f568b9a5640 1 -- 172.21.15.29:0/2403431205 wait complete. 2024-09-21T01:49:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.278+0000 7f568b9a5640 1 Processor -- start 2024-09-21T01:49:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 -- start start 2024-09-21T01:49:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684103b90 0x7f5684074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56841094d0 0x7f568406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5684111900 con 0x7f5684109e20 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5684111780 con 0x7f56841094d0 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.279+0000 7f568b9a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5684111a80 con 0x7f5684103b90 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.280+0000 7f5688f40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56841094d0 0x7f568406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.280+0000 7f5689f42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.280+0000 7f5689f42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40732/0 (socket says 172.21.15.29:40732) 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.280+0000 7f5689f42640 1 -- 172.21.15.29:0/1012686150 learned_addr learned my addr 172.21.15.29:0/1012686150 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.280+0000 7f5689741640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684103b90 0x7f5684074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 -- 172.21.15.29:0/1012686150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684103b90 msgr2=0x7f5684074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684103b90 0x7f5684074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 -- 172.21.15.29:0/1012686150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56841094d0 msgr2=0x7f568406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56841094d0 0x7f568406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 -- 172.21.15.29:0/1012686150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568406ed10 con 0x7f5684109e20 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f5689f42640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f567c003ba0 tx=0x7f567c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f56767fc640 1 -- 172.21.15.29:0/1012686150 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c004240 con 0x7f5684109e20 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f56767fc640 1 -- 172.21.15.29:0/1012686150 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f567c040070 con 0x7f5684109e20 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f56767fc640 1 -- 172.21.15.29:0/1012686150 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c045b10 con 0x7f5684109e20 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5684073a30 con 0x7f5684109e20 2024-09-21T01:49:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.281+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5684073f20 con 0x7f5684109e20 2024-09-21T01:49:54.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.285+0000 7f56767fc640 1 -- 172.21.15.29:0/1012686150 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f567c03b430 con 0x7f5684109e20 2024-09-21T01:49:54.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.285+0000 7f5691f19740 1 Processor -- start 2024-09-21T01:49:54.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 -- start start 2024-09-21T01:49:54.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706bf390 0x55de70a880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55de709fde50 con 0x55de706e0070 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de709ed280 con 0x55de706b71b0 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5691f19740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55de709f1090 con 0x55de706bf390 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5689741640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5689741640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39824/0 (socket says 172.21.15.29:39824) 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.286+0000 7f5689741640 1 -- 172.21.15.29:0/2812412734 learned_addr learned my addr 172.21.15.29:0/2812412734 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5688f40640 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706bf390 0x55de70a880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 -- 172.21.15.29:0/2812412734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706bf390 msgr2=0x55de70a880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706bf390 0x55de70a880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 -- 172.21.15.29:0/2812412734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 msgr2=0x55de70a87650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 -- 172.21.15.29:0/2812412734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7074b650 con 0x55de706b71b0 2024-09-21T01:49:54.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.287+0000 7f5689741640 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f568000bd10 tx=0x7f56800107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449d851570a3c5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f56767fc640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 0x7f56700b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f56767fc640 1 -- 172.21.15.29:0/1012686150 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f567c108380 con 0x7f5684109e20 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f5688f40640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 0x7f56700b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f566f7fe640 1 -- 172.21.15.29:0/2812412734 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680004030 con 0x55de706b71b0 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f566f7fe640 1 -- 172.21.15.29:0/2812412734 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56800035f0 con 0x55de706b71b0 2024-09-21T01:49:54.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.288+0000 7f566f7fe640 1 -- 172.21.15.29:0/2812412734 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800117b0 con 0x55de706b71b0 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 -- 172.21.15.29:0/2812412734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 msgr2=0x55de70a87b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f568000bd10 tx=0x7f56800107a0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5688f40640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 0x7f56700b8b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5684102e50 tx=0x7f567800a870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 -- 172.21.15.29:0/2812412734 shutdown_connections 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706bf390 0x55de70a880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706b71b0 0x55de70a87b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 --2- 172.21.15.29:0/2812412734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.289+0000 7f5691f19740 1 -- 172.21.15.29:0/2812412734 >> 172.21.15.29:0/2812412734 conn(0x55de706d4460 msgr2=0x55de70a85540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.291+0000 7f5691f19740 1 -- 172.21.15.29:0/2812412734 shutdown_connections 2024-09-21T01:49:54.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.291+0000 7f5691f19740 1 -- 172.21.15.29:0/2812412734 wait complete. 2024-09-21T01:49:54.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.292+0000 7f5691f19740 1 -- start start 2024-09-21T01:49:54.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.292+0000 7f5691f19740 1 Processor -- start 2024-09-21T01:49:54.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a92d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a9f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55de7084e700 con 0x55de706b71b0 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de70988110 con 0x55de706bf390 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55de70a95e30 con 0x55de706e0070 2024-09-21T01:49:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.293+0000 7f5691f19740 10 client.0 _ll_register_callbacks cb 0x55de709a1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:54.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5688f40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5688f40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39834/0 (socket says 172.21.15.29:39834) 2024-09-21T01:49:54.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5688f40640 1 -- 172.21.15.29:0/1043359979 learned_addr learned my addr 172.21.15.29:0/1043359979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:54.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5689f42640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a9f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5689741640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a92d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.294+0000 7f5689f42640 1 -- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 msgr2=0x55de70a932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5689f42640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5689f42640 1 -- 172.21.15.29:0/1043359979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 msgr2=0x55de70a92d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5689f42640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a92d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5689f42640 1 -- 172.21.15.29:0/1043359979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de709b5c40 con 0x55de706e0070 2024-09-21T01:49:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5688f40640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f5689f42640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a9f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f567c006010 tx=0x7f567c0cde10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f6506eae5d6ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.295+0000 7f566cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.296+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0d7c50 con 0x55de706e0070 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.296+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f567c0c8bc0 con 0x55de706e0070 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.296+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56800035f0 con 0x55de706e0070 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f567c0cebc0 con 0x55de706e0070 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 10 client.25429 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de70aa1bd0 con 0x55de706e0070 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56800035f0 con 0x55de706e0070 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f567c107040 con 0x55de706e0070 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 10 client.25429 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 20 client.25429 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de7069e9f0 0x55de70aa5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 4 client.25429 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.297+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de709ffda0 con 0x55de7069e9f0 2024-09-21T01:49:54.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:53 smithi066 ceph-mon[35001]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 920 op/s 2024-09-21T01:49:54.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.299+0000 7f5689741640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de7069e9f0 0x55de70aa5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.302+0000 7f5689741640 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de7069e9f0 0x55de70aa5180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df91f1f64f1e711 server_cookie=21a10e0a22bdd2af in_seq=0 out_seq=0 2024-09-21T01:49:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.302+0000 7f566cff9640 10 client.25429 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:54.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.305+0000 7f566cff9640 1 -- 172.21.15.29:0/1043359979 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de709ffda0 con 0x55de7069e9f0 2024-09-21T01:49:54.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.305+0000 7f566cff9640 10 client.25429 handle_command_reply: tid=0 2024-09-21T01:49:54.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.305+0000 7f5691f19740 1 client.25429 shutdown 2024-09-21T01:49:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 msgr2=0x55de70a9f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a9f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f567c006010 tx=0x7f567c0cde10 comp rx=0 tx=0).stop 2024-09-21T01:49:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 shutdown_connections 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de7069e9f0 0x55de70aa5180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a9f1b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 --2- 172.21.15.29:0/1043359979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a92d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 >> 172.21.15.29:0/1043359979 conn(0x55de706d4460 msgr2=0x55de70a8a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.313+0000 7f566cff9640 0 client.25429 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.314+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 shutdown_connections 2024-09-21T01:49:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.314+0000 7f5691f19740 1 -- 172.21.15.29:0/1043359979 wait complete. 2024-09-21T01:49:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.314+0000 7f5691f19740 20 client.25429 trim_cache size 0 max 16384 2024-09-21T01:49:54.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 Processor -- start 2024-09-21T01:49:54.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 -- start start 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f566800e3c0 con 0x55de706b71b0 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5691f19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f566800fff0 con 0x55de706bf390 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.345+0000 7f5689741640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5691f19740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f566800ea90 con 0x55de706e0070 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689741640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a93230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40768/0 (socket says 172.21.15.29:40768) 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689741640 1 -- 172.21.15.29:0/3310657175 learned_addr learned my addr 172.21.15.29:0/3310657175 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5688f40640 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 -- 172.21.15.29:0/3310657175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 msgr2=0x55de70a9f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a9f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 -- 172.21.15.29:0/3310657175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 msgr2=0x55de70a93230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a93230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 -- 172.21.15.29:0/3310657175 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de70a94030 con 0x55de706e0070 2024-09-21T01:49:54.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.346+0000 7f5689f42640 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a8ff60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f567c0cd9b0 tx=0x7f567c0cdf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429f9c0deeff4f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.347+0000 7f566effd640 1 -- 172.21.15.29:0/3310657175 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0cdc30 con 0x55de706e0070 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.347+0000 7f566effd640 1 -- 172.21.15.29:0/3310657175 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f567c036040 con 0x55de706e0070 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.347+0000 7f566effd640 1 -- 172.21.15.29:0/3310657175 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0d7a50 con 0x55de706e0070 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 -- 172.21.15.29:0/3310657175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 msgr2=0x55de70a8ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a8ff60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f567c0cd9b0 tx=0x7f567c0cdf40 comp rx=0 tx=0).stop 2024-09-21T01:49:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 -- 172.21.15.29:0/3310657175 shutdown_connections 2024-09-21T01:49:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706e0070 0x55de70a8ff60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a9f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 --2- 172.21.15.29:0/3310657175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706b71b0 0x55de70a93230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.348+0000 7f5691f19740 1 -- 172.21.15.29:0/3310657175 >> 172.21.15.29:0/3310657175 conn(0x55de706d4460 msgr2=0x55de70a92b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.350+0000 7f5691f19740 1 -- 172.21.15.29:0/3310657175 shutdown_connections 2024-09-21T01:49:54.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.350+0000 7f5691f19740 1 -- 172.21.15.29:0/3310657175 wait complete. 2024-09-21T01:49:54.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.351+0000 7f5691f19740 1 -- start start 2024-09-21T01:49:54.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.351+0000 7f5691f19740 1 Processor -- start 2024-09-21T01:49:54.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.352+0000 7f5691f19740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 0x55de70a97220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.352+0000 7f5691f19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.352+0000 7f5691f19740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a97ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5691f19740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f566800f860 con 0x55de706e0070 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5689741640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 0x55de70a97220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5689741640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 0x55de70a97220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50206/0 (socket says 172.21.15.29:50206) 2024-09-21T01:49:54.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5689741640 1 -- 172.21.15.29:0/1845851470 learned_addr learned my addr 172.21.15.29:0/1845851470 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:54.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f566800f190 con 0x55de706bf390 2024-09-21T01:49:54.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f566800c3a0 con 0x55de706b71b0 2024-09-21T01:49:54.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.353+0000 7f5691f19740 10 client.0 _ll_register_callbacks cb 0x7f567c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:54.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.354+0000 7f5689f42640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a97ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.354+0000 7f5688f40640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.355+0000 7f5689f42640 1 -- 172.21.15.29:0/1845851470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 msgr2=0x55de70a97220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.355+0000 7f5689f42640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 0x55de70a97220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.355+0000 7f5689f42640 1 -- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 msgr2=0x55de70a97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:54.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.355+0000 7f5689f42640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a97760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.355+0000 7f5689f42640 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de70b0ec70 con 0x55de706e0070 2024-09-21T01:49:54.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f5689f42640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a97ca0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f567c02ae70 tx=0x7f567c003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608a9a97d52de190 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:54.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f566cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:49:54.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0e8070 con 0x55de706e0070 2024-09-21T01:49:54.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f567c0d7c60 con 0x55de706e0070 2024-09-21T01:49:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c036040 con 0x55de706e0070 2024-09-21T01:49:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.356+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f567c0cdc30 con 0x55de706e0070 2024-09-21T01:49:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.357+0000 7f5691f19740 10 client.15693 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.357+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de70b0ec70 con 0x55de706e0070 2024-09-21T01:49:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.357+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f567c0ef020 con 0x55de706e0070 2024-09-21T01:49:54.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.357+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f567c107070 con 0x55de706e0070 2024-09-21T01:49:54.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 10 client.15693 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:54.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:54.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 20 client.15693 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:54.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de70673da0 0x55de70b12370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:54.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 4 client.15693 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:49:54.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f567c036040 con 0x55de70673da0 2024-09-21T01:49:54.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.358+0000 7f5689741640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de70673da0 0x55de70b12370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:54.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.359+0000 7f5689741640 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de70673da0 0x55de70b12370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7475519eb2afb4aa server_cookie=7a3a57832fa75c22 in_seq=0 out_seq=0 2024-09-21T01:49:54.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.359+0000 7f566cff9640 10 client.15693 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:54.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f566cff9640 1 -- 172.21.15.29:0/1845851470 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x7f567c036040 con 0x55de70673da0 2024-09-21T01:49:54.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f566cff9640 10 client.15693 handle_command_reply: tid=0 2024-09-21T01:49:54.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 client.15693 shutdown 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 msgr2=0x55de70a97ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a97ca0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f567c02ae70 tx=0x7f567c003f00 comp rx=0 tx=0).stop 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 shutdown_connections 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55de70673da0 0x55de70b12370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de706e0070 0x55de70a97ca0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de706bf390 0x55de70a97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f566cff9640 0 client.15693 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 --2- 172.21.15.29:0/1845851470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de706b71b0 0x55de70a97220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.360+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 >> 172.21.15.29:0/1845851470 conn(0x55de706d4460 msgr2=0x55de70a94200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.361+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 shutdown_connections 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.361+0000 7f5691f19740 1 -- 172.21.15.29:0/1845851470 wait complete. 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.361+0000 7f5691f19740 20 client.15693 trim_cache size 0 max 16384 2024-09-21T01:49:54.363 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:49:54.364 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:49:54.365 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 msgr2=0x7f56700b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 0x7f56700b8b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5684102e50 tx=0x7f567800a870 comp rx=0 tx=0).stop 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 msgr2=0x7f568406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f567c003ba0 tx=0x7f567c003bd0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 shutdown_connections 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f56700b6610 0x7f56700b8b20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5684109e20 0x7f568406e5d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56841094d0 0x7f568406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 --2- 172.21.15.29:0/1012686150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5684103b90 0x7f5684074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 >> 172.21.15.29:0/1012686150 conn(0x7f56840ff600 msgr2=0x7f5684108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 shutdown_connections 2024-09-21T01:49:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:54.363+0000 7f568b9a5640 1 -- 172.21.15.29:0/1012686150 wait complete. 2024-09-21T01:49:55.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:49:55.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:49:55.715 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.003+0000 7f053ec07640 1 -- 172.21.15.29:0/2339820684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053806c040 msgr2=0x7f053810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.003+0000 7f053ec07640 1 --2- 172.21.15.29:0/2339820684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053806c040 0x7f053810a5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0528009860 tx=0x7f052802a340 comp rx=0 tx=0).stop 2024-09-21T01:49:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.012+0000 7f053ec07640 1 -- 172.21.15.29:0/2339820684 shutdown_connections 2024-09-21T01:49:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.012+0000 7f053ec07640 1 --2- 172.21.15.29:0/2339820684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053810ade0 0x7f053810b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.012+0000 7f053ec07640 1 --2- 172.21.15.29:0/2339820684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810bf40 0x7f0538110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.012+0000 7f053ec07640 1 --2- 172.21.15.29:0/2339820684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053806c040 0x7f053810a5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.012+0000 7f053ec07640 1 -- 172.21.15.29:0/2339820684 >> 172.21.15.29:0/2339820684 conn(0x7f0538070c70 msgr2=0x7f0538073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.013+0000 7f053ec07640 1 -- 172.21.15.29:0/2339820684 shutdown_connections 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.014+0000 7f053ec07640 1 -- 172.21.15.29:0/2339820684 wait complete. 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.015+0000 7f053ec07640 1 Processor -- start 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.015+0000 7f053ec07640 1 -- start start 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053806c040 0x7f05381a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810ade0 0x7f05381a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0538119d60 con 0x7f053810ade0 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0538119be0 con 0x7f053806c040 2024-09-21T01:49:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053ec07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0538119ee0 con 0x7f053810bf40 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50232/0 (socket says 172.21.15.29:50232) 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 -- 172.21.15.29:0/2794332323 learned_addr learned my addr 172.21.15.29:0/2794332323 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f0537fff640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810ade0 0x7f05381a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 -- 172.21.15.29:0/2794332323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053806c040 msgr2=0x7f05381a05c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053806c040 0x7f05381a05c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 -- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810ade0 msgr2=0x7f05381a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810ade0 0x7f05381a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.016+0000 7f053d1a4640 1 -- 172.21.15.29:0/2794332323 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05381a1f70 con 0x7f053810bf40 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.017+0000 7f053d1a4640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e940 tx=0x7f052c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.017+0000 7f0535ffb640 1 -- 172.21.15.29:0/2794332323 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c00cd80 con 0x7f053810bf40 2024-09-21T01:49:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.018+0000 7f0535ffb640 1 -- 172.21.15.29:0/2794332323 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c0035f0 con 0x7f053810bf40 2024-09-21T01:49:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.018+0000 7f0535ffb640 1 -- 172.21.15.29:0/2794332323 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c010690 con 0x7f053810bf40 2024-09-21T01:49:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.018+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05381a2260 con 0x7f053810bf40 2024-09-21T01:49:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.018+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381a8800 con 0x7f053810bf40 2024-09-21T01:49:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.028+0000 7f0545c4c740 1 Processor -- start 2024-09-21T01:49:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 -- start start 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f182f0 0x562d122e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562d1225b6f0 con 0x562d11f411b0 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562d1224ab20 con 0x562d11f204d0 2024-09-21T01:49:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.031+0000 7f0545c4c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562d1224e930 con 0x562d11f182f0 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.032+0000 7f0535ffb640 1 -- 172.21.15.29:0/2794332323 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f052c004030 con 0x7f053810bf40 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.033+0000 7f0535ffb640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 0x7f05200b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.033+0000 7f0535ffb640 1 -- 172.21.15.29:0/2794332323 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f052c0e7600 con 0x7f053810bf40 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39874/0 (socket says 172.21.15.29:39874) 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 -- 172.21.15.29:0/900673571 learned_addr learned my addr 172.21.15.29:0/900673571 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 -- 172.21.15.29:0/900673571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f182f0 msgr2=0x562d122e94c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f182f0 0x562d122e94c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 -- 172.21.15.29:0/900673571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 msgr2=0x562d122e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.034+0000 7f0537fff640 1 -- 172.21.15.29:0/900673571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d12210530 con 0x562d11f204d0 2024-09-21T01:49:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.039+0000 7f053d1a4640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 0x7f05200b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.039+0000 7f0537fff640 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05300113e0 tx=0x7f05300118a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563d6cc45c6b5d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.040+0000 7f051effd640 1 -- 172.21.15.29:0/900673571 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530012980 con 0x562d11f204d0 2024-09-21T01:49:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.040+0000 7f051effd640 1 -- 172.21.15.29:0/900673571 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530016030 con 0x562d11f204d0 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 -- 172.21.15.29:0/900673571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 msgr2=0x562d122ea010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05300113e0 tx=0x7f05300118a0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 -- 172.21.15.29:0/900673571 shutdown_connections 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f204d0 0x562d122ea010 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f182f0 0x562d122e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 --2- 172.21.15.29:0/900673571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 -- 172.21.15.29:0/900673571 >> 172.21.15.29:0/900673571 conn(0x562d11f35300 msgr2=0x562d122e48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 -- 172.21.15.29:0/900673571 shutdown_connections 2024-09-21T01:49:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.042+0000 7f0545c4c740 1 -- 172.21.15.29:0/900673571 wait complete. 2024-09-21T01:49:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.043+0000 7f0545c4c740 1 -- start start 2024-09-21T01:49:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.043+0000 7f0545c4c740 1 Processor -- start 2024-09-21T01:49:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.043+0000 7f053d1a4640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 0x7f05200b8b20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f052c004770 tx=0x7f052c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f182f0 0x562d122f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f411b0 0x562d122f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562d12182ab0 con 0x562d11f182f0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562d121f0730 con 0x562d11f411b0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562d122eab80 con 0x562d11f204d0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0545c4c740 10 client.0 _ll_register_callbacks cb 0x562d122446b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f411b0 0x562d122f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f053c9a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f053c9a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50260/0 (socket says 172.21.15.29:50260) 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.044+0000 7f053c9a3640 1 -- 172.21.15.29:0/1065898341 learned_addr learned my addr 172.21.15.29:0/1065898341 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 -- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f411b0 msgr2=0x562d122f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f411b0 0x562d122f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 -- 172.21.15.29:0/1065898341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f182f0 msgr2=0x562d122f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f182f0 0x562d122f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 -- 172.21.15.29:0/1065898341 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d11fac790 con 0x562d11f204d0 2024-09-21T01:49:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.045+0000 7f053c9a3640 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05280304b0 tx=0x7f0528009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d591d84ddddbdc1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0517fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528037040 con 0x562d11f204d0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0528030690 con 0x562d11f204d0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0530016030 con 0x562d11f204d0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0528030890 con 0x562d11f204d0 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0545c4c740 10 client.25444 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.046+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562d12025620 con 0x562d11f204d0 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530016030 con 0x562d11f204d0 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f052803a070 con 0x562d11f204d0 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 10 client.25444 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 20 client.25444 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d11effd80 0x562d12300400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 4 client.25444 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.047+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1225d640 con 0x562d11effd80 2024-09-21T01:49:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.048+0000 7f0537fff640 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d11effd80 0x562d12300400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.049+0000 7f0517fff640 10 client.25444 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.049+0000 7f0537fff640 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d11effd80 0x562d12300400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de3c33a50bf1760 server_cookie=bb675eac915670c8 in_seq=0 out_seq=0 2024-09-21T01:49:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0517fff640 1 -- 172.21.15.29:0/1065898341 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d1225d640 con 0x562d11effd80 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0517fff640 10 client.25444 handle_command_reply: tid=0 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 client.25444 shutdown 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 msgr2=0x562d122f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05280304b0 tx=0x7f0528009520 comp rx=0 tx=0).stop 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 shutdown_connections 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d11effd80 0x562d12300400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f411b0 0x562d122f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122f2d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 --2- 172.21.15.29:0/1065898341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f182f0 0x562d122f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0517fff640 0 client.25444 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 >> 172.21.15.29:0/1065898341 conn(0x562d11f35300 msgr2=0x562d122e2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 shutdown_connections 2024-09-21T01:49:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 1 -- 172.21.15.29:0/1065898341 wait complete. 2024-09-21T01:49:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.050+0000 7f0545c4c740 20 client.25444 trim_cache size 0 max 16384 2024-09-21T01:49:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 Processor -- start 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 -- start start 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39892/0 (socket says 172.21.15.29:39892) 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 -- 172.21.15.29:0/3961730375 learned_addr learned my addr 172.21.15.29:0/3961730375 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f052400e590 con 0x562d11f411b0 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05240101c0 con 0x562d11f182f0 2024-09-21T01:49:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f052400ec60 con 0x562d11f204d0 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 -- 172.21.15.29:0/3961730375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 msgr2=0x562d122e4220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122e4220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 -- 172.21.15.29:0/3961730375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 msgr2=0x562d122f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.075+0000 7f0537fff640 1 -- 172.21.15.29:0/3961730375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d122f1650 con 0x562d11f182f0 2024-09-21T01:49:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0537fff640 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05300113e0 tx=0x7f053000fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98abb155fd98459 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f051e7fc640 1 -- 172.21.15.29:0/3961730375 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053000f940 con 0x562d11f182f0 2024-09-21T01:49:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f051e7fc640 1 -- 172.21.15.29:0/3961730375 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530016030 con 0x562d11f182f0 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 msgr2=0x562d122f1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05300113e0 tx=0x7f053000fcb0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 shutdown_connections 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f411b0 0x562d122f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f204d0 0x562d122e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- 172.21.15.29:0/3961730375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11f182f0 0x562d122f1110 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05300113e0 tx=0x7f053000fcb0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 >> 172.21.15.29:0/3961730375 conn(0x562d11f35300 msgr2=0x562d122df7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 shutdown_connections 2024-09-21T01:49:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- 172.21.15.29:0/3961730375 wait complete. 2024-09-21T01:49:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- start start 2024-09-21T01:49:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 Processor -- start 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 0x562d122f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f411b0 0x562d122f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 0x562d122f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f052400fa30 con 0x562d11f204d0 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f052400f360 con 0x562d11ed51c0 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f052400c570 con 0x562d11f411b0 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0545c4c740 10 client.0 _ll_register_callbacks cb 0x7f0528035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0537fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 0x562d122f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0537fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 0x562d122f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0537fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 0x562d122f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40830/0 (socket says 172.21.15.29:40830) 2024-09-21T01:49:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.095+0000 7f0537fff640 1 -- 172.21.15.29:0/2448273673 learned_addr learned my addr 172.21.15.29:0/2448273673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 -- 172.21.15.29:0/2448273673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f411b0 msgr2=0x562d122f7600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f411b0 0x562d122f7600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 -- 172.21.15.29:0/2448273673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 msgr2=0x562d122f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 0x562d122f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 -- 172.21.15.29:0/2448273673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d122f88d0 con 0x562d11ed51c0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0537fff640 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 0x562d122f8170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f053000f600 tx=0x7f0530005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c96d8d4b491749d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053003b040 con 0x562d11ed51c0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0530016030 con 0x562d11ed51c0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f053000f940 con 0x562d11ed51c0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0530047050 con 0x562d11ed51c0 2024-09-21T01:49:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 10 client.25391 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562d122f9bc0 con 0x562d11ed51c0 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530016030 con 0x562d11ed51c0 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0530012780 con 0x562d11ed51c0 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 10 client.25391 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 20 client.25391 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d122fa400 0x562d1236ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 4 client.25391 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:49:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.100+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d122590f0 con 0x562d122fa400 2024-09-21T01:49:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.114+0000 7f053c9a3640 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d122fa400 0x562d1236ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.114+0000 7f0517fff640 10 client.25391 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.114+0000 7f053c9a3640 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d122fa400 0x562d1236ce10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5e9e16543f93bb server_cookie=bc03cb764a3941cd in_seq=0 out_seq=0 2024-09-21T01:49:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0517fff640 1 -- 172.21.15.29:0/2448273673 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d122590f0 con 0x562d122fa400 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0517fff640 10 client.25391 handle_command_reply: tid=0 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 client.25391 shutdown 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 msgr2=0x562d122f8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 0x562d122f8170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f053000f600 tx=0x7f0530005bd0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 shutdown_connections 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562d122fa400 0x562d1236ce10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562d11ed51c0 0x562d122f8170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562d11f411b0 0x562d122f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 --2- 172.21.15.29:0/2448273673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d11f204d0 0x562d122f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 >> 172.21.15.29:0/2448273673 conn(0x562d11f35300 msgr2=0x562d122e4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0517fff640 0 client.25391 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 shutdown_connections 2024-09-21T01:49:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.126+0000 7f0545c4c740 1 -- 172.21.15.29:0/2448273673 wait complete. 2024-09-21T01:49:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.127+0000 7f0545c4c740 20 client.25391 trim_cache size 0 max 16384 2024-09-21T01:49:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:49:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:49:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:49:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.127+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 msgr2=0x7f05200b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.127+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 0x7f05200b8b20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f052c004770 tx=0x7f052c0046c0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.127+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 msgr2=0x7f05381a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.127+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e940 tx=0x7f052c00ee00 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 shutdown_connections 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05200b6610 0x7f05200b8b20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f053810bf40 0x7f05381a17e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f053810ade0 0x7f05381a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 --2- 172.21.15.29:0/2794332323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053806c040 0x7f05381a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 >> 172.21.15.29:0/2794332323 conn(0x7f0538070c70 msgr2=0x7f0538072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 shutdown_connections 2024-09-21T01:49:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:49:56.128+0000 7f053ec07640 1 -- 172.21.15.29:0/2794332323 wait complete. 2024-09-21T01:49:56.179 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:55 smithi029 ceph-mon[31228]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-21T01:49:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:55 smithi190 ceph-mon[34368]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-21T01:49:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:55 smithi066 ceph-mon[35001]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.326+0000 7f5165bd8640 1 -- 172.21.15.29:0/477262641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 msgr2=0x7f516010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.326+0000 7f5165bd8640 1 --2- 172.21.15.29:0/477262641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f516010d8b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5154009900 tx=0x7f515402e810 comp rx=0 tx=0).stop 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 -- 172.21.15.29:0/477262641 shutdown_connections 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 --2- 172.21.15.29:0/477262641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 0x7f5160110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 --2- 172.21.15.29:0/477262641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f516010d8b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 --2- 172.21.15.29:0/477262641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 -- 172.21.15.29:0/477262641 >> 172.21.15.29:0/477262641 conn(0x7f5160079730 msgr2=0x7f5160079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 -- 172.21.15.29:0/477262641 shutdown_connections 2024-09-21T01:49:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.327+0000 7f5165bd8640 1 -- 172.21.15.29:0/477262641 wait complete. 2024-09-21T01:49:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.328+0000 7f5165bd8640 1 Processor -- start 2024-09-21T01:49:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.328+0000 7f5165bd8640 1 -- start start 2024-09-21T01:49:56.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.328+0000 7f5165bd8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.328+0000 7f5165bd8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f5160198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.328+0000 7f5165bd8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 0x7f516019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f5165bd8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5160111880 con 0x7f5160105270 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f5165bd8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5160111700 con 0x7f5160104840 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f5165bd8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5160111a00 con 0x7f516010ddf0 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39920/0 (socket says 172.21.15.29:39920) 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 -- 172.21.15.29:0/2390679375 learned_addr learned my addr 172.21.15.29:0/2390679375 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515effd640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f5160198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515ffff640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 0x7f516019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 -- 172.21.15.29:0/2390679375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 msgr2=0x7f516019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 0x7f516019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 -- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 msgr2=0x7f5160198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f5160198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 -- 172.21.15.29:0/2390679375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516019e200 con 0x7f5160104840 2024-09-21T01:49:56.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515f7fe640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9c0 tx=0x7f515000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515000cd80 con 0x7f5160104840 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.329+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516019e4f0 con 0x7f5160104840 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.330+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51500035f0 con 0x7f5160104840 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.330+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51601a4c50 con 0x7f5160104840 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.330+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150010690 con 0x7f5160104840 2024-09-21T01:49:56.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.331+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5160104c50 con 0x7f5160104840 2024-09-21T01:49:56.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.333+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5150004020 con 0x7f5160104840 2024-09-21T01:49:56.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.334+0000 7f515cff9640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 0x7f51480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:56.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.335+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f51500e7620 con 0x7f5160104840 2024-09-21T01:49:56.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.335+0000 7f515effd640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 0x7f51480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:56.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.336+0000 7f515effd640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 0x7f51480b8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5154006010 tx=0x7f5154000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:56.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.339+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51500adec0 con 0x7f5160104840 2024-09-21T01:49:56.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.714+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51601993c0 con 0x7f5160104840 2024-09-21T01:49:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.716+0000 7f515cff9640 1 -- 172.21.15.29:0/2390679375 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11978 (secure 0 0 0) 0x7f51500b22a0 con 0x7f5160104840 2024-09-21T01:49:56.718 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:49:56.718 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":31,"btime":"2024-09-21T01:49:01:726666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:49:01.247055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:49:56.718 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 31 2024-09-21T01:49:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 msgr2=0x7f51480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 0x7f51480b8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5154006010 tx=0x7f5154000bc0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 msgr2=0x7f5160198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9c0 tx=0x7f515000ee80 comp rx=0 tx=0).stop 2024-09-21T01:49:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 shutdown_connections 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f51480b6590 0x7f51480b8aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f516010ddf0 0x7f516019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5160105270 0x7f5160198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 --2- 172.21.15.29:0/2390679375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5160104840 0x7f5160198290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 >> 172.21.15.29:0/2390679375 conn(0x7f5160079730 msgr2=0x7f51600ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 shutdown_connections 2024-09-21T01:49:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:56.720+0000 7f5165bd8640 1 -- 172.21.15.29:0/2390679375 wait complete. 2024-09-21T01:49:57.097 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:56 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2390679375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:57.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:56 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2390679375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:57.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:56 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2390679375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:49:57.663 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:49:57.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 80.0 s, 3501.0 tps, lat 28.439 ms stddev 29.761 2024-09-21T01:49:58.062 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:49:58.190 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:57 smithi029 ceph-mon[31228]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 933 op/s 2024-09-21T01:49:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:57 smithi190 ceph-mon[34368]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 933 op/s 2024-09-21T01:49:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:57 smithi066 ceph-mon[35001]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 933 op/s 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.664+0000 7f5305d9b640 1 -- 172.21.15.29:0/1039752745 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300037070 con 0x7f5308108cf0 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.664+0000 7f530cb5b640 1 -- 172.21.15.29:0/1039752745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308108cf0 msgr2=0x7f530810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.664+0000 7f530cb5b640 1 --2- 172.21.15.29:0/1039752745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308108cf0 0x7f530810f520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5300009860 tx=0x7f530002a340 comp rx=0 tx=0).stop 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 -- 172.21.15.29:0/1039752745 shutdown_connections 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 --2- 172.21.15.29:0/1039752745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308108cf0 0x7f530810f520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 --2- 172.21.15.29:0/1039752745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f53081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 --2- 172.21.15.29:0/1039752745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5308102a60 0x7f5308102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 -- 172.21.15.29:0/1039752745 >> 172.21.15.29:0/1039752745 conn(0x7f5308079730 msgr2=0x7f5308079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 -- 172.21.15.29:0/1039752745 shutdown_connections 2024-09-21T01:49:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.666+0000 7f530cb5b640 1 -- 172.21.15.29:0/1039752745 wait complete. 2024-09-21T01:49:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.668+0000 7f530cb5b640 1 Processor -- start 2024-09-21T01:49:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.668+0000 7f530cb5b640 1 -- start start 2024-09-21T01:49:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f530819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5308108cf0 0x7f53081a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5308111930 con 0x7f53081083a0 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53081117b0 con 0x7f5308102a60 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530cb5b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5308111ab0 con 0x7f5308108cf0 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39944/0 (socket says 172.21.15.29:39944) 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.669+0000 7f530659c640 1 -- 172.21.15.29:0/811071468 learned_addr learned my addr 172.21.15.29:0/811071468 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:58.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f530819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 -- 172.21.15.29:0/811071468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5308108cf0 msgr2=0x7f53081a1ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:49:58.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5308108cf0 0x7f53081a1ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 -- 172.21.15.29:0/811071468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 msgr2=0x7f530819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f52fffff640 1 -- 172.21.15.29:0/811071468 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53081a25d0 con 0x7f53081083a0 2024-09-21T01:49:58.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.670+0000 7f530659c640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:49:58.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.671+0000 7f52fffff640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f530819cc00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52f000b3c0 tx=0x7f52f000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.672+0000 7f52ff7fe640 1 -- 172.21.15.29:0/811071468 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0011830 con 0x7f53081083a0 2024-09-21T01:49:58.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.672+0000 7f52ff7fe640 1 -- 172.21.15.29:0/811071468 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f0016070 con 0x7f53081083a0 2024-09-21T01:49:58.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.672+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53081a28c0 con 0x7f53081083a0 2024-09-21T01:49:58.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.672+0000 7f52ff7fe640 1 -- 172.21.15.29:0/811071468 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0010dc0 con 0x7f53081083a0 2024-09-21T01:49:58.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.672+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53081a9020 con 0x7f53081083a0 2024-09-21T01:49:58.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.676+0000 7f53130cf740 1 Processor -- start 2024-09-21T01:49:58.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.677+0000 7f52ff7fe640 1 -- 172.21.15.29:0/811071468 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f52f001b4a0 con 0x7f53081083a0 2024-09-21T01:49:58.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 -- start start 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab96bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab96c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5620ab8e1ea0 con 0x5620ab5a3390 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f53130cf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5620ab8d12d0 con 0x5620ab59b1b0 2024-09-21T01:49:58.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f5306d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f5306d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50314/0 (socket says 172.21.15.29:50314) 2024-09-21T01:49:58.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.678+0000 7f5306d9d640 1 -- 172.21.15.29:0/3549737381 learned_addr learned my addr 172.21.15.29:0/3549737381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:58.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5620ab8d50e0 con 0x5620ab5c4070 2024-09-21T01:49:58.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f52fffff640 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab96c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 -- 172.21.15.29:0/3549737381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 msgr2=0x5620ab96bbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab96bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 -- 172.21.15.29:0/3549737381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 msgr2=0x5620ab96c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab96c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 -- 172.21.15.29:0/3549737381 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ab62f650 con 0x5620ab5c4070 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.679+0000 7f5306d9d640 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5300035040 tx=0x7f5300009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac1d898cddff8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f52ff7fe640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 0x7f52f40b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f52fcff9640 1 -- 172.21.15.29:0/3549737381 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530003f020 con 0x5620ab5c4070 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f530659c640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 0x7f52f40b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f52ff7fe640 1 -- 172.21.15.29:0/811071468 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52f00e6770 con 0x7f53081083a0 2024-09-21T01:49:58.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f52fcff9640 1 -- 172.21.15.29:0/3549737381 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5300030340 con 0x5620ab5c4070 2024-09-21T01:49:58.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.680+0000 7f52fcff9640 1 -- 172.21.15.29:0/3549737381 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300037040 con 0x5620ab5c4070 2024-09-21T01:49:58.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 msgr2=0x5620ab96b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5300035040 tx=0x7f5300009450 comp rx=0 tx=0).stop 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 shutdown_connections 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab96c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab96bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 --2- 172.21.15.29:0/3549737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5c4070 0x5620ab96b6a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.681+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 >> 172.21.15.29:0/3549737381 conn(0x5620ab5b8460 msgr2=0x5620ab969590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.682+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 shutdown_connections 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.682+0000 7f530659c640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 0x7f52f40b8bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5620ab96a320 tx=0x7f52f80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.682+0000 7f53130cf740 1 -- 172.21.15.29:0/3549737381 wait complete. 2024-09-21T01:49:58.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.683+0000 7f53130cf740 1 -- start start 2024-09-21T01:49:58.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.683+0000 7f53130cf740 1 Processor -- start 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab977280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab98b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5620ab732700 con 0x5620ab5c4070 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5620ab86c160 con 0x5620ab59b1b0 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5620ab979e20 con 0x5620ab5a3390 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f53130cf740 10 client.0 _ll_register_callbacks cb 0x5620ab8859e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f52fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab977280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39946/0 (socket says 172.21.15.29:39946) 2024-09-21T01:49:58.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f52fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab977280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50330/0 (socket says 172.21.15.29:50330) 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 -- 172.21.15.29:0/2457104806 learned_addr learned my addr 172.21.15.29:0/2457104806 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f5306d9d640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab98b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 -- 172.21.15.29:0/2457104806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 msgr2=0x5620ab977280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab977280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 -- 172.21.15.29:0/2457104806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 msgr2=0x5620ab98b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab98b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.685+0000 7f530659c640 1 -- 172.21.15.29:0/2457104806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ab899c90 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.686+0000 7f530659c640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52f801c6b0 tx=0x7f52f8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ba4efdcc4732a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.686+0000 7f52edffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.687+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f802a060 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.687+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5300030340 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.687+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52f801c0d0 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.687+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52f8025410 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 10 client.25406 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5620ab98e360 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f801a8c0 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f52f800b040 con 0x5620ab59b1b0 2024-09-21T01:49:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 10 client.25406 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 10 client.25406 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 20 client.25406 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab5829f0 0x5620ab991c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 4 client.25406 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5300030340 con 0x5620ab5829f0 2024-09-21T01:49:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.688+0000 7f52fffff640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab5829f0 0x5620ab991c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.689+0000 7f52edffb640 10 client.25406 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.689+0000 7f52fffff640 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab5829f0 0x5620ab991c00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=efc12b85a71fad27 server_cookie=81fe38782a640b92 in_seq=0 out_seq=0 2024-09-21T01:49:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.690+0000 7f52edffb640 1 -- 172.21.15.29:0/2457104806 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5300030340 con 0x5620ab5829f0 2024-09-21T01:49:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.690+0000 7f52edffb640 10 client.25406 handle_command_reply: tid=0 2024-09-21T01:49:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.690+0000 7f53130cf740 1 client.25406 shutdown 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 msgr2=0x5620ab976d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52f801c6b0 tx=0x7f52f8004ab0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 shutdown_connections 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab5829f0 0x5620ab991c00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab98b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab977280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 --2- 172.21.15.29:0/2457104806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab976d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 >> 172.21.15.29:0/2457104806 conn(0x5620ab5b8460 msgr2=0x5620ab96e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.691+0000 7f52edffb640 0 client.25406 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.692+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 shutdown_connections 2024-09-21T01:49:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.692+0000 7f53130cf740 1 -- 172.21.15.29:0/2457104806 wait complete. 2024-09-21T01:49:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.692+0000 7f53130cf740 20 client.25406 trim_cache size 0 max 16384 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.717+0000 7f53130cf740 1 Processor -- start 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.717+0000 7f53130cf740 1 -- start start 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab98b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab5c4070 0x5620ab973f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f530659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f530659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50344/0 (socket says 172.21.15.29:50344) 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f530659c640 1 -- 172.21.15.29:0/725374130 learned_addr learned my addr 172.21.15.29:0/725374130 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f5306d9d640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab5c4070 0x5620ab973f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53081ad1e0 con 0x5620ab5a3390 2024-09-21T01:49:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53081aee10 con 0x5620ab5c4070 2024-09-21T01:49:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53081ad8b0 con 0x5620ab59b1b0 2024-09-21T01:49:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.718+0000 7f52fffff640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab98b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 -- 172.21.15.29:0/725374130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab5c4070 msgr2=0x5620ab973f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab5c4070 0x5620ab973f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 -- 172.21.15.29:0/725374130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 msgr2=0x5620ab98b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab98b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 -- 172.21.15.29:0/725374130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ab978000 con 0x5620ab59b1b0 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f52fffff640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab98b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.719+0000 7f530659c640 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52f8030640 tx=0x7f52f800cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4866e1c99348b19c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.720+0000 7f52effff640 1 -- 172.21.15.29:0/725374130 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f801c0d0 con 0x5620ab59b1b0 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.720+0000 7f52effff640 1 -- 172.21.15.29:0/725374130 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f801c270 con 0x5620ab59b1b0 2024-09-21T01:49:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.720+0000 7f52effff640 1 -- 172.21.15.29:0/725374130 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8025d60 con 0x5620ab59b1b0 2024-09-21T01:49:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 msgr2=0x5620ab977200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52f8030640 tx=0x7f52f800cfd0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 shutdown_connections 2024-09-21T01:49:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab5c4070 0x5620ab973f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5a3390 0x5620ab98b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 --2- 172.21.15.29:0/725374130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab59b1b0 0x5620ab977200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.721+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 >> 172.21.15.29:0/725374130 conn(0x5620ab5b8460 msgr2=0x5620ab976af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.722+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 shutdown_connections 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.722+0000 7f53130cf740 1 -- 172.21.15.29:0/725374130 wait complete. 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.722+0000 7f53130cf740 1 -- start start 2024-09-21T01:49:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.722+0000 7f53130cf740 1 Processor -- start 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab9837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab983ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f530659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39956/0 (socket says 172.21.15.29:39956) 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f530659c640 1 -- 172.21.15.29:0/868402904 learned_addr learned my addr 172.21.15.29:0/868402904 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53081ae680 con 0x5620ab5c4070 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53081adfb0 con 0x5620ab59b1b0 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53081ab280 con 0x5620ab5a3390 2024-09-21T01:49:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.723+0000 7f53130cf740 10 client.0 _ll_register_callbacks cb 0x7f52f80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f52fffff640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab9837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f530659c640 1 -- 172.21.15.29:0/868402904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 msgr2=0x5620ab9837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f530659c640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab9837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f530659c640 1 -- 172.21.15.29:0/868402904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 msgr2=0x5620ab983ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f530659c640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab983ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.724+0000 7f530659c640 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ab9fb600 con 0x5620ab59b1b0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f530659c640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52f800a010 tx=0x7f52f8025840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d345cd17ca3d1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f52edffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f801c0d0 con 0x5620ab59b1b0 2024-09-21T01:49:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52f8022040 con 0x5620ab59b1b0 2024-09-21T01:49:58.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620ab8e3df0 con 0x5620ab59b1b0 2024-09-21T01:49:58.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.725+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52f8030400 con 0x5620ab59b1b0 2024-09-21T01:49:58.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.726+0000 7f53130cf740 10 client.25412 fetch_fsmap learned FSMap version 31 2024-09-21T01:49:58.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.726+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5620ab9fc720 con 0x5620ab59b1b0 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.726+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f803e070 con 0x5620ab59b1b0 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.726+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f52f800b070 con 0x5620ab59b1b0 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 10 client.25412 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 10 client.25412 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 20 client.25412 populate_metadata read hostname 'smithi029' 2024-09-21T01:49:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab557da0 0x5620ab9fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 4 client.25412 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620ab8e3df0 con 0x5620ab557da0 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.727+0000 7f52fffff640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab557da0 0x5620ab9fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f52edffb640 10 client.25412 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f52fffff640 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab557da0 0x5620ab9fed40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f98537634bddbf58 server_cookie=a05e02e376a9b04d in_seq=0 out_seq=0 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f52edffb640 1 -- 172.21.15.29:0/868402904 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620ab8e3df0 con 0x5620ab557da0 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f52edffb640 10 client.25412 handle_command_reply: tid=0 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f53130cf740 1 client.25412 shutdown 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 msgr2=0x5620ab9882b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52f800a010 tx=0x7f52f8025840 comp rx=0 tx=0).stop 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 shutdown_connections 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.728+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5620ab557da0 0x5620ab9fed40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f52edffb640 0 client.25412 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620ab5c4070 0x5620ab983ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620ab5a3390 0x5620ab9837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 --2- 172.21.15.29:0/868402904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620ab59b1b0 0x5620ab9882b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 >> 172.21.15.29:0/868402904 conn(0x5620ab5b8460 msgr2=0x5620ab96c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 shutdown_connections 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 1 -- 172.21.15.29:0/868402904 wait complete. 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.729+0000 7f53130cf740 20 client.25412 trim_cache size 0 max 16384 2024-09-21T01:49:58.731 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.730+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 msgr2=0x7f52f40b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.730+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 0x7f52f40b8bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5620ab96a320 tx=0x7f52f80095c0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.730+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 msgr2=0x7f530819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.730+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f530819cc00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52f000b3c0 tx=0x7f52f000b880 comp rx=0 tx=0).stop 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 shutdown_connections 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52f40b66e0 0x7f52f40b8bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5308108cf0 0x7f53081a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53081083a0 0x7f530819cc00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 --2- 172.21.15.29:0/811071468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5308102a60 0x7f530819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:49:58.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 >> 172.21.15.29:0/811071468 conn(0x7f5308079730 msgr2=0x7f5308106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:49:58.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 shutdown_connections 2024-09-21T01:49:58.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:49:58.731+0000 7f530cb5b640 1 -- 172.21.15.29:0/811071468 wait complete. 2024-09-21T01:49:59.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:50:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:00.185 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:49:59 smithi029 ceph-mon[31228]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 MiB/s wr, 739 op/s 2024-09-21T01:50:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:49:59 smithi190 ceph-mon[34368]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 MiB/s wr, 739 op/s 2024-09-21T01:50:00.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:49:59 smithi066 ceph-mon[35001]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 MiB/s wr, 739 op/s 2024-09-21T01:50:00.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.871+0000 7f18c020f640 1 -- 172.21.15.29:0/72643250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8077380 msgr2=0x7f18b80757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.871+0000 7f18c020f640 1 --2- 172.21.15.29:0/72643250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8077380 0x7f18b80757b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009900 tx=0x7f18b402a420 comp rx=0 tx=0).stop 2024-09-21T01:50:00.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 -- 172.21.15.29:0/72643250 shutdown_connections 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 --2- 172.21.15.29:0/72643250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 0x7f18b8116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 --2- 172.21.15.29:0/72643250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8075dd0 0x7f18b80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 --2- 172.21.15.29:0/72643250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8077380 0x7f18b80757b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 -- 172.21.15.29:0/72643250 >> 172.21.15.29:0/72643250 conn(0x7f18b80ff5e0 msgr2=0x7f18b8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 -- 172.21.15.29:0/72643250 shutdown_connections 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.872+0000 7f18c020f640 1 -- 172.21.15.29:0/72643250 wait complete. 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 Processor -- start 2024-09-21T01:50:00.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 -- start start 2024-09-21T01:50:00.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8077380 0x7f18b81a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 0x7f18b81a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18b811a130 con 0x7f18b8114440 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8119fb0 con 0x7f18b8075dd0 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.873+0000 7f18c020f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18b811a2b0 con 0x7f18b8077380 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39964/0 (socket says 172.21.15.29:39964) 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 -- 172.21.15.29:0/1740178378 learned_addr learned my addr 172.21.15.29:0/1740178378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:00.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18be7ac640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 0x7f18b81a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8077380 0x7f18b81a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 -- 172.21.15.29:0/1740178378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8077380 msgr2=0x7f18b81a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8077380 0x7f18b81a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 -- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 msgr2=0x7f18b81a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 0x7f18b81a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.874+0000 7f18bdfab640 1 -- 172.21.15.29:0/1740178378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81a6870 con 0x7f18b8075dd0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009850 tx=0x7f18b4030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18aeffd640 1 -- 172.21.15.29:0/1740178378 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b403f070 con 0x7f18b8075dd0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18aeffd640 1 -- 172.21.15.29:0/1740178378 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b4004460 con 0x7f18b8075dd0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18aeffd640 1 -- 172.21.15.29:0/1740178378 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4035030 con 0x7f18b8075dd0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a6b00 con 0x7f18b8075dd0 2024-09-21T01:50:00.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.876+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81ad2c0 con 0x7f18b8075dd0 2024-09-21T01:50:00.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.879+0000 7f18aeffd640 1 -- 172.21.15.29:0/1740178378 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f18b4042070 con 0x7f18b8075dd0 2024-09-21T01:50:00.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.880+0000 7f18c6783740 1 Processor -- start 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 -- start start 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b395543ea0 con 0x55b395205390 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b3955332d0 con 0x55b3951fd1b0 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18c6783740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b3955370e0 con 0x55b395226070 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18be7ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.881+0000 7f18be7ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50370/0 (socket says 172.21.15.29:50370) 2024-09-21T01:50:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 -- 172.21.15.29:0/175178533 learned_addr learned my addr 172.21.15.29:0/175178533 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18bdfab640 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18aeffd640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 0x7f18a40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 -- 172.21.15.29:0/175178533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 msgr2=0x55b3955cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 -- 172.21.15.29:0/175178533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 msgr2=0x55b3955ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.882+0000 7f18be7ac640 1 -- 172.21.15.29:0/175178533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b395291650 con 0x55b395226070 2024-09-21T01:50:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18be7ac640 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18b0010510 tx=0x7f18b0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb9fdf92ebdddb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18aeffd640 1 -- 172.21.15.29:0/1740178378 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f18b41080c0 con 0x7f18b8075dd0 2024-09-21T01:50:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18be7ac640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 0x7f18a40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18a3fff640 1 -- 172.21.15.29:0/175178533 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0013800 con 0x55b395226070 2024-09-21T01:50:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18a3fff640 1 -- 172.21.15.29:0/175178533 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b0013e50 con 0x55b395226070 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.883+0000 7f18a3fff640 1 -- 172.21.15.29:0/175178533 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b000e630 con 0x55b395226070 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 -- 172.21.15.29:0/175178533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 msgr2=0x55b3955cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18b0010510 tx=0x7f18b0009f90 comp rx=0 tx=0).stop 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18be7ac640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 0x7f18a40b8b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18b0010fd0 tx=0x7f18b001eef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 -- 172.21.15.29:0/175178533 shutdown_connections 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 --2- 172.21.15.29:0/175178533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395226070 0x55b3955cd6a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.885+0000 7f18c6783740 1 -- 172.21.15.29:0/175178533 >> 172.21.15.29:0/175178533 conn(0x55b39521a460 msgr2=0x55b3955cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.888+0000 7f18c6783740 1 -- 172.21.15.29:0/175178533 shutdown_connections 2024-09-21T01:50:00.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.888+0000 7f18c6783740 1 -- 172.21.15.29:0/175178533 wait complete. 2024-09-21T01:50:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.889+0000 7f18c6783740 1 -- start start 2024-09-21T01:50:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.889+0000 7f18c6783740 1 Processor -- start 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395205390 0x55b3955d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395226070 0x55b3955f6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b395394700 con 0x55b395226070 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b3954ce160 con 0x55b3951fd1b0 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18c6783740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b3955ece90 con 0x55b395205390 2024-09-21T01:50:00.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39970/0 (socket says 172.21.15.29:39970) 2024-09-21T01:50:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.891+0000 7f18bdfab640 1 -- 172.21.15.29:0/3296469925 learned_addr learned my addr 172.21.15.29:0/3296469925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.892+0000 7f18c6783740 10 client.0 _ll_register_callbacks cb 0x55b3954e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.892+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395205390 0x55b3955d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.892+0000 7f18be7ac640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395226070 0x55b3955f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.893+0000 7f18bdfab640 1 -- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395205390 msgr2=0x55b3955d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.893+0000 7f18bdfab640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395205390 0x55b3955d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.893+0000 7f18bdfab640 1 -- 172.21.15.29:0/3296469925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395226070 msgr2=0x55b3955f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.893+0000 7f18bdfab640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395226070 0x55b3955f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.893+0000 7f18bdfab640 1 -- 172.21.15.29:0/3296469925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3954fbc90 con 0x55b3951fd1b0 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.894+0000 7f18bdfab640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18b402a940 tx=0x7f18b403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16cc60a52c88f961 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.894+0000 7f18a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.901+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4049070 con 0x55b3951fd1b0 2024-09-21T01:50:00.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.901+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f18b410b320 con 0x55b3951fd1b0 2024-09-21T01:50:00.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.904+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b0013e50 con 0x55b3951fd1b0 2024-09-21T01:50:00.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.905+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18b40ce080 con 0x55b3951fd1b0 2024-09-21T01:50:00.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.908+0000 7f18c6783740 10 client.25424 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:00.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.908+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b3955f8ba0 con 0x55b3951fd1b0 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.910+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b0013e50 con 0x55b3951fd1b0 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f18b40c9c00 con 0x55b3951fd1b0 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 10 client.25424 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 10 client.25424 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 20 client.25424 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b3951e49f0 0x55b3955fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 4 client.25424 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:00.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.911+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b395545df0 con 0x55b3951e49f0 2024-09-21T01:50:00.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.912+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b3951e49f0 0x55b3955fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.916+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b3951e49f0 0x55b3955fc0d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f34d341e56007805 server_cookie=5784ed43d1d7c207 in_seq=0 out_seq=0 2024-09-21T01:50:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.916+0000 7f18a17fa640 10 client.25424 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.918+0000 7f18a17fa640 1 -- 172.21.15.29:0/3296469925 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b395545df0 con 0x55b3951e49f0 2024-09-21T01:50:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.918+0000 7f18a17fa640 10 client.25424 handle_command_reply: tid=0 2024-09-21T01:50:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.918+0000 7f18c6783740 1 client.25424 shutdown 2024-09-21T01:50:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.919+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 msgr2=0x55b3955d8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.919+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18b402a940 tx=0x7f18b403c000 comp rx=0 tx=0).stop 2024-09-21T01:50:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 shutdown_connections 2024-09-21T01:50:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395226070 0x55b3955f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b3951e49f0 0x55b3955fc0d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b395205390 0x55b3955d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 --2- 172.21.15.29:0/3296469925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955d8d00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18a17fa640 0 client.25424 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.920+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 >> 172.21.15.29:0/3296469925 conn(0x55b39521a460 msgr2=0x55b3955d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.921+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 shutdown_connections 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.921+0000 7f18c6783740 1 -- 172.21.15.29:0/3296469925 wait complete. 2024-09-21T01:50:00.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.921+0000 7f18c6783740 20 client.25424 trim_cache size 0 max 16384 2024-09-21T01:50:00.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 Processor -- start 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 -- start start 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 0x55b3955da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8110020 con 0x55b395205390 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81aedb0 con 0x55b3951fd1b0 2024-09-21T01:50:00.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.944+0000 7f18c6783740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ad850 con 0x55b3951e49f0 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18be7ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18be7ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40924/0 (socket says 172.21.15.29:40924) 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18be7ac640 1 -- 172.21.15.29:0/50426605 learned_addr learned my addr 172.21.15.29:0/50426605 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 0x55b3955da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 -- 172.21.15.29:0/50426605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 msgr2=0x55b3955cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 -- 172.21.15.29:0/50426605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 msgr2=0x55b3955cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.945+0000 7f18bdfab640 1 -- 172.21.15.29:0/50426605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3955db0d0 con 0x55b3951e49f0 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.946+0000 7f18bdfab640 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 0x55b3955da220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18b40cb520 tx=0x7f18b402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4adacb5e1951368d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.946+0000 7f18a37fe640 1 -- 172.21.15.29:0/50426605 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4049040 con 0x55b3951e49f0 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.946+0000 7f18a37fe640 1 -- 172.21.15.29:0/50426605 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b40cddc0 con 0x55b3951e49f0 2024-09-21T01:50:00.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.946+0000 7f18a37fe640 1 -- 172.21.15.29:0/50426605 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40cda20 con 0x55b3951e49f0 2024-09-21T01:50:00.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 -- 172.21.15.29:0/50426605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 msgr2=0x55b3955da220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 0x55b3955da220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18b40cb520 tx=0x7f18b402a420 comp rx=0 tx=0).stop 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 -- 172.21.15.29:0/50426605 shutdown_connections 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b395205390 0x55b3955cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3951fd1b0 0x55b3955cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 --2- 172.21.15.29:0/50426605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951e49f0 0x55b3955da220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 -- 172.21.15.29:0/50426605 >> 172.21.15.29:0/50426605 conn(0x55b39521a460 msgr2=0x55b3955cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 -- 172.21.15.29:0/50426605 shutdown_connections 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.947+0000 7f18c6783740 1 -- 172.21.15.29:0/50426605 wait complete. 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 -- start start 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 Processor -- start 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951fd1b0 0x55b3955ee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b395205390 0x55b3955eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ae620 con 0x55b3951e49f0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81adf50 con 0x55b395205390 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18b810e000 con 0x55b3951fd1b0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.948+0000 7f18c6783740 10 client.0 _ll_register_callbacks cb 0x7f18b4005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40930/0 (socket says 172.21.15.29:40930) 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 -- 172.21.15.29:0/1255407647 learned_addr learned my addr 172.21.15.29:0/1255407647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18be7ac640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b395205390 0x55b3955eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951fd1b0 0x55b3955ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 -- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951fd1b0 msgr2=0x55b3955ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951fd1b0 0x55b3955ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 -- 172.21.15.29:0/1255407647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b395205390 msgr2=0x55b3955eedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b395205390 0x55b3955eedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 -- 172.21.15.29:0/1255407647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b395665cd0 con 0x55b3951e49f0 2024-09-21T01:50:00.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18bdfab640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18b40c9fd0 tx=0x7f18b4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ed1b2bab71b6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:00.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.949+0000 7f18a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:00.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4049040 con 0x55b3951e49f0 2024-09-21T01:50:00.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f18b40f1d80 con 0x55b3951e49f0 2024-09-21T01:50:00.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b40cddc0 con 0x55b3951e49f0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18b40cdc00 con 0x55b3951e49f0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18c6783740 10 client.15738 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.951+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b395665cd0 con 0x55b3951e49f0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b40d7c20 con 0x55b3951e49f0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f18b407aad0 con 0x55b3951e49f0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 10 client.15738 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 10 client.15738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 20 client.15738 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b395226070 0x55b3956692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 4 client.15738 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f18b40cddc0 con 0x55b395226070 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.952+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b395226070 0x55b3956692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18a17fa640 10 client.15738 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:00.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18bd7aa640 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b395226070 0x55b3956692d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df23e4315e28c02f server_cookie=1e4370844212b47f in_seq=0 out_seq=0 2024-09-21T01:50:00.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18a17fa640 1 -- 172.21.15.29:0/1255407647 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f18b40cddc0 con 0x55b395226070 2024-09-21T01:50:00.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18a17fa640 10 client.15738 handle_command_reply: tid=0 2024-09-21T01:50:00.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18c6783740 1 client.15738 shutdown 2024-09-21T01:50:00.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 msgr2=0x55b3955ee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.953+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18b40c9fd0 tx=0x7f18b4004610 comp rx=0 tx=0).stop 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 shutdown_connections 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b395205390 0x55b3955eedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55b395226070 0x55b3956692d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b3951fd1b0 0x55b3955ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 --2- 172.21.15.29:0/1255407647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3951e49f0 0x55b3955ee330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18a17fa640 0 client.15738 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 >> 172.21.15.29:0/1255407647 conn(0x55b39521a460 msgr2=0x55b3955cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 shutdown_connections 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 1 -- 172.21.15.29:0/1255407647 wait complete. 2024-09-21T01:50:00.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.954+0000 7f18c6783740 20 client.15738 trim_cache size 0 max 16384 2024-09-21T01:50:00.957 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:00.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 msgr2=0x7f18a40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 0x7f18a40b8b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18b0010fd0 tx=0x7f18b001eef0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 msgr2=0x7f18b81a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009850 tx=0x7f18b4030650 comp rx=0 tx=0).stop 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 shutdown_connections 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18a40b6610 0x7f18a40b8b20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18b8114440 0x7f18b81a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8077380 0x7f18b81a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 --2- 172.21.15.29:0/1740178378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b8075dd0 0x7f18b81a0900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.955+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 >> 172.21.15.29:0/1740178378 conn(0x7f18b80ff5e0 msgr2=0x7f18b8110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.956+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 shutdown_connections 2024-09-21T01:50:00.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:00.956+0000 7f18c020f640 1 -- 172.21.15.29:0/1740178378 wait complete. 2024-09-21T01:50:01.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:00 smithi029 ceph-mon[31228]: overall HEALTH_OK 2024-09-21T01:50:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:00 smithi190 ceph-mon[34368]: overall HEALTH_OK 2024-09-21T01:50:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:00 smithi066 ceph-mon[35001]: overall HEALTH_OK 2024-09-21T01:50:01.838 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:50:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:01 smithi029 ceph-mon[31228]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-21T01:50:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:01 smithi190 ceph-mon[34368]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-21T01:50:02.248 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:01 smithi066 ceph-mon[35001]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 -- 172.21.15.29:0/2132139169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 msgr2=0x7fa088110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 --2- 172.21.15.29:0/2132139169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa088110100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0840074e0 tx=0x7fa08402b0a0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 -- 172.21.15.29:0/2132139169 shutdown_connections 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 --2- 172.21.15.29:0/2132139169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa088110640 0x7fa088116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 --2- 172.21.15.29:0/2132139169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa088110100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 --2- 172.21.15.29:0/2132139169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08810a650 0x7fa08810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 -- 172.21.15.29:0/2132139169 >> 172.21.15.29:0/2132139169 conn(0x7fa08806ee50 msgr2=0x7fa08806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.845+0000 7fa08f109640 1 -- 172.21.15.29:0/2132139169 shutdown_connections 2024-09-21T01:50:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.846+0000 7fa08f109640 1 -- 172.21.15.29:0/2132139169 wait complete. 2024-09-21T01:50:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 Processor -- start 2024-09-21T01:50:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 -- start start 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa08810a650 0x7fa0881a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa088110640 0x7fa0881aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa08811e370 con 0x7fa08810a650 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa08811e1f0 con 0x7fa08810fcf0 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.847+0000 7fa08f109640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa08811e4f0 con 0x7fa088110640 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45386/0 (socket says 172.21.15.29:45386) 2024-09-21T01:50:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 -- 172.21.15.29:0/3055473173 learned_addr learned my addr 172.21.15.29:0/3055473173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 -- 172.21.15.29:0/3055473173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa088110640 msgr2=0x7fa0881aa600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa088110640 0x7fa0881aa600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 -- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa08810a650 msgr2=0x7fa0881a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa08810a650 0x7fa0881a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 -- 172.21.15.29:0/3055473173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0881aad30 con 0x7fa08810fcf0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.848+0000 7fa08d906640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0840074b0 tx=0x7fa08402bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.849+0000 7fa07f7fe640 1 -- 172.21.15.29:0/3055473173 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084009ec0 con 0x7fa08810fcf0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.849+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0881aafc0 con 0x7fa08810fcf0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.849+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0881b1780 con 0x7fa08810fcf0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.849+0000 7fa07f7fe640 1 -- 172.21.15.29:0/3055473173 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08402be00 con 0x7fa08810fcf0 2024-09-21T01:50:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.849+0000 7fa07f7fe640 1 -- 172.21.15.29:0/3055473173 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08400a200 con 0x7fa08810fcf0 2024-09-21T01:50:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.853+0000 7fa0968df740 1 Processor -- start 2024-09-21T01:50:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.853+0000 7fa0968df740 1 -- start start 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.853+0000 7fa07f7fe640 1 -- 172.21.15.29:0/3055473173 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa084042020 con 0x7fa08810fcf0 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x56172620b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 0x56172620b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e3a390 0x56172620baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561726178ea0 con 0x561725e5b070 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617261682d0 con 0x561725e321b0 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa0968df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56172616c0e0 con 0x561725e3a390 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa08e908640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x56172620b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa08e908640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x56172620b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51894/0 (socket says 172.21.15.29:51894) 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa08e908640 1 -- 172.21.15.29:0/2792638589 learned_addr learned my addr 172.21.15.29:0/2792638589 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa08e107640 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 0x56172620b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa08d906640 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e3a390 0x56172620baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.854+0000 7fa07f7fe640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 0x7fa0740b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 -- 172.21.15.29:0/2792638589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e3a390 msgr2=0x56172620baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e3a390 0x56172620baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 -- 172.21.15.29:0/2792638589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 msgr2=0x56172620b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x56172620b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 -- 172.21.15.29:0/2792638589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561725ec6650 con 0x561725e321b0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e908640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 0x7fa0740b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa08e107640 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 0x56172620b5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa078009720 tx=0x7fa078009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0eefad1ef878c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.855+0000 7fa07f7fe640 1 -- 172.21.15.29:0/3055473173 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa0841062e0 con 0x7fa08810fcf0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.856+0000 7fa07cd9a640 1 -- 172.21.15.29:0/2792638589 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078015020 con 0x561725e321b0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.856+0000 7fa07cd9a640 1 -- 172.21.15.29:0/2792638589 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa078009e30 con 0x561725e321b0 2024-09-21T01:50:02.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.856+0000 7fa07cd9a640 1 -- 172.21.15.29:0/2792638589 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078015020 con 0x561725e321b0 2024-09-21T01:50:02.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 -- 172.21.15.29:0/2792638589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 msgr2=0x56172620b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa08e908640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 0x7fa0740b8d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa080013720 tx=0x7fa080004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 0x56172620b5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa078009720 tx=0x7fa078009be0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 -- 172.21.15.29:0/2792638589 shutdown_connections 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e3a390 0x56172620baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e321b0 0x56172620b5b0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa078009720 tx=0x7fa078009be0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 --2- 172.21.15.29:0/2792638589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x56172620b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 -- 172.21.15.29:0/2792638589 >> 172.21.15.29:0/2792638589 conn(0x561725e4f460 msgr2=0x561726208f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 -- 172.21.15.29:0/2792638589 shutdown_connections 2024-09-21T01:50:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.858+0000 7fa0968df740 1 -- 172.21.15.29:0/2792638589 wait complete. 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.859+0000 7fa0968df740 1 -- start start 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.859+0000 7fa0968df740 1 Processor -- start 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.859+0000 7fa0968df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726216d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45394/0 (socket says 172.21.15.29:45394) 2024-09-21T01:50:02.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 -- 172.21.15.29:0/3636563718 learned_addr learned my addr 172.21.15.29:0/3636563718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561725fc9700 con 0x561725e5b070 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561726103160 con 0x561725e3a390 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561726219850 con 0x561725e199f0 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08d906640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726216d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa0968df740 10 client.0 _ll_register_callbacks cb 0x56172611c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e908640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 -- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 msgr2=0x561726222b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 -- 172.21.15.29:0/3636563718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 msgr2=0x561726216d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726216d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561726130c90 con 0x561725e3a390 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e908640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08d906640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726216d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:02.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa08e107640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa078006a90 tx=0x7fa078006df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b40fb298179321 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.860+0000 7fa071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:02.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.861+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078015020 con 0x561725e3a390 2024-09-21T01:50:02.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.861+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa07800bad0 con 0x561725e3a390 2024-09-21T01:50:02.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.861+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa078009e30 con 0x561725e3a390 2024-09-21T01:50:02.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.862+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa078011630 con 0x561725e3a390 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.862+0000 7fa0968df740 10 client.25442 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.862+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561726225580 con 0x561725e3a390 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa078011820 con 0x561725e3a390 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa078030420 con 0x561725e3a390 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 10 client.25442 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 10 client.25442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 20 client.25442 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561725deeda0 0x561726228ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 4 client.25442 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa078009e30 con 0x561725deeda0 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.863+0000 7fa08d906640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561725deeda0 0x561726228ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.864+0000 7fa071ffb640 10 client.25442 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:02.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.864+0000 7fa08d906640 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561725deeda0 0x561726228ce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6ac9b91c183c8d6 server_cookie=517e1b7116801bda in_seq=0 out_seq=0 2024-09-21T01:50:02.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.865+0000 7fa071ffb640 1 -- 172.21.15.29:0/3636563718 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa078009e30 con 0x561725deeda0 2024-09-21T01:50:02.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.865+0000 7fa071ffb640 10 client.25442 handle_command_reply: tid=0 2024-09-21T01:50:02.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.865+0000 7fa0968df740 1 client.25442 shutdown 2024-09-21T01:50:02.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.866+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 msgr2=0x561726216820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.866+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa078006a90 tx=0x7fa078006df0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 shutdown_connections 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561725deeda0 0x561726228ce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726216d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 --2- 172.21.15.29:0/3636563718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x561726216820 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 >> 172.21.15.29:0/3636563718 conn(0x561725e4f460 msgr2=0x56172620e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa071ffb640 0 client.25442 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 shutdown_connections 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 1 -- 172.21.15.29:0/3636563718 wait complete. 2024-09-21T01:50:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.867+0000 7fa0968df740 20 client.25442 trim_cache size 0 max 16384 2024-09-21T01:50:02.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.889+0000 7fa0968df740 1 Processor -- start 2024-09-21T01:50:02.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.889+0000 7fa0968df740 1 -- start start 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726208cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b5890 con 0x561725e5b070 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b74c0 con 0x561725e3a390 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa0968df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b5f60 con 0x561725e199f0 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45402/0 (socket says 172.21.15.29:45402) 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 -- 172.21.15.29:0/3291922144 learned_addr learned my addr 172.21.15.29:0/3291922144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08e908640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726208cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08e107640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 -- 172.21.15.29:0/3291922144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 msgr2=0x561726222a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 -- 172.21.15.29:0/3291922144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 msgr2=0x561726208cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726208cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08d906640 1 -- 172.21.15.29:0/3291922144 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726218f60 con 0x561725e3a390 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.890+0000 7fa08e908640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726208cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.891+0000 7fa08d906640 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0840cc180 tx=0x7fa0840c8c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcdec40696b8148b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.891+0000 7fa073fff640 1 -- 172.21.15.29:0/3291922144 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0840cc250 con 0x561725e3a390 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.891+0000 7fa073fff640 1 -- 172.21.15.29:0/3291922144 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08408de50 con 0x561725e3a390 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.891+0000 7fa073fff640 1 -- 172.21.15.29:0/3291922144 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0840cbbf0 con 0x561725e3a390 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.892+0000 7fa0968df740 1 -- 172.21.15.29:0/3291922144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 msgr2=0x5617262087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.892+0000 7fa0968df740 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0840cc180 tx=0x7fa0840c8c00 comp rx=0 tx=0).stop 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 -- 172.21.15.29:0/3291922144 shutdown_connections 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e5b070 0x561726208cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e3a390 0x5617262087b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 --2- 172.21.15.29:0/3291922144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e199f0 0x561726222a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 -- 172.21.15.29:0/3291922144 >> 172.21.15.29:0/3291922144 conn(0x561725e4f460 msgr2=0x5617262150a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 -- 172.21.15.29:0/3291922144 shutdown_connections 2024-09-21T01:50:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 -- 172.21.15.29:0/3291922144 wait complete. 2024-09-21T01:50:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.893+0000 7fa0968df740 1 -- start start 2024-09-21T01:50:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa0968df740 1 Processor -- start 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa0968df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa0968df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e3a390 0x56172621a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa08e107640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa0968df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e5b070 0x56172621af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.894+0000 7fa08e107640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45418/0 (socket says 172.21.15.29:45418) 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 -- 172.21.15.29:0/409784348 learned_addr learned my addr 172.21.15.29:0/409784348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:02.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b6d30 con 0x561725e3a390 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b6660 con 0x561725e199f0 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b3890 con 0x561725e5b070 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa0968df740 10 client.0 _ll_register_callbacks cb 0x7fa080007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e908640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e5b070 0x56172621af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 -- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e5b070 msgr2=0x56172621af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e5b070 0x56172621af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 -- 172.21.15.29:0/409784348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e3a390 msgr2=0x56172621a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e3a390 0x56172621a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.895+0000 7fa08e107640 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172621b6d0 con 0x561725e199f0 2024-09-21T01:50:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.896+0000 7fa08e107640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa078007fd0 tx=0x7fa078006d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0dfce2b075ac2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078011830 con 0x561725e199f0 2024-09-21T01:50:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa078011aa0 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa08408de50 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa078030c70 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa0968df740 10 client.25448 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.897+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5617262936f0 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07800bd80 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa078026830 con 0x561725e199f0 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 10 client.25448 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 10 client.25448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 20 client.25448 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561726293d90 0x5617262961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 4 client.25448 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa08408de50 con 0x561726293d90 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.898+0000 7fa08d906640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561726293d90 0x5617262961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.899+0000 7fa071ffb640 10 client.25448 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:02.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.899+0000 7fa08d906640 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561726293d90 0x5617262961b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=588e0394c4e3a981 server_cookie=8016cb12b08379c7 in_seq=0 out_seq=0 2024-09-21T01:50:02.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.900+0000 7fa071ffb640 1 -- 172.21.15.29:0/409784348 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa08408de50 con 0x561726293d90 2024-09-21T01:50:02.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.900+0000 7fa071ffb640 10 client.25448 handle_command_reply: tid=0 2024-09-21T01:50:02.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.900+0000 7fa0968df740 1 client.25448 shutdown 2024-09-21T01:50:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 msgr2=0x56172621f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa078007fd0 tx=0x7fa078006d70 comp rx=0 tx=0).stop 2024-09-21T01:50:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 shutdown_connections 2024-09-21T01:50:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561726293d90 0x5617262961b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561725e5b070 0x56172621af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561725e3a390 0x56172621a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 --2- 172.21.15.29:0/409784348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561725e199f0 0x56172621f3f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 >> 172.21.15.29:0/409784348 conn(0x561725e4f460 msgr2=0x5617262087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa071ffb640 0 client.25448 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 shutdown_connections 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 1 -- 172.21.15.29:0/409784348 wait complete. 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.901+0000 7fa0968df740 20 client.25448 trim_cache size 0 max 16384 2024-09-21T01:50:02.902 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.902+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 msgr2=0x7fa0740b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.902+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 0x7fa0740b8d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa080013720 tx=0x7fa080004a40 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.902+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 msgr2=0x7fa0881a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0840074b0 tx=0x7fa08402bc10 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 shutdown_connections 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0740b6840 0x7fa0740b8d50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa088110640 0x7fa0881aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa08810fcf0 0x7fa0881a5360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 --2- 172.21.15.29:0/3055473173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa08810a650 0x7fa0881a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.903+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 >> 172.21.15.29:0/3055473173 conn(0x7fa08806ee50 msgr2=0x7fa08810dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:02.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.904+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 shutdown_connections 2024-09-21T01:50:02.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:02.904+0000 7fa08f109640 1 -- 172.21.15.29:0/3055473173 wait complete. 2024-09-21T01:50:02.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 85.0 s, 3732.2 tps, lat 26.795 ms stddev 29.642 2024-09-21T01:50:03.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:02 smithi029 ceph-mon[31228]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 720 op/s 2024-09-21T01:50:03.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:02 smithi190 ceph-mon[34368]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 720 op/s 2024-09-21T01:50:03.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:02 smithi066 ceph-mon[35001]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 720 op/s 2024-09-21T01:50:05.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:50:05.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:05 smithi066 ceph-mon[35001]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-21T01:50:05.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:05 smithi029 ceph-mon[31228]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-21T01:50:05.614 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:05 smithi190 ceph-mon[34368]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-21T01:50:06.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.792+0000 7fe5260dd640 1 -- 172.21.15.29:0/3242958780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5201027a0 msgr2=0x7fe520102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.792+0000 7fe5260dd640 1 --2- 172.21.15.29:0/3242958780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5201027a0 0x7fe520102bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe510009860 tx=0x7fe51002a360 comp rx=0 tx=0).stop 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 -- 172.21.15.29:0/3242958780 shutdown_connections 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 --2- 172.21.15.29:0/3242958780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520107be0 0x7fe5201080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 --2- 172.21.15.29:0/3242958780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201030f0 0x7fe520107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 --2- 172.21.15.29:0/3242958780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5201027a0 0x7fe520102bb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 -- 172.21.15.29:0/3242958780 >> 172.21.15.29:0/3242958780 conn(0x7fe520079730 msgr2=0x7fe520079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 -- 172.21.15.29:0/3242958780 shutdown_connections 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.793+0000 7fe5260dd640 1 -- 172.21.15.29:0/3242958780 wait complete. 2024-09-21T01:50:06.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.794+0000 7fe5260dd640 1 Processor -- start 2024-09-21T01:50:06.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.795+0000 7fe5260dd640 1 -- start start 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.795+0000 7fe5260dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe5260dd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 0x7fe520110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe5260dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520107be0 0x7fe5201106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe5260dd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe52011a090 con 0x7fe5201030f0 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe5260dd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe520119f10 con 0x7fe5201027a0 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe5260dd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe52011a210 con 0x7fe520107be0 2024-09-21T01:50:06.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45438/0 (socket says 172.21.15.29:45438) 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 -- 172.21.15.29:0/2569687914 learned_addr learned my addr 172.21.15.29:0/2569687914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51ffff640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520107be0 0x7fe5201106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51effd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 0x7fe520110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 -- 172.21.15.29:0/2569687914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520107be0 msgr2=0x7fe5201106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520107be0 0x7fe5201106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 -- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 msgr2=0x7fe520110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 0x7fe520110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51f7fe640 1 -- 172.21.15.29:0/2569687914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5201159d0 con 0x7fe5201027a0 2024-09-21T01:50:06.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.796+0000 7fe51effd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 0x7fe520110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:06.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.797+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe510035040 tx=0x7fe510032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.797+0000 7fe51cff9640 1 -- 172.21.15.29:0/2569687914 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510037050 con 0x7fe5201027a0 2024-09-21T01:50:06.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.797+0000 7fe51cff9640 1 -- 172.21.15.29:0/2569687914 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510043070 con 0x7fe5201027a0 2024-09-21T01:50:06.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.797+0000 7fe51cff9640 1 -- 172.21.15.29:0/2569687914 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510003c70 con 0x7fe5201027a0 2024-09-21T01:50:06.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.797+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe520115c60 con 0x7fe5201027a0 2024-09-21T01:50:06.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.798+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5201160f0 con 0x7fe5201027a0 2024-09-21T01:50:06.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.802+0000 7fe51cff9640 1 -- 172.21.15.29:0/2569687914 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe51003e450 con 0x7fe5201027a0 2024-09-21T01:50:06.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.802+0000 7fe52c651740 1 Processor -- start 2024-09-21T01:50:06.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 -- start start 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c388522ea0 con 0x55c3881e4390 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c3885122d0 con 0x55c3881dc1b0 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.803+0000 7fe52c651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c3885160e0 con 0x55c388205070 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.804+0000 7fe51effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.804+0000 7fe51effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51950/0 (socket says 172.21.15.29:51950) 2024-09-21T01:50:06.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.804+0000 7fe51effd640 1 -- 172.21.15.29:0/2145058771 learned_addr learned my addr 172.21.15.29:0/2145058771 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:06.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.804+0000 7fe51cff9640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 0x7fe5080b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.804+0000 7fe51cff9640 1 -- 172.21.15.29:0/2569687914 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe510107620 con 0x7fe5201027a0 2024-09-21T01:50:06.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51ffff640 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 -- 172.21.15.29:0/2145058771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 msgr2=0x55c3885ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 -- 172.21.15.29:0/2145058771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 msgr2=0x55c3885acbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:50:06.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885acbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 -- 172.21.15.29:0/2145058771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c388270650 con 0x55c3881e4390 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51ffff640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 0x7fe5080b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe51effd640 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe514005c80 tx=0x7fe51400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ac75316bd2f344 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.805+0000 7fe505ffb640 1 -- 172.21.15.29:0/2145058771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015070 con 0x55c3881e4390 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe505ffb640 1 -- 172.21.15.29:0/2145058771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5140035f0 con 0x55c3881e4390 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe505ffb640 1 -- 172.21.15.29:0/2145058771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015070 con 0x55c3881e4390 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe52c651740 1 -- 172.21.15.29:0/2145058771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 msgr2=0x55c3885ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe52c651740 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe514005c80 tx=0x7fe51400f730 comp rx=0 tx=0).stop 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe51ffff640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 0x7fe5080b8b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe518005fd0 tx=0x7fe518004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.806+0000 7fe52c651740 1 -- 172.21.15.29:0/2145058771 shutdown_connections 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881e4390 0x55c3885ad120 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 --2- 172.21.15.29:0/2145058771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 -- 172.21.15.29:0/2145058771 >> 172.21.15.29:0/2145058771 conn(0x55c3881f9460 msgr2=0x55c3885aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 -- 172.21.15.29:0/2145058771 shutdown_connections 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 -- 172.21.15.29:0/2145058771 wait complete. 2024-09-21T01:50:06.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 -- start start 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.807+0000 7fe52c651740 1 Processor -- start 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c388373700 con 0x55c3881dc1b0 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c3884ad160 con 0x55c3881e4390 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c3885bae80 con 0x55c388205070 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe52c651740 10 client.0 _ll_register_callbacks cb 0x55c3884c69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:06.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe51effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe51effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45444/0 (socket says 172.21.15.29:45444) 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.808+0000 7fe51effd640 1 -- 172.21.15.29:0/3220151189 learned_addr learned my addr 172.21.15.29:0/3220151189 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 -- 172.21.15.29:0/3220151189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 msgr2=0x55c3885c4200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885c4200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 -- 172.21.15.29:0/3220151189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 msgr2=0x55c3885b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 -- 172.21.15.29:0/3220151189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3884dac90 con 0x55c3881e4390 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe51effd640 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe514003ef0 tx=0x7fe5140039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5860beab0e7569 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.809+0000 7fe4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015030 con 0x55c3881e4390 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe514005480 con 0x55c3881e4390 2024-09-21T01:50:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5140035f0 con 0x55c3881e4390 2024-09-21T01:50:06.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe514011440 con 0x55c3881e4390 2024-09-21T01:50:06.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe52c651740 10 client.25457 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:06.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3885c6c00 con 0x55c3881e4390 2024-09-21T01:50:06.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.810+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5140035f0 con 0x55c3881e4390 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe514024540 con 0x55c3881e4390 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 10 client.25457 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 10 client.25457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 20 client.25457 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c3881c39f0 0x55c3885c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 4 client.25457 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:06.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.811+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c388524df0 con 0x55c3881c39f0 2024-09-21T01:50:06.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.813+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c3881c39f0 0x55c3885c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.813+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c3881c39f0 0x55c3885c9f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7c1d5a02e5b5984 server_cookie=21886b33804afd15 in_seq=0 out_seq=0 2024-09-21T01:50:06.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.813+0000 7fe4feffd640 10 client.25457 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:50:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.814+0000 7fe4feffd640 1 -- 172.21.15.29:0/3220151189 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c388524df0 con 0x55c3881c39f0 2024-09-21T01:50:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe4feffd640 10 client.25457 handle_command_reply: tid=0 2024-09-21T01:50:06.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 client.25457 shutdown 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 msgr2=0x55c3885b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe514003ef0 tx=0x7fe5140039d0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 shutdown_connections 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c3881c39f0 0x55c3885c9f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885b82b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 --2- 172.21.15.29:0/3220151189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 >> 172.21.15.29:0/3220151189 conn(0x55c3881f9460 msgr2=0x55c3885af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.815+0000 7fe4feffd640 0 client.25457 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:50:06.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.816+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 shutdown_connections 2024-09-21T01:50:06.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.816+0000 7fe52c651740 1 -- 172.21.15.29:0/3220151189 wait complete. 2024-09-21T01:50:06.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.816+0000 7fe52c651740 20 client.25457 trim_cache size 0 max 16384 2024-09-21T01:50:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 Processor -- start 2024-09-21T01:50:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 -- start start 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c00e4e0 con 0x55c3881dc1b0 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c010110 con 0x55c3881e4390 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.839+0000 7fe52c651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c00ebb0 con 0x55c388205070 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:42410/0 (socket says 172.21.15.29:42410) 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 -- 172.21.15.29:0/3417460774 learned_addr learned my addr 172.21.15.29:0/3417460774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51effd640 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 -- 172.21.15.29:0/3417460774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 msgr2=0x55c3885c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 -- 172.21.15.29:0/3417460774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 msgr2=0x55c3885b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.840+0000 7fe51ffff640 1 -- 172.21.15.29:0/3417460774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3885b8e10 con 0x55c388205070 2024-09-21T01:50:06.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.841+0000 7fe51ffff640 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe518013780 tx=0x7fe518014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df746700a85ebefd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.841+0000 7fe5057fa640 1 -- 172.21.15.29:0/3417460774 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51801c890 con 0x55c388205070 2024-09-21T01:50:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.841+0000 7fe5057fa640 1 -- 172.21.15.29:0/3417460774 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51801d400 con 0x55c388205070 2024-09-21T01:50:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.841+0000 7fe5057fa640 1 -- 172.21.15.29:0/3417460774 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518025650 con 0x55c388205070 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.842+0000 7fe52c651740 1 -- 172.21.15.29:0/3417460774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 msgr2=0x55c3885b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.842+0000 7fe52c651740 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe518013780 tx=0x7fe518014000 comp rx=0 tx=0).stop 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 -- 172.21.15.29:0/3417460774 shutdown_connections 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c388205070 0x55c3885b4f60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881e4390 0x55c3885c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 --2- 172.21.15.29:0/3417460774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c3881dc1b0 0x55c3885b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 -- 172.21.15.29:0/3417460774 >> 172.21.15.29:0/3417460774 conn(0x55c3881f9460 msgr2=0x55c3885b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 -- 172.21.15.29:0/3417460774 shutdown_connections 2024-09-21T01:50:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.843+0000 7fe52c651740 1 -- 172.21.15.29:0/3417460774 wait complete. 2024-09-21T01:50:06.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.844+0000 7fe52c651740 1 -- start start 2024-09-21T01:50:06.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.845+0000 7fe52c651740 1 Processor -- start 2024-09-21T01:50:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.845+0000 7fe52c651740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c3881e4390 0x55c3885bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 0x55c3885bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c00f980 con 0x55c388205070 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c00f2b0 con 0x55c3881dc1b0 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c00c4c0 con 0x55c3881e4390 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe52c651740 10 client.0 _ll_register_callbacks cb 0x7fe51400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885c0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45462/0 (socket says 172.21.15.29:45462) 2024-09-21T01:50:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.846+0000 7fe51f7fe640 1 -- 172.21.15.29:0/1489335101 learned_addr learned my addr 172.21.15.29:0/1489335101 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51effd640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c3881e4390 0x55c3885bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 0x55c3885bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 -- 172.21.15.29:0/1489335101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c3881e4390 msgr2=0x55c3885bbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c3881e4390 0x55c3885bbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 -- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 msgr2=0x55c3885c0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885c0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.847+0000 7fe51ffff640 1 -- 172.21.15.29:0/1489335101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3885bcd50 con 0x55c388205070 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.848+0000 7fe51ffff640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 0x55c3885bc5c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe518027040 tx=0x7fe5180132f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512d7adc3d05b31b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.848+0000 7fe4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.848+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518034020 con 0x55c388205070 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.849+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe51801d7c0 con 0x55c388205070 2024-09-21T01:50:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.849+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51801d400 con 0x55c388205070 2024-09-21T01:50:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.849+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe518013e20 con 0x55c388205070 2024-09-21T01:50:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 10 client.15762 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c388634780 con 0x55c388205070 2024-09-21T01:50:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51801da20 con 0x55c388205070 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe518019030 con 0x55c388205070 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 10 client.15762 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 20 client.15762 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c388198da0 0x55c388636df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 4 client.15762 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.850+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe51801d400 con 0x55c388198da0 2024-09-21T01:50:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.851+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c388198da0 0x55c388636df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe51f7fe640 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c388198da0 0x55c388636df0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25d661e724122be2 server_cookie=8088143549af45b4 in_seq=0 out_seq=0 2024-09-21T01:50:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe4feffd640 10 client.15762 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:50:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe4feffd640 1 -- 172.21.15.29:0/1489335101 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe51801d400 con 0x55c388198da0 2024-09-21T01:50:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe4feffd640 10 client.15762 handle_command_reply: tid=0 2024-09-21T01:50:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 client.15762 shutdown 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 msgr2=0x55c3885bc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 0x55c3885bc5c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe518027040 tx=0x7fe5180132f0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 shutdown_connections 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55c388198da0 0x55c388636df0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c388205070 0x55c3885bc5c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c3881e4390 0x55c3885bbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 --2- 172.21.15.29:0/1489335101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3881dc1b0 0x55c3885c0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.853+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 >> 172.21.15.29:0/1489335101 conn(0x55c3881f9460 msgr2=0x55c3885ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.854+0000 7fe4feffd640 0 client.15762 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.854+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 shutdown_connections 2024-09-21T01:50:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.854+0000 7fe52c651740 1 -- 172.21.15.29:0/1489335101 wait complete. 2024-09-21T01:50:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.854+0000 7fe52c651740 20 client.15762 trim_cache size 0 max 16384 2024-09-21T01:50:06.856 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.855+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 msgr2=0x7fe5080b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.855+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 0x7fe5080b8b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe518005fd0 tx=0x7fe518004a40 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.855+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 msgr2=0x7fe520116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.855+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe510035040 tx=0x7fe510032d50 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 shutdown_connections 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5080b6660 0x7fe5080b8b70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520107be0 0x7fe5201106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5201030f0 0x7fe520110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 --2- 172.21.15.29:0/2569687914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5201027a0 0x7fe520116d60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:06.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.856+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 >> 172.21.15.29:0/2569687914 conn(0x7fe520079730 msgr2=0x7fe5201055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:06.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.857+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 shutdown_connections 2024-09-21T01:50:06.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:06.857+0000 7fe5260dd640 1 -- 172.21.15.29:0/2569687914 wait complete. 2024-09-21T01:50:07.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:07 smithi066 ceph-mon[35001]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-21T01:50:07.591 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:07 smithi029 ceph-mon[31228]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-21T01:50:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:07 smithi190 ceph-mon[34368]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-21T01:50:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 90.0 s, 3506.6 tps, lat 28.590 ms stddev 39.547 2024-09-21T01:50:07.969 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:50:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 -- 172.21.15.29:0/3219252132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c1083a0 msgr2=0x7f5a5c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 --2- 172.21.15.29:0/3219252132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c1083a0 0x7f5a5c1087b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ca0 tx=0x7f5a4402a8f0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 -- 172.21.15.29:0/3219252132 shutdown_connections 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 --2- 172.21.15.29:0/3219252132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 --2- 172.21.15.29:0/3219252132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c1083a0 0x7f5a5c1087b0 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ca0 tx=0x7f5a4402a8f0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 --2- 172.21.15.29:0/3219252132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a5c102a60 0x7f5a5c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.026+0000 7f5a61722640 1 -- 172.21.15.29:0/3219252132 >> 172.21.15.29:0/3219252132 conn(0x7f5a5c079730 msgr2=0x7f5a5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.028+0000 7f5a61722640 1 -- 172.21.15.29:0/3219252132 shutdown_connections 2024-09-21T01:50:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.028+0000 7f5a61722640 1 -- 172.21.15.29:0/3219252132 wait complete. 2024-09-21T01:50:09.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.029+0000 7f5a61722640 1 Processor -- start 2024-09-21T01:50:09.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.029+0000 7f5a61722640 1 -- start start 2024-09-21T01:50:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a5c102a60 0x7f5a5c1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c1a65c0 0x7f5a5c1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1119a0 con 0x7f5a5c108cf0 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c1a65c0 0x7f5a5c1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a5a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a5a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52000/0 (socket says 172.21.15.29:52000) 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a5a7fc640 1 -- 172.21.15.29:0/1528067281 learned_addr learned my addr 172.21.15.29:0/1528067281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c111820 con 0x7f5a5c1a65c0 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.030+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c111b20 con 0x7f5a5c102a60 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 -- 172.21.15.29:0/1528067281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a5c102a60 msgr2=0x7f5a5c1a0e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a5c102a60 0x7f5a5c1a0e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 -- 172.21.15.29:0/1528067281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c1a65c0 msgr2=0x7f5a5c1a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c1a65c0 0x7f5a5c1a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 -- 172.21.15.29:0/1528067281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c1a71a0 con 0x7f5a5c108cf0 2024-09-21T01:50:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.031+0000 7f5a5a7fc640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009bf0 tx=0x7f5a44003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.032+0000 7f5a53fff640 1 -- 172.21.15.29:0/1528067281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44004240 con 0x7f5a5c108cf0 2024-09-21T01:50:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.032+0000 7f5a53fff640 1 -- 172.21.15.29:0/1528067281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4402d070 con 0x7f5a5c108cf0 2024-09-21T01:50:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.032+0000 7f5a53fff640 1 -- 172.21.15.29:0/1528067281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44030c90 con 0x7f5a5c108cf0 2024-09-21T01:50:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.032+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c1ad740 con 0x7f5a5c108cf0 2024-09-21T01:50:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.032+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c1adae0 con 0x7f5a5c108cf0 2024-09-21T01:50:09.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.037+0000 7f5a67c96740 1 Processor -- start 2024-09-21T01:50:09.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.037+0000 7f5a67c96740 1 -- start start 2024-09-21T01:50:09.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.037+0000 7f5a53fff640 1 -- 172.21.15.29:0/1528067281 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5a440043e0 con 0x7f5a5c108cf0 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3874390 0x55f8c3c3cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3bb2ea0 con 0x55f8c386c1b0 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3ba22d0 con 0x55f8c3895070 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a67c96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3ba60e0 con 0x55f8c3874390 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3874390 0x55f8c3c3cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45482/0 (socket says 172.21.15.29:45482) 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/167309247 learned_addr learned my addr 172.21.15.29:0/167309247 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/167309247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3874390 msgr2=0x55f8c3c3cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3874390 0x55f8c3c3cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.038+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/167309247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 msgr2=0x55f8c3c3c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5affd640 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/167309247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c3900650 con 0x55f8c3895070 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5affd640 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a53fff640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 0x7f5a3c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a540099c0 tx=0x7f5a54009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef232ff021f01f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a5affd640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 0x7f5a3c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a53fff640 1 -- 172.21.15.29:0/1528067281 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a44108720 con 0x7f5a5c108cf0 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a517fa640 1 -- 172.21.15.29:0/167309247 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54017070 con 0x55f8c3895070 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a517fa640 1 -- 172.21.15.29:0/167309247 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a54013470 con 0x55f8c3895070 2024-09-21T01:50:09.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.039+0000 7f5a517fa640 1 -- 172.21.15.29:0/167309247 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54012630 con 0x55f8c3895070 2024-09-21T01:50:09.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a5affd640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 0x7f5a3c0b8b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55f8c3c3af40 tx=0x7f5a4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 -- 172.21.15.29:0/167309247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 msgr2=0x55f8c3c3c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a540099c0 tx=0x7f5a54009e80 comp rx=0 tx=0).stop 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 -- 172.21.15.29:0/167309247 shutdown_connections 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3874390 0x55f8c3c3cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 --2- 172.21.15.29:0/167309247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c3c2c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.041+0000 7f5a67c96740 1 -- 172.21.15.29:0/167309247 >> 172.21.15.29:0/167309247 conn(0x55f8c3889460 msgr2=0x55f8c3c3a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.042+0000 7f5a67c96740 1 -- 172.21.15.29:0/167309247 shutdown_connections 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.042+0000 7f5a67c96740 1 -- 172.21.15.29:0/167309247 wait complete. 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.042+0000 7f5a67c96740 1 -- start start 2024-09-21T01:50:09.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.042+0000 7f5a67c96740 1 Processor -- start 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c47f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3a03700 con 0x55f8c3874390 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3b3d160 con 0x55f8c3895070 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3c4aaa0 con 0x55f8c386c1b0 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a5a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c47f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:42440/0 (socket says 172.21.15.29:42440) 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a5affd640 1 -- 172.21.15.29:0/1471446131 learned_addr learned my addr 172.21.15.29:0/1471446131 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.043+0000 7f5a67c96740 10 client.0 _ll_register_callbacks cb 0x55f8c3b569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 -- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 msgr2=0x55f8c3c53d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c53d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 -- 172.21.15.29:0/1471446131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 msgr2=0x55f8c3c47f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c47f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 -- 172.21.15.29:0/1471446131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c3b6ac90 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a5affd640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00cde0 tx=0x7f5a4c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b52f2dd6841aca8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c023060 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a54013470 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.044+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a4c01e600 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a4c01e800 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 10 client.25507 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f8c3c567d0 con 0x55f8c386c1b0 2024-09-21T01:50:09.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4c01ee50 con 0x55f8c386c1b0 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5a4c029b90 con 0x55f8c386c1b0 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 10 client.25507 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 20 client.25507 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c38539f0 0x55f8c3c59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 4 client.25507 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.045+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a54013470 con 0x55f8c38539f0 2024-09-21T01:50:09.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.046+0000 7f5a5a7fc640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c38539f0 0x55f8c3c59d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.047+0000 7f5a5a7fc640 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c38539f0 0x55f8c3c59d40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed796812c55d5292 server_cookie=df4e79d6350b710d in_seq=0 out_seq=0 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.047+0000 7f5a4a7fc640 10 client.25507 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/1471446131 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5a54013470 con 0x55f8c38539f0 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a4a7fc640 10 client.25507 handle_command_reply: tid=0 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 client.25507 shutdown 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 msgr2=0x55f8c3c47a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00cde0 tx=0x7f5a4c004770 comp rx=0 tx=0).stop 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 shutdown_connections 2024-09-21T01:50:09.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c38539f0 0x55f8c3c59d40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3895070 0x55f8c3c53d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c47f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 --2- 172.21.15.29:0/1471446131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c386c1b0 0x55f8c3c47a20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 >> 172.21.15.29:0/1471446131 conn(0x55f8c3889460 msgr2=0x55f8c3c3f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.048+0000 7f5a4a7fc640 0 client.25507 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.049+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 shutdown_connections 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.049+0000 7f5a67c96740 1 -- 172.21.15.29:0/1471446131 wait complete. 2024-09-21T01:50:09.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.049+0000 7f5a67c96740 20 client.25507 trim_cache size 0 max 16384 2024-09-21T01:50:09.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.068+0000 7f5a67c96740 1 Processor -- start 2024-09-21T01:50:09.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 -- start start 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b60b0 con 0x55f8c3874390 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a5affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a5affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45498/0 (socket says 172.21.15.29:45498) 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a5affd640 1 -- 172.21.15.29:0/3473882976 learned_addr learned my addr 172.21.15.29:0/3473882976 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b7ce0 con 0x55f8c386c1b0 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.069+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b6780 con 0x55f8c3895070 2024-09-21T01:50:09.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c44c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 -- 172.21.15.29:0/3473882976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 msgr2=0x55f8c3c44c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c44c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 -- 172.21.15.29:0/3473882976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 msgr2=0x55f8c3c3f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 -- 172.21.15.29:0/3473882976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c3c48ac0 con 0x55f8c386c1b0 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.070+0000 7f5a5affd640 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c008010 tx=0x7f5a4c00c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab2e99ac54548a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.071+0000 7f5a50ff9640 1 -- 172.21.15.29:0/3473882976 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c02e030 con 0x55f8c386c1b0 2024-09-21T01:50:09.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.071+0000 7f5a50ff9640 1 -- 172.21.15.29:0/3473882976 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4c030070 con 0x55f8c386c1b0 2024-09-21T01:50:09.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.071+0000 7f5a50ff9640 1 -- 172.21.15.29:0/3473882976 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c004020 con 0x55f8c386c1b0 2024-09-21T01:50:09.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 msgr2=0x55f8c3c3f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c008010 tx=0x7f5a4c00c5f0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 shutdown_connections 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c44c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c3874390 0x55f8c3c3f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 --2- 172.21.15.29:0/3473882976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c386c1b0 0x55f8c3c3f3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 >> 172.21.15.29:0/3473882976 conn(0x55f8c3889460 msgr2=0x55f8c3c462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.073+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 shutdown_connections 2024-09-21T01:50:09.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.074+0000 7f5a67c96740 1 -- 172.21.15.29:0/3473882976 wait complete. 2024-09-21T01:50:09.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.074+0000 7f5a67c96740 1 -- start start 2024-09-21T01:50:09.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 Processor -- start 2024-09-21T01:50:09.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c4bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3874390 0x55f8c3c4c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b7550 con 0x55f8c386c1b0 2024-09-21T01:50:09.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b6e80 con 0x55f8c3874390 2024-09-21T01:50:09.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1b40b0 con 0x55f8c3895070 2024-09-21T01:50:09.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a67c96740 10 client.0 _ll_register_callbacks cb 0x7f5a4c00ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:09.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:42468/0 (socket says 172.21.15.29:42468) 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.075+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/2912514702 learned_addr learned my addr 172.21.15.29:0/2912514702 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5affd640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c4bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3874390 msgr2=0x55f8c3c4c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3874390 0x55f8c3c4c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/2912514702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 msgr2=0x55f8c3c4bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c4bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 -- 172.21.15.29:0/2912514702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c3cc3800 con 0x55f8c3895070 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5affd640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a5b7fe640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e5a0 tx=0x7f5a54009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35aacc0de9cae065 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5403c060 con 0x55f8c3895070 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a54004310 con 0x55f8c3895070 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4c030070 con 0x55f8c3895070 2024-09-21T01:50:09.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a54017040 con 0x55f8c3895070 2024-09-21T01:50:09.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a67c96740 10 client.25513 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:09.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.076+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f8c3cc4a80 con 0x55f8c3895070 2024-09-21T01:50:09.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.079+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4c030070 con 0x55f8c3895070 2024-09-21T01:50:09.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.079+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5a540038a0 con 0x55f8c3895070 2024-09-21T01:50:09.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 10 client.25513 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 10 client.25513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 20 client.25513 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c3828da0 0x55f8c3cc6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 4 client.25513 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8c3bb4df0 con 0x55f8c3828da0 2024-09-21T01:50:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.080+0000 7f5a5affd640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c3828da0 0x55f8c3cc6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.081+0000 7f5a4a7fc640 10 client.25513 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.081+0000 7f5a5affd640 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c3828da0 0x55f8c3cc6ef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3f8af2b87a4bf6 server_cookie=96f1b6df5275a17b in_seq=0 out_seq=0 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a4a7fc640 1 -- 172.21.15.29:0/2912514702 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8c3bb4df0 con 0x55f8c3828da0 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a4a7fc640 10 client.25513 handle_command_reply: tid=0 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 client.25513 shutdown 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 msgr2=0x55f8c3c4c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e5a0 tx=0x7f5a54009700 comp rx=0 tx=0).stop 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 shutdown_connections 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f8c3828da0 0x55f8c3cc6ef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8c3895070 0x55f8c3c4c850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8c3874390 0x55f8c3c4c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 --2- 172.21.15.29:0/2912514702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f8c386c1b0 0x55f8c3c4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 >> 172.21.15.29:0/2912514702 conn(0x55f8c3889460 msgr2=0x55f8c3c3f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a4a7fc640 0 client.25513 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 shutdown_connections 2024-09-21T01:50:09.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 1 -- 172.21.15.29:0/2912514702 wait complete. 2024-09-21T01:50:09.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.082+0000 7f5a67c96740 20 client.25513 trim_cache size 0 max 16384 2024-09-21T01:50:09.085 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:09.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.083+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 msgr2=0x7f5a3c0b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.083+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 0x7f5a3c0b8b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55f8c3c3af40 tx=0x7f5a4c008040 comp rx=0 tx=0).stop 2024-09-21T01:50:09.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.083+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 msgr2=0x7f5a5c1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.083+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009bf0 tx=0x7f5a44003ce0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 shutdown_connections 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a3c0b6640 0x7f5a3c0b8b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a5c1a65c0 0x7f5a5c1a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a5c108cf0 0x7f5a5c1a13e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 --2- 172.21.15.29:0/1528067281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a5c102a60 0x7f5a5c1a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 >> 172.21.15.29:0/1528067281 conn(0x7f5a5c079730 msgr2=0x7f5a5c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 shutdown_connections 2024-09-21T01:50:09.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:09.084+0000 7f5a61722640 1 -- 172.21.15.29:0/1528067281 wait complete. 2024-09-21T01:50:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 -- 172.21.15.29:0/227901324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 msgr2=0x7effbc104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 --2- 172.21.15.29:0/227901324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 0x7effbc104b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7effac009900 tx=0x7effac02a410 comp rx=0 tx=0).stop 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 -- 172.21.15.29:0/227901324 shutdown_connections 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 --2- 172.21.15.29:0/227901324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 --2- 172.21.15.29:0/227901324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 0x7effbc104b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 --2- 172.21.15.29:0/227901324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 0x7effbc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 -- 172.21.15.29:0/227901324 >> 172.21.15.29:0/227901324 conn(0x7effbc0ff020 msgr2=0x7effbc1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 -- 172.21.15.29:0/227901324 shutdown_connections 2024-09-21T01:50:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.284+0000 7effc2070640 1 -- 172.21.15.29:0/227901324 wait complete. 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 Processor -- start 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 -- start start 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 0x7effbc1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 0x7effbc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effbc077200 con 0x7effbc1036c0 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effbc077080 con 0x7effbc1050e0 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.285+0000 7effc2070640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effbc077380 con 0x7effbc104740 2024-09-21T01:50:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45536/0 (socket says 172.21.15.29:45536) 2024-09-21T01:50:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 -- 172.21.15.29:0/1752481368 learned_addr learned my addr 172.21.15.29:0/1752481368 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbb7fe640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 0x7effbc1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 -- 172.21.15.29:0/1752481368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 msgr2=0x7effbc1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 0x7effbc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 -- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 msgr2=0x7effbc1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 0x7effbc1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.286+0000 7effbbfff640 1 -- 172.21.15.29:0/1752481368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1a1fb0 con 0x7effbc1050e0 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.287+0000 7effbbfff640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effb400eb10 tx=0x7effb400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.287+0000 7effb8ff9640 1 -- 172.21.15.29:0/1752481368 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4004b20 con 0x7effbc1050e0 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.287+0000 7effb8ff9640 1 -- 172.21.15.29:0/1752481368 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb4004cc0 con 0x7effbc1050e0 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.288+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1a22a0 con 0x7effbc1050e0 2024-09-21T01:50:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.288+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc1a8990 con 0x7effbc1050e0 2024-09-21T01:50:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.288+0000 7effb8ff9640 1 -- 172.21.15.29:0/1752481368 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4010670 con 0x7effbc1050e0 2024-09-21T01:50:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.291+0000 7effc8fdd740 1 Processor -- start 2024-09-21T01:50:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effb8ff9640 1 -- 172.21.15.29:0/1752481368 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7effb40108f0 con 0x7effbc1050e0 2024-09-21T01:50:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 -- start start 2024-09-21T01:50:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efc0e1b0 0x55c4effb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbed4d0 0x55c4effb7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c4eff286f0 con 0x55c4efbed4d0 2024-09-21T01:50:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c4eff17b20 con 0x55c4efc0e1b0 2024-09-21T01:50:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.292+0000 7effc8fdd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c4eff1b930 con 0x55c4efbe52f0 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:42492/0 (socket says 172.21.15.29:42492) 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 -- 172.21.15.29:0/326592179 learned_addr learned my addr 172.21.15.29:0/326592179 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 -- 172.21.15.29:0/326592179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efc0e1b0 msgr2=0x55c4effb5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efc0e1b0 0x55c4effb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 -- 172.21.15.29:0/326592179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbed4d0 msgr2=0x55c4effb7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.293+0000 7effbb7fe640 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbed4d0 0x55c4effb7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.294+0000 7effbb7fe640 1 -- 172.21.15.29:0/326592179 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4efedd530 con 0x55c4efbe52f0 2024-09-21T01:50:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.294+0000 7effb8ff9640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 0x7effa40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.294+0000 7effb8ff9640 1 -- 172.21.15.29:0/1752481368 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7effb40e77d0 con 0x7effbc1050e0 2024-09-21T01:50:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.294+0000 7effbb7fe640 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb0009cf0 tx=0x7effb0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f5fa5e4e38c81ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.294+0000 7effa1ffb640 1 -- 172.21.15.29:0/326592179 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0004230 con 0x55c4efbe52f0 2024-09-21T01:50:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.295+0000 7effa1ffb640 1 -- 172.21.15.29:0/326592179 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb00043d0 con 0x55c4efbe52f0 2024-09-21T01:50:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.295+0000 7effbaffd640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 0x7effa40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.295+0000 7effa1ffb640 1 -- 172.21.15.29:0/326592179 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0019750 con 0x55c4efbe52f0 2024-09-21T01:50:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effbaffd640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 0x7effa40b8b70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effac005f70 tx=0x7effac005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- 172.21.15.29:0/326592179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 msgr2=0x55c4effb64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb0009cf0 tx=0x7effb0011810 comp rx=0 tx=0).stop 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- 172.21.15.29:0/326592179 shutdown_connections 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbed4d0 0x55c4effb7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb64c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 --2- 172.21.15.29:0/326592179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efc0e1b0 0x55c4effb5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- 172.21.15.29:0/326592179 >> 172.21.15.29:0/326592179 conn(0x55c4efc02300 msgr2=0x55c4effb18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- 172.21.15.29:0/326592179 shutdown_connections 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- 172.21.15.29:0/326592179 wait complete. 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.296+0000 7effc8fdd740 1 -- start start 2024-09-21T01:50:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 Processor -- start 2024-09-21T01:50:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effbfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efc0e1b0 0x55c4effc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c4efe4fab0 con 0x55c4efbe52f0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c4efebd730 con 0x55c4efbed4d0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c4effb7bd0 con 0x55c4efc0e1b0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effc8fdd740 10 client.0 _ll_register_callbacks cb 0x55c4eff116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effbbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efc0e1b0 0x55c4effc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52048/0 (socket says 172.21.15.29:52048) 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.297+0000 7effbb7fe640 1 -- 172.21.15.29:0/2278906341 learned_addr learned my addr 172.21.15.29:0/2278906341 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbaffd640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effbfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 -- 172.21.15.29:0/2278906341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efc0e1b0 msgr2=0x55c4effc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efc0e1b0 0x55c4effc0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 -- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 msgr2=0x55c4effbfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effbfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 -- 172.21.15.29:0/2278906341 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4efc79790 con 0x55c4efbe52f0 2024-09-21T01:50:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7effbb7fe640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7effb0000c00 tx=0x7effb0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d12c89464243af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.298+0000 7eff9affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0027070 con 0x55c4efbe52f0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7effb0005240 con 0x55c4efbe52f0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb00043d0 con 0x55c4efbe52f0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7effb0005440 con 0x55c4efbe52f0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7effc8fdd740 10 client.15780 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4efcf2620 con 0x55c4efbe52f0 2024-09-21T01:50:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.299+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb00055e0 con 0x55c4efbe52f0 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7effb0019620 con 0x55c4efbe52f0 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 10 client.15780 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 20 client.15780 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efbccd80 0x55c4effccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 4 client.15780 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7effb00043d0 con 0x55c4efbccd80 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.300+0000 7effbaffd640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efbccd80 0x55c4effccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.301+0000 7effbaffd640 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efbccd80 0x55c4effccb50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190c722c1c0621b6 server_cookie=28176cfd138ba48e in_seq=0 out_seq=0 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.301+0000 7eff9affd640 10 client.15780 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.302+0000 7eff9affd640 1 -- 172.21.15.29:0/2278906341 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7effb00043d0 con 0x55c4efbccd80 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.302+0000 7eff9affd640 10 client.15780 handle_command_reply: tid=0 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.303+0000 7effc8fdd740 1 client.15780 shutdown 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.303+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 msgr2=0x55c4effbf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.303+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7effb0000c00 tx=0x7effb0003820 comp rx=0 tx=0).stop 2024-09-21T01:50:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 shutdown_connections 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efbccd80 0x55c4effccb50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7eff9affd640 0 client.15780 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efc0e1b0 0x55c4effc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effbfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 --2- 172.21.15.29:0/2278906341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efbe52f0 0x55c4effbf5d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 >> 172.21.15.29:0/2278906341 conn(0x55c4efc02300 msgr2=0x55c4effafea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 shutdown_connections 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 1 -- 172.21.15.29:0/2278906341 wait complete. 2024-09-21T01:50:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.304+0000 7effc8fdd740 20 client.15780 trim_cache size 0 max 16384 2024-09-21T01:50:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.324+0000 7effc8fdd740 1 Processor -- start 2024-09-21T01:50:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.324+0000 7effc8fdd740 1 -- start start 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1b15f0 con 0x55c4efc0e1b0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1b3220 con 0x55c4efbed4d0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effc8fdd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1b1cc0 con 0x55c4efbe52f0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45562/0 (socket says 172.21.15.29:45562) 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 -- 172.21.15.29:0/3487806165 learned_addr learned my addr 172.21.15.29:0/3487806165 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52052/0 (socket says 172.21.15.29:52052) 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbb7fe640 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 -- 172.21.15.29:0/3487806165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 msgr2=0x55c4effb1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 -- 172.21.15.29:0/3487806165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 msgr2=0x55c4effc76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 -- 172.21.15.29:0/3487806165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4effbd6c0 con 0x55c4efbed4d0 2024-09-21T01:50:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.325+0000 7effbaffd640 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7effac005640 tx=0x7effac03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9f0c925f69fe6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.326+0000 7effa17fa640 1 -- 172.21.15.29:0/3487806165 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac035040 con 0x55c4efbed4d0 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.326+0000 7effa17fa640 1 -- 172.21.15.29:0/3487806165 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effac040030 con 0x55c4efbed4d0 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.326+0000 7effa17fa640 1 -- 172.21.15.29:0/3487806165 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac004f70 con 0x55c4efbed4d0 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- 172.21.15.29:0/3487806165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 msgr2=0x55c4effb6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7effac005640 tx=0x7effac03d000 comp rx=0 tx=0).stop 2024-09-21T01:50:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- 172.21.15.29:0/3487806165 shutdown_connections 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effb6c90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3487806165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effb1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- 172.21.15.29:0/3487806165 >> 172.21.15.29:0/3487806165 conn(0x55c4efc02300 msgr2=0x55c4effac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- 172.21.15.29:0/3487806165 shutdown_connections 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- 172.21.15.29:0/3487806165 wait complete. 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.327+0000 7effc8fdd740 1 -- start start 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 Processor -- start 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effc7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1b2a90 con 0x55c4efc0e1b0 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1b23c0 con 0x55c4efbed4d0 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1af5d0 con 0x55c4efbe52f0 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:42526/0 (socket says 172.21.15.29:42526) 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effbb7fe640 1 -- 172.21.15.29:0/3064055005 learned_addr learned my addr 172.21.15.29:0/3064055005 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effbaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effc7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.328+0000 7effc8fdd740 10 client.0 _ll_register_callbacks cb 0x7effb0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 -- 172.21.15.29:0/3064055005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 msgr2=0x55c4effc6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 -- 172.21.15.29:0/3064055005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 msgr2=0x55c4effc2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 -- 172.21.15.29:0/3064055005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4effc3710 con 0x55c4efbed4d0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbb7fe640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbbfff640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effbaffd640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effc7660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effac02aa00 tx=0x7effac005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2282fbd200b9e767 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7eff9affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac035040 con 0x55c4efbed4d0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7effac004d10 con 0x55c4efbed4d0 2024-09-21T01:50:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effac040030 con 0x55c4efbed4d0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7effac004950 con 0x55c4efbed4d0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effc8fdd740 10 client.25490 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.329+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4effc4d20 con 0x55c4efbed4d0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effac040030 con 0x55c4efbed4d0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7effac05f030 con 0x55c4efbed4d0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 10 client.25490 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 10 client.25490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 20 client.25490 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efba21c0 0x55c4f0039e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 4 client.25490 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4eff2a640 con 0x55c4efba21c0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effbb7fe640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efba21c0 0x55c4f0039e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7effbb7fe640 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efba21c0 0x55c4f0039e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e4dc53bce8bd78 server_cookie=40f324abb20da54b in_seq=0 out_seq=0 2024-09-21T01:50:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.330+0000 7eff9affd640 10 client.25490 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.346+0000 7eff9affd640 1 -- 172.21.15.29:0/3064055005 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4eff2a640 con 0x55c4efba21c0 2024-09-21T01:50:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.346+0000 7eff9affd640 10 client.25490 handle_command_reply: tid=0 2024-09-21T01:50:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 client.25490 shutdown 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 msgr2=0x55c4effc7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effc7660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effac02aa00 tx=0x7effac005270 comp rx=0 tx=0).stop 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 shutdown_connections 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c4efba21c0 0x55c4f0039e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c4efc0e1b0 0x55c4effc2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4efbed4d0 0x55c4effc7660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7eff9affd640 0 client.25490 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 --2- 172.21.15.29:0/3064055005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c4efbe52f0 0x55c4effc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 >> 172.21.15.29:0/3064055005 conn(0x55c4efc02300 msgr2=0x55c4effb6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 shutdown_connections 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 1 -- 172.21.15.29:0/3064055005 wait complete. 2024-09-21T01:50:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.347+0000 7effc8fdd740 20 client.25490 trim_cache size 0 max 16384 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.348+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 msgr2=0x7effa40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.348+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 0x7effa40b8b70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effac005f70 tx=0x7effac005de0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.348+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 msgr2=0x7effbc1a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.348+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effb400eb10 tx=0x7effb400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 shutdown_connections 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7effa40b6660 0x7effa40b8b70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc1050e0 0x7effbc1a1790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effbc104740 0x7effbc1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 --2- 172.21.15.29:0/1752481368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effbc1036c0 0x7effbc1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.349+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 >> 172.21.15.29:0/1752481368 conn(0x7effbc0ff020 msgr2=0x7effbc10fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.350+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 shutdown_connections 2024-09-21T01:50:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:09.350+0000 7effc2070640 1 -- 172.21.15.29:0/1752481368 wait complete. 2024-09-21T01:50:09.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:09 smithi066 ceph-mon[35001]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-21T01:50:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:09 smithi029 ceph-mon[31228]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-21T01:50:09.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:09 smithi190 ceph-mon[34368]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-21T01:50:11.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:11 smithi066 ceph-mon[35001]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-09-21T01:50:11.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:11 smithi029 ceph-mon[31228]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-09-21T01:50:11.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:11 smithi190 ceph-mon[34368]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-09-21T01:50:12.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:50:12.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:50:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 95.0 s, 3828.2 tps, lat 26.129 ms stddev 31.038 2024-09-21T01:50:13.136 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:13.503 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:13 smithi029 ceph-mon[31228]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-21T01:50:13.548 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:13 smithi066 ceph-mon[35001]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-21T01:50:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:13 smithi190 ceph-mon[34368]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.774+0000 7fac370e4640 1 -- 172.21.15.29:0/1161735336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30107b30 msgr2=0x7fac30107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.774+0000 7fac370e4640 1 --2- 172.21.15.29:0/1161735336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30107b30 0x7fac30107f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac28009900 tx=0x7fac2802a400 comp rx=0 tx=0).stop 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 -- 172.21.15.29:0/1161735336 shutdown_connections 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 --2- 172.21.15.29:0/1161735336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac30108480 0x7fac3010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 --2- 172.21.15.29:0/1161735336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30107b30 0x7fac30107f40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 --2- 172.21.15.29:0/1161735336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac30102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 -- 172.21.15.29:0/1161735336 >> 172.21.15.29:0/1161735336 conn(0x7fac30077c60 msgr2=0x7fac300780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 -- 172.21.15.29:0/1161735336 shutdown_connections 2024-09-21T01:50:13.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.775+0000 7fac370e4640 1 -- 172.21.15.29:0/1161735336 wait complete. 2024-09-21T01:50:13.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.776+0000 7fac370e4640 1 Processor -- start 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.776+0000 7fac370e4640 1 -- start start 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.776+0000 7fac370e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.776+0000 7fac370e4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac30107b30 0x7fac300718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac370e4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 0x7fac30071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac370e4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac3006d6a0 con 0x7fac30108480 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac370e4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac3006d520 con 0x7fac301021f0 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac370e4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac3006d820 con 0x7fac30107b30 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:33888/0 (socket says 172.21.15.29:33888) 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 -- 172.21.15.29:0/231992371 learned_addr learned my addr 172.21.15.29:0/231992371 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac35681640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 0x7fac30071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 -- 172.21.15.29:0/231992371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac30107b30 msgr2=0x7fac300718f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:50:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac30107b30 0x7fac300718f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 -- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 msgr2=0x7fac30071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 0x7fac30071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac34e80640 1 -- 172.21.15.29:0/231992371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301b0fb0 con 0x7fac301021f0 2024-09-21T01:50:13.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.777+0000 7fac35681640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 0x7fac30071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:13.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.778+0000 7fac34e80640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac1800ea60 tx=0x7fac1800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.779+0000 7fac2dffb640 1 -- 172.21.15.29:0/231992371 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cde0 con 0x7fac301021f0 2024-09-21T01:50:13.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.779+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301b12a0 con 0x7fac301021f0 2024-09-21T01:50:13.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.779+0000 7fac2dffb640 1 -- 172.21.15.29:0/231992371 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180035f0 con 0x7fac301021f0 2024-09-21T01:50:13.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.779+0000 7fac2dffb640 1 -- 172.21.15.29:0/231992371 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18010690 con 0x7fac301021f0 2024-09-21T01:50:13.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.779+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301b17e0 con 0x7fac301021f0 2024-09-21T01:50:13.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.784+0000 7fac3d658740 1 Processor -- start 2024-09-21T01:50:13.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.785+0000 7fac2dffb640 1 -- 172.21.15.29:0/231992371 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fac18010830 con 0x7fac301021f0 2024-09-21T01:50:13.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.785+0000 7fac3d658740 1 -- start start 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 0x562c17d7b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d7beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562c17cf1e60 con 0x562c179d4350 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562c17ce1290 con 0x562c179ab490 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac3d658740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562c17ce50a0 con 0x562c179b3670 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac34e80640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 0x562c17d7b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac2ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d7beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac2ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d7beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51164/0 (socket says 172.21.15.29:51164) 2024-09-21T01:50:13.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.786+0000 7fac2ffff640 1 -- 172.21.15.29:0/4846942 learned_addr learned my addr 172.21.15.29:0/4846942 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.787+0000 7fac34e80640 1 -- 172.21.15.29:0/4846942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 msgr2=0x562c17d7beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.787+0000 7fac34e80640 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d7beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.787+0000 7fac34e80640 1 -- 172.21.15.29:0/4846942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 msgr2=0x562c17d7b430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.787+0000 7fac34e80640 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d7b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.787+0000 7fac34e80640 1 -- 172.21.15.29:0/4846942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c17a3f930 con 0x562c179ab490 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.788+0000 7fac2dffb640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 0x7fac1c0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.788+0000 7fac2dffb640 1 -- 172.21.15.29:0/231992371 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac180e7740 con 0x7fac301021f0 2024-09-21T01:50:13.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.788+0000 7fac34e80640 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 0x562c17d7b970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac180e5190 tx=0x7fac180edb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8533f3a116b223b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.790+0000 7fac26ffd640 1 -- 172.21.15.29:0/4846942 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180a9ed0 con 0x562c179ab490 2024-09-21T01:50:13.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.790+0000 7fac26ffd640 1 -- 172.21.15.29:0/4846942 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180ad390 con 0x562c179ab490 2024-09-21T01:50:13.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.791+0000 7fac35681640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 0x7fac1c0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.791+0000 7fac3d658740 1 -- 172.21.15.29:0/4846942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 msgr2=0x562c17d7b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.791+0000 7fac3d658740 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 0x562c17d7b970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac180e5190 tx=0x7fac180edb20 comp rx=0 tx=0).stop 2024-09-21T01:50:13.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.799+0000 7fac3d658740 1 -- 172.21.15.29:0/4846942 shutdown_connections 2024-09-21T01:50:13.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.799+0000 7fac3d658740 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d7beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.799+0000 7fac3d658740 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179ab490 0x562c17d7b970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.799+0000 7fac3d658740 1 --2- 172.21.15.29:0/4846942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.799+0000 7fac3d658740 1 -- 172.21.15.29:0/4846942 >> 172.21.15.29:0/4846942 conn(0x562c179c8740 msgr2=0x562c17d79320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.800+0000 7fac3d658740 1 -- 172.21.15.29:0/4846942 shutdown_connections 2024-09-21T01:50:13.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.800+0000 7fac3d658740 1 -- 172.21.15.29:0/4846942 wait complete. 2024-09-21T01:50:13.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.801+0000 7fac3d658740 1 -- start start 2024-09-21T01:50:13.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.801+0000 7fac3d658740 1 Processor -- start 2024-09-21T01:50:13.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac35681640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 0x7fac1c0b8bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac20005fd0 tx=0x7fac20009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d86f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d9b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562c17b42890 con 0x562c179b3670 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562c17c7bd40 con 0x562c179d4350 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562c17d92470 con 0x562c179ab490 2024-09-21T01:50:13.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.802+0000 7fac3d658740 10 client.0 _ll_register_callbacks cb 0x562c17cdfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:13.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.806+0000 7fac2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.806+0000 7fac2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d874c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46698/0 (socket says 172.21.15.29:46698) 2024-09-21T01:50:13.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.806+0000 7fac2ffff640 1 -- 172.21.15.29:0/1418112295 learned_addr learned my addr 172.21.15.29:0/1418112295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:13.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.806+0000 7fac35681640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d9b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.806+0000 7fac34e80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d86f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 -- 172.21.15.29:0/1418112295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 msgr2=0x562c17d86f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d86f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 -- 172.21.15.29:0/1418112295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 msgr2=0x562c17d874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 -- 172.21.15.29:0/1418112295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c17d898c0 con 0x562c179d4350 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac35681640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d9b7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fac20010c30 tx=0x7fac20004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcacfc750f38454 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2000eea0 con 0x562c179d4350 2024-09-21T01:50:13.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.809+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac180ad390 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.811+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac2001e420 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.812+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac2001e620 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 10 client.25502 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c17d8c2a0 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180ad390 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fac20026860 con 0x562c179d4350 2024-09-21T01:50:13.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 10 client.25502 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:13.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:13.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 20 client.25502 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17992cd0 0x562c17d8fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 4 client.25502 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.813+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c17cf3db0 con 0x562c17992cd0 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.814+0000 7fac34e80640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17992cd0 0x562c17d8fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.815+0000 7fac34e80640 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17992cd0 0x562c17d8fc90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f15ece734936025 server_cookie=23d8b7e642c0828d in_seq=0 out_seq=0 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.816+0000 7fac0ffff640 10 client.25502 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.816+0000 7fac0ffff640 1 -- 172.21.15.29:0/1418112295 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c17cf3db0 con 0x562c17992cd0 2024-09-21T01:50:13.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.816+0000 7fac0ffff640 10 client.25502 handle_command_reply: tid=0 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 client.25502 shutdown 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 msgr2=0x562c17d9b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d9b7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fac20010c30 tx=0x7fac20004ab0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 shutdown_connections 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17992cd0 0x562c17d8fc90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d9b7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 --2- 172.21.15.29:0/1418112295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d86f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 >> 172.21.15.29:0/1418112295 conn(0x562c179c8740 msgr2=0x562c17d7c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.817+0000 7fac0ffff640 0 client.25502 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.818+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 shutdown_connections 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.818+0000 7fac3d658740 1 -- 172.21.15.29:0/1418112295 wait complete. 2024-09-21T01:50:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.818+0000 7fac3d658740 20 client.25502 trim_cache size 0 max 16384 2024-09-21T01:50:13.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.853+0000 7fac3d658740 1 Processor -- start 2024-09-21T01:50:13.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.853+0000 7fac3d658740 1 -- start start 2024-09-21T01:50:13.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d91450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b5990 con 0x562c179b3670 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b75c0 con 0x562c179d4350 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac3d658740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b6060 con 0x562c179ab490 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51188/0 (socket says 172.21.15.29:51188) 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 -- 172.21.15.29:0/2230335883 learned_addr learned my addr 172.21.15.29:0/2230335883 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac35681640 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d91450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 -- 172.21.15.29:0/2230335883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 msgr2=0x562c17d91450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d91450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.854+0000 7fac34e80640 1 -- 172.21.15.29:0/2230335883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 msgr2=0x562c17d889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac34e80640 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac34e80640 1 -- 172.21.15.29:0/2230335883 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c17992f00 con 0x562c179ab490 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac34e80640 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac180ad580 tx=0x7fac1800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59388659b7184622 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac267fc640 1 -- 172.21.15.29:0/2230335883 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180ad1d0 con 0x562c179ab490 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac267fc640 1 -- 172.21.15.29:0/2230335883 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180ed300 con 0x562c179ab490 2024-09-21T01:50:13.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.855+0000 7fac267fc640 1 -- 172.21.15.29:0/2230335883 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180ad650 con 0x562c179ab490 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.856+0000 7fac3d658740 1 -- 172.21.15.29:0/2230335883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 msgr2=0x562c17d81550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.856+0000 7fac3d658740 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac180ad580 tx=0x7fac1800ef90 comp rx=0 tx=0).stop 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 -- 172.21.15.29:0/2230335883 shutdown_connections 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c179d4350 0x562c17d91450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179b3670 0x562c17d889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 --2- 172.21.15.29:0/2230335883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179ab490 0x562c17d81550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 -- 172.21.15.29:0/2230335883 >> 172.21.15.29:0/2230335883 conn(0x562c179c8740 msgr2=0x562c17d882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 -- 172.21.15.29:0/2230335883 shutdown_connections 2024-09-21T01:50:13.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 -- 172.21.15.29:0/2230335883 wait complete. 2024-09-21T01:50:13.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 -- start start 2024-09-21T01:50:13.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.857+0000 7fac3d658740 1 Processor -- start 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac3d658740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac3d658740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac34e80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac34e80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51194/0 (socket says 172.21.15.29:51194) 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac34e80640 1 -- 172.21.15.29:0/1978488955 learned_addr learned my addr 172.21.15.29:0/1978488955 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.858+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c17968080 0x562c17d92700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac2ffff640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b6e30 con 0x562c179d4350 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac35681640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c17968080 0x562c17d92700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b6760 con 0x562c17968080 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac301b3990 con 0x562c179b3670 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac34e80640 1 -- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c17968080 msgr2=0x562c17d92700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac34e80640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c17968080 0x562c17d92700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac34e80640 1 -- 172.21.15.29:0/1978488955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 msgr2=0x562c17d97c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac34e80640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d97c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.859+0000 7fac34e80640 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c17d98400 con 0x562c179b3670 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac34e80640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac1800c760 tx=0x7fac180a90b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604474675b5c05c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac3d658740 10 client.0 _ll_register_callbacks cb 0x7fac200098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac180ed300 con 0x562c179b3670 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac0ffff640 10 client.25543 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac0ffff640 1 -- 172.21.15.29:0/1978488955 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180a9720 con 0x562c179b3670 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac0ffff640 1 -- 172.21.15.29:0/1978488955 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac180a9af0 con 0x562c179b3670 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac0ffff640 1 -- 172.21.15.29:0/1978488955 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac180ad650 con 0x562c179b3670 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac3d658740 10 client.25543 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:13.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.860+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=31}) -- 0x562c17e15070 con 0x562c179b3670 2024-09-21T01:50:13.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac0ffff640 1 -- 172.21.15.29:0/1978488955 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fac180b6e10 con 0x562c179b3670 2024-09-21T01:50:13.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 10 client.25543 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:13.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:13.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 20 client.25543 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17e15240 0x562c17e176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 4 client.25543 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fac180ed300 con 0x562c17e15240 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.861+0000 7fac2ffff640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17e15240 0x562c17e176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.862+0000 7fac2ffff640 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17e15240 0x562c17e176c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685f0d05cf254764 server_cookie=2f7a10fa2ff98a43 in_seq=0 out_seq=0 2024-09-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.862+0000 7fac0ffff640 10 client.25543 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.864+0000 7fac0ffff640 1 -- 172.21.15.29:0/1978488955 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fac180ed300 con 0x562c17e15240 2024-09-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.864+0000 7fac0ffff640 10 client.25543 handle_command_reply: tid=0 2024-09-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.864+0000 7fac3d658740 1 client.25543 shutdown 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 msgr2=0x562c17d97750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac1800c760 tx=0x7fac180a90b0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 shutdown_connections 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562c17e15240 0x562c17e176c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c17968080 0x562c17d92700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c179d4350 0x562c17d97c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac0ffff640 0 client.25543 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 --2- 172.21.15.29:0/1978488955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c179b3670 0x562c17d97750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 >> 172.21.15.29:0/1978488955 conn(0x562c179c8740 msgr2=0x562c17d88040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.865+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 shutdown_connections 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.866+0000 7fac3d658740 1 -- 172.21.15.29:0/1978488955 wait complete. 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.866+0000 7fac3d658740 20 client.25543 trim_cache size 0 max 16384 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "91217b03-65df-4a37-8426-4ace79f8bff9", 2024-09-21T01:50:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:50:13.868 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 msgr2=0x7fac1c0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 0x7fac1c0b8bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac20005fd0 tx=0x7fac20009500 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 msgr2=0x7fac300713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac1800ea60 tx=0x7fac1800ef20 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 shutdown_connections 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fac1c0b66e0 0x7fac1c0b8bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.867+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac30108480 0x7fac30071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.868+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac30107b30 0x7fac300718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.868+0000 7fac370e4640 1 --2- 172.21.15.29:0/231992371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac301021f0 0x7fac300713b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.868+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 >> 172.21.15.29:0/231992371 conn(0x7fac30077c60 msgr2=0x7fac30105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.868+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 shutdown_connections 2024-09-21T01:50:13.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:13.868+0000 7fac370e4640 1 -- 172.21.15.29:0/231992371 wait complete. 2024-09-21T01:50:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:14 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:14 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:14 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:14.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:14 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:14.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:14 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:14.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:14 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:14.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:14 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:14.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:14 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:14.679 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:50:14.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:14 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:14.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:14 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:14.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:14 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:14.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:14 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:15.548 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.549 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:15 smithi029 ceph-mon[31228]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 735 op/s 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.551 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:15 smithi066 ceph-mon[35001]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 735 op/s 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.690+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3368334702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3810de50 msgr2=0x7f8a38110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.690+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3368334702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3810de50 0x7f8a38110340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009ca0 tx=0x7f8a3002a8f0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.691+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3368334702 shutdown_connections 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.691+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3368334702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3810de50 0x7f8a38110340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.691+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3368334702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a3810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.691+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3368334702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a381049f0 0x7f8a38104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.691+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3368334702 >> 172.21.15.29:0/3368334702 conn(0x7f8a38079750 msgr2=0x7f8a38079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.692+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3368334702 shutdown_connections 2024-09-21T01:50:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.692+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3368334702 wait complete. 2024-09-21T01:50:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.693+0000 7f8a3ddbe640 1 Processor -- start 2024-09-21T01:50:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.693+0000 7f8a3ddbe640 1 -- start start 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 0x7f8a38102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a38102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 0x7f8a381031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38113c00 con 0x7f8a38105340 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38113a80 con 0x7f8a3810de50 2024-09-21T01:50:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a3ddbe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38113d80 con 0x7f8a381049f0 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 0x7f8a38102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 0x7f8a38102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51218/0 (socket says 172.21.15.29:51218) 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a377fe640 1 -- 172.21.15.29:0/3073782252 learned_addr learned my addr 172.21.15.29:0/3073782252 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a37fff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 0x7f8a381031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.694+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a38102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 -- 172.21.15.29:0/3073782252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 msgr2=0x7f8a38102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 0x7f8a38102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 -- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 msgr2=0x7f8a38102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a38102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 -- 172.21.15.29:0/3073782252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a381a8870 con 0x7f8a3810de50 2024-09-21T01:50:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a38102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a37fff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 0x7f8a381031b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002ae10 tx=0x7f8a30031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.695+0000 7f8a357fa640 1 -- 172.21.15.29:0/3073782252 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30003db0 con 0x7f8a3810de50 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.696+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a381a8b00 con 0x7f8a3810de50 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.696+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a9090 con 0x7f8a3810de50 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.696+0000 7f8a357fa640 1 -- 172.21.15.29:0/3073782252 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3002d070 con 0x7f8a3810de50 2024-09-21T01:50:15.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.696+0000 7f8a357fa640 1 -- 172.21.15.29:0/3073782252 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3002fbd0 con 0x7f8a3810de50 2024-09-21T01:50:15.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.699+0000 7f8a44332740 1 Processor -- start 2024-09-21T01:50:15.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.700+0000 7f8a357fa640 1 -- 172.21.15.29:0/3073782252 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8a3002fd70 con 0x7f8a3810de50 2024-09-21T01:50:15.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.700+0000 7f8a44332740 1 -- start start 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c789316390 0x55c7896decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c789654e50 con 0x55c78930e1b0 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c789644280 con 0x55c789337070 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a44332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c789648090 con 0x55c789316390 2024-09-21T01:50:15.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c789316390 0x55c7896decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46732/0 (socket says 172.21.15.29:46732) 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a377fe640 1 -- 172.21.15.29:0/2632963433 learned_addr learned my addr 172.21.15.29:0/2632963433 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.701+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 -- 172.21.15.29:0/2632963433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c789316390 msgr2=0x55c7896decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c789316390 0x55c7896decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 -- 172.21.15.29:0/2632963433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 msgr2=0x55c7896de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 -- 172.21.15.29:0/2632963433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7893a2650 con 0x55c78930e1b0 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.702+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009ae0 tx=0x7f8a2800e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86471389af8d5786 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.703+0000 7f8a357fa640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 0x7f8a240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.703+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 0x7f8a240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.704+0000 7f8a357fa640 1 -- 172.21.15.29:0/3073782252 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a30108410 con 0x7f8a3810de50 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.704+0000 7f8a2dffb640 1 -- 172.21.15.29:0/2632963433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28017070 con 0x55c78930e1b0 2024-09-21T01:50:15.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.704+0000 7f8a2dffb640 1 -- 172.21.15.29:0/2632963433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2800ebb0 con 0x55c78930e1b0 2024-09-21T01:50:15.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.704+0000 7f8a2dffb640 1 -- 172.21.15.29:0/2632963433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28012630 con 0x55c78930e1b0 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 0x7f8a240b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a3810fba0 tx=0x7f8a20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 -- 172.21.15.29:0/2632963433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 msgr2=0x55c7896de7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009ae0 tx=0x7f8a2800e1d0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 -- 172.21.15.29:0/2632963433 shutdown_connections 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c789316390 0x55c7896decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78930e1b0 0x55c7896de7b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 --2- 172.21.15.29:0/2632963433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.705+0000 7f8a44332740 1 -- 172.21.15.29:0/2632963433 >> 172.21.15.29:0/2632963433 conn(0x55c78932b460 msgr2=0x55c7896dc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.706+0000 7f8a44332740 1 -- 172.21.15.29:0/2632963433 shutdown_connections 2024-09-21T01:50:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.707+0000 7f8a44332740 1 -- 172.21.15.29:0/2632963433 wait complete. 2024-09-21T01:50:15.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.708+0000 7f8a44332740 1 -- start start 2024-09-21T01:50:15.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.708+0000 7f8a44332740 1 Processor -- start 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c7894a5700 con 0x55c789316390 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c7895df110 con 0x55c789337070 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c7896ecaa0 con 0x55c78930e1b0 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.709+0000 7f8a44332740 10 client.0 _ll_register_callbacks cb 0x55c7895f8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51232/0 (socket says 172.21.15.29:51232) 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:33990/0 (socket says 172.21.15.29:33990) 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a377fe640 1 -- 172.21.15.29:0/2035663711 learned_addr learned my addr 172.21.15.29:0/2035663711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 -- 172.21.15.29:0/2035663711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 msgr2=0x55c7896e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 -- 172.21.15.29:0/2035663711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 msgr2=0x55c7896e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 -- 172.21.15.29:0/2035663711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78960cc40 con 0x55c789337070 2024-09-21T01:50:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.710+0000 7f8a37fff640 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a300ce3a0 tx=0x7f8a300ce0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20149e02d2d1986d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300b4070 con 0x55c789337070 2024-09-21T01:50:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a300ca910 con 0x55c789337070 2024-09-21T01:50:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2800ebb0 con 0x55c789337070 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a300d7ae0 con 0x55c789337070 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a44332740 10 client.25517 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.711+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c7896f8760 con 0x55c789337070 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2800ebb0 con 0x55c789337070 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8a300cd370 con 0x55c789337070 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 10 client.25517 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 20 client.25517 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892f59f0 0x55c7896fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 4 client.25517 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c789656da0 con 0x55c7892f59f0 2024-09-21T01:50:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.712+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892f59f0 0x55c7896fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.714+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892f59f0 0x55c7896fbd30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07e61a6b0843673 server_cookie=292db61f4b692f7f in_seq=0 out_seq=0 2024-09-21T01:50:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.714+0000 7f8a16ffd640 10 client.25517 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.715+0000 7f8a16ffd640 1 -- 172.21.15.29:0/2035663711 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c789656da0 con 0x55c7892f59f0 2024-09-21T01:50:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.715+0000 7f8a16ffd640 10 client.25517 handle_command_reply: tid=0 2024-09-21T01:50:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.715+0000 7f8a44332740 1 client.25517 shutdown 2024-09-21T01:50:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.715+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 msgr2=0x55c7896ea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.715+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a300ce3a0 tx=0x7f8a300ce0b0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 shutdown_connections 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892f59f0 0x55c7896fbd30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896ea320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 --2- 172.21.15.29:0/2035663711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a16ffd640 0 client.25517 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 >> 172.21.15.29:0/2035663711 conn(0x55c78932b460 msgr2=0x55c7896e1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 shutdown_connections 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 1 -- 172.21.15.29:0/2035663711 wait complete. 2024-09-21T01:50:15.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.716+0000 7f8a44332740 20 client.25517 trim_cache size 0 max 16384 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:50:15.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:15 smithi190 ceph-mon[34368]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 735 op/s 2024-09-21T01:50:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.744+0000 7f8a44332740 1 Processor -- start 2024-09-21T01:50:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.744+0000 7f8a44332740 1 -- start start 2024-09-21T01:50:15.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800e570 con 0x55c789316390 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180101a0 con 0x55c789337070 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a44332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800ec40 con 0x55c78930e1b0 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a377fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51242/0 (socket says 172.21.15.29:51242) 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a377fe640 1 -- 172.21.15.29:0/2070337533 learned_addr learned my addr 172.21.15.29:0/2070337533 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.745+0000 7f8a37fff640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 -- 172.21.15.29:0/2070337533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 msgr2=0x55c7896e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 -- 172.21.15.29:0/2070337533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 msgr2=0x55c7896e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 -- 172.21.15.29:0/2070337533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7896e8800 con 0x55c78930e1b0 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a37fff640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a377fe640 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a28013d70 tx=0x7f8a28008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90994cb4be932b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a2d7fa640 1 -- 172.21.15.29:0/2070337533 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28017040 con 0x55c78930e1b0 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a2d7fa640 1 -- 172.21.15.29:0/2070337533 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280044f0 con 0x55c78930e1b0 2024-09-21T01:50:15.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.746+0000 7f8a2d7fa640 1 -- 172.21.15.29:0/2070337533 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28003d00 con 0x55c78930e1b0 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.747+0000 7f8a44332740 1 -- 172.21.15.29:0/2070337533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 msgr2=0x55c7896eb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.747+0000 7f8a44332740 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a28013d70 tx=0x7f8a28008040 comp rx=0 tx=0).stop 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.748+0000 7f8a44332740 1 -- 172.21.15.29:0/2070337533 shutdown_connections 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.748+0000 7f8a44332740 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789337070 0x55c7896e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.748+0000 7f8a44332740 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789316390 0x55c7896e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.748+0000 7f8a44332740 1 --2- 172.21.15.29:0/2070337533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896eb4d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.748+0000 7f8a44332740 1 -- 172.21.15.29:0/2070337533 >> 172.21.15.29:0/2070337533 conn(0x55c78932b460 msgr2=0x55c7896eadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.749+0000 7f8a44332740 1 -- 172.21.15.29:0/2070337533 shutdown_connections 2024-09-21T01:50:15.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.749+0000 7f8a44332740 1 -- 172.21.15.29:0/2070337533 wait complete. 2024-09-21T01:50:15.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 -- start start 2024-09-21T01:50:15.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 Processor -- start 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789316390 0x55c7896edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 0x55c7896ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800fa10 con 0x55c789337070 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800f340 con 0x55c789316390 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800c510 con 0x55c78930e1b0 2024-09-21T01:50:15.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.750+0000 7f8a44332740 10 client.0 _ll_register_callbacks cb 0x7f8a30009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 0x55c7896ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 0x55c7896ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46752/0 (socket says 172.21.15.29:46752) 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a37fff640 1 -- 172.21.15.29:0/303446539 learned_addr learned my addr 172.21.15.29:0/303446539 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789316390 0x55c7896edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 -- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789316390 msgr2=0x55c7896edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789316390 0x55c7896edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 -- 172.21.15.29:0/303446539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 msgr2=0x55c7896ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 0x55c7896ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 -- 172.21.15.29:0/303446539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7896ee9f0 con 0x55c78930e1b0 2024-09-21T01:50:15.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.751+0000 7f8a377fe640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896f26c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a28008010 tx=0x7f8a28013820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1554425248c235 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:15.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.752+0000 7f8a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:15.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.752+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28017050 con 0x55c78930e1b0 2024-09-21T01:50:15.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.752+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a28013b70 con 0x55c78930e1b0 2024-09-21T01:50:15.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.752+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a280044f0 con 0x55c78930e1b0 2024-09-21T01:50:15.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a28005b80 con 0x55c78930e1b0 2024-09-21T01:50:15.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 10 client.25561 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:15.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c7897667b0 con 0x55c78930e1b0 2024-09-21T01:50:15.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280044f0 con 0x55c78930e1b0 2024-09-21T01:50:15.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8a28003a10 con 0x55c78930e1b0 2024-09-21T01:50:15.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 10 client.25561 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:15.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 20 client.25561 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892cada0 0x55c789768bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 4 client.25561 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.753+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c789652850 con 0x55c7892cada0 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892cada0 0x55c789768bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a2ffff640 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892cada0 0x55c789768bc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846a1b5592aafa95 server_cookie=ae1a40a8c2afa9fb in_seq=0 out_seq=0 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a16ffd640 10 client.25561 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a16ffd640 1 -- 172.21.15.29:0/303446539 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55c789652850 con 0x55c7892cada0 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a16ffd640 10 client.25561 handle_command_reply: tid=0 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.754+0000 7f8a44332740 1 client.25561 shutdown 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 msgr2=0x55c7896f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896f26c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a28008010 tx=0x7f8a28013820 comp rx=0 tx=0).stop 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 shutdown_connections 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c7892cada0 0x55c789768bc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c789337070 0x55c7896ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c789316390 0x55c7896edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 --2- 172.21.15.29:0/303446539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c78930e1b0 0x55c7896f26c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a16ffd640 0 client.25561 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 >> 172.21.15.29:0/303446539 conn(0x55c78932b460 msgr2=0x55c7896e1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 shutdown_connections 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.755+0000 7f8a44332740 1 -- 172.21.15.29:0/303446539 wait complete. 2024-09-21T01:50:15.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.756+0000 7f8a44332740 20 client.25561 trim_cache size 0 max 16384 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "91217b03-65df-4a37-8426-4ace79f8bff9", 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:50:15.758 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:15.759 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:50:15.760 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 msgr2=0x7f8a240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 0x7f8a240b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a3810fba0 tx=0x7f8a20008040 comp rx=0 tx=0).stop 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 msgr2=0x7f8a381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 0x7f8a381031b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002ae10 tx=0x7f8a30031a20 comp rx=0 tx=0).stop 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 shutdown_connections 2024-09-21T01:50:15.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a240b6590 0x7f8a240b8aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3810de50 0x7f8a381031b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a38105340 0x7f8a38102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 --2- 172.21.15.29:0/3073782252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a381049f0 0x7f8a38102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 >> 172.21.15.29:0/3073782252 conn(0x7f8a38079750 msgr2=0x7f8a38100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 shutdown_connections 2024-09-21T01:50:15.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:15.758+0000 7f8a3ddbe640 1 -- 172.21.15.29:0/3073782252 wait complete. 2024-09-21T01:50:16.767 INFO:tasks.cephfs.filesystem:scrub status for tag:91217b03-65df-4a37-8426-4ace79f8bff9 - {'path': '/', 'tag': '91217b03-65df-4a37-8426-4ace79f8bff9', 'options': 'recursive,force'} 2024-09-21T01:50:17.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:17 smithi066 ceph-mon[35001]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-09-21T01:50:17.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:17 smithi029 ceph-mon[31228]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-09-21T01:50:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:17 smithi190 ceph-mon[34368]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-09-21T01:50:17.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 100.0 s, 4127.0 tps, lat 24.211 ms stddev 27.353 2024-09-21T01:50:19.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:19 smithi190 ceph-mon[34368]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-21T01:50:19.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:19 smithi066 ceph-mon[35001]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-21T01:50:19.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:19 smithi029 ceph-mon[31228]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-21T01:50:20.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:20 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:50:20.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:20 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:50:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:20 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:50:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:21 smithi066 ceph-mon[35001]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-21T01:50:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:50:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:50:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:50:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:50:21.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:21 smithi029 ceph-mon[31228]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-21T01:50:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:50:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:50:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:50:21.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:50:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:21 smithi190 ceph-mon[34368]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-21T01:50:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:50:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:50:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:50:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:50:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.512+0000 7f169b939640 1 -- 172.21.15.29:0/2583780219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694077c20 msgr2=0x7f1694078090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.512+0000 7f169b939640 1 --2- 172.21.15.29:0/2583780219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694077c20 0x7f1694078090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802a8b0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 -- 172.21.15.29:0/2583780219 shutdown_connections 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 --2- 172.21.15.29:0/2583780219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694079640 0x7f1694079fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 --2- 172.21.15.29:0/2583780219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694078ca0 0x7f16940790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 --2- 172.21.15.29:0/2583780219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694077c20 0x7f1694078090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 -- 172.21.15.29:0/2583780219 >> 172.21.15.29:0/2583780219 conn(0x7f1694101220 msgr2=0x7f16941036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 -- 172.21.15.29:0/2583780219 shutdown_connections 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.513+0000 7f169b939640 1 -- 172.21.15.29:0/2583780219 wait complete. 2024-09-21T01:50:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.514+0000 7f169b939640 1 Processor -- start 2024-09-21T01:50:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 -- start start 2024-09-21T01:50:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694077c20 0x7f16941a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694078ca0 0x7f16941a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1694115980 con 0x7f1694079640 2024-09-21T01:50:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1694115800 con 0x7f1694078ca0 2024-09-21T01:50:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.515+0000 7f169b939640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1694115b00 con 0x7f1694077c20 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f16996d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694077c20 0x7f16941a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35936/0 (socket says 172.21.15.29:35936) 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 -- 172.21.15.29:0/697967796 learned_addr learned my addr 172.21.15.29:0/697967796 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1698ed4640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694078ca0 0x7f16941a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 -- 172.21.15.29:0/697967796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694077c20 msgr2=0x7f16941a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694077c20 0x7f16941a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 -- 172.21.15.29:0/697967796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694078ca0 msgr2=0x7f16941a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694078ca0 0x7f16941a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.516+0000 7f1699ed6640 1 -- 172.21.15.29:0/697967796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16941a1ee0 con 0x7f1694079640 2024-09-21T01:50:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.517+0000 7f1699ed6640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e970 tx=0x7f168c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.517+0000 7f16867fc640 1 -- 172.21.15.29:0/697967796 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ccc0 con 0x7f1694079640 2024-09-21T01:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.517+0000 7f16867fc640 1 -- 172.21.15.29:0/697967796 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c00ce60 con 0x7f1694079640 2024-09-21T01:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.517+0000 7f16867fc640 1 -- 172.21.15.29:0/697967796 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c010430 con 0x7f1694079640 2024-09-21T01:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.518+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16941a21d0 con 0x7f1694079640 2024-09-21T01:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.518+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16941a8770 con 0x7f1694079640 2024-09-21T01:50:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.523+0000 7f16a28a5740 1 Processor -- start 2024-09-21T01:50:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.524+0000 7f16867fc640 1 -- 172.21.15.29:0/697967796 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f168c004ff0 con 0x7f1694079640 2024-09-21T01:50:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.524+0000 7f16a28a5740 1 -- start start 2024-09-21T01:50:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 0x555b8c4eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b8c45c6f0 con 0x555b8c1421b0 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b8c44bb20 con 0x555b8c1192f0 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16a28a5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b8c44f930 con 0x555b8c1214d0 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16996d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16996d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:34444/0 (socket says 172.21.15.29:34444) 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f16996d5640 1 -- 172.21.15.29:0/2210939248 learned_addr learned my addr 172.21.15.29:0/2210939248 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f1698ed4640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 0x555b8c4eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.525+0000 7f1698ed4640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 -- 172.21.15.29:0/2210939248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 msgr2=0x555b8c4eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 0x555b8c4eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 -- 172.21.15.29:0/2210939248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 msgr2=0x555b8c4e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 -- 172.21.15.29:0/2210939248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8c411530 con 0x555b8c1192f0 2024-09-21T01:50:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f1698ed4640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 0x555b8c4eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f16996d5640 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1688035000 tx=0x7f1688030f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae24d70d30e028d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.526+0000 7f167f7fe640 1 -- 172.21.15.29:0/2210939248 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688030900 con 0x555b8c1192f0 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.527+0000 7f167f7fe640 1 -- 172.21.15.29:0/2210939248 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1688036040 con 0x555b8c1192f0 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.527+0000 7f167f7fe640 1 -- 172.21.15.29:0/2210939248 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880045b0 con 0x555b8c1192f0 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.527+0000 7f16867fc640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 0x7f16800b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.528+0000 7f16867fc640 1 -- 172.21.15.29:0/697967796 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f168c0e77d0 con 0x7f1694079640 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.528+0000 7f16a28a5740 1 -- 172.21.15.29:0/2210939248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 msgr2=0x555b8c4ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.528+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1688035000 tx=0x7f1688030f40 comp rx=0 tx=0).stop 2024-09-21T01:50:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.528+0000 7f1698ed4640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 0x7f16800b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.529+0000 7f16a28a5740 1 -- 172.21.15.29:0/2210939248 shutdown_connections 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.529+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1214d0 0x555b8c4eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.529+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1192f0 0x555b8c4ea4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.529+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2210939248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.529+0000 7f16a28a5740 1 -- 172.21.15.29:0/2210939248 >> 172.21.15.29:0/2210939248 conn(0x555b8c136300 msgr2=0x555b8c4e58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.530+0000 7f1698ed4640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 0x7f16800b8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16900058c0 tx=0x7f1690012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.530+0000 7f16a28a5740 1 -- 172.21.15.29:0/2210939248 shutdown_connections 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.530+0000 7f16a28a5740 1 -- 172.21.15.29:0/2210939248 wait complete. 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.530+0000 7f16a28a5740 1 -- start start 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.531+0000 7f16a28a5740 1 Processor -- start 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.531+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c504570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.531+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1421b0 0x555b8c5057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f16a28a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b8c383ab0 con 0x555b8c1214d0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f16a28a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b8c3f1730 con 0x555b8c1421b0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f16a28a5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b8c4ebad0 con 0x555b8c1192f0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f16a28a5740 10 client.0 _ll_register_callbacks cb 0x555b8c4456b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f1698ed4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f1698ed4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35942/0 (socket says 172.21.15.29:35942) 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.532+0000 7f1698ed4640 1 -- 172.21.15.29:0/3279263225 learned_addr learned my addr 172.21.15.29:0/3279263225 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f16996d5640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c504570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1699ed6640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1421b0 0x555b8c5057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 -- 172.21.15.29:0/3279263225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 msgr2=0x555b8c504570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c504570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 -- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1421b0 msgr2=0x555b8c5057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1421b0 0x555b8c5057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 -- 172.21.15.29:0/3279263225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8c1ad790 con 0x555b8c1214d0 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.533+0000 7f1698ed4640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f169000a5b0 tx=0x7f169000a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a522e2df9630a9af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.534+0000 7f167cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.534+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690023070 con 0x555b8c1214d0 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.534+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1690028660 con 0x555b8c1214d0 2024-09-21T01:50:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.534+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688036040 con 0x555b8c1214d0 2024-09-21T01:50:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.535+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1690028860 con 0x555b8c1214d0 2024-09-21T01:50:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.536+0000 7f16a28a5740 10 client.15825 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.536+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b8c226620 con 0x555b8c1214d0 2024-09-21T01:50:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.537+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1688036040 con 0x555b8c1214d0 2024-09-21T01:50:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.537+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f169001fb30 con 0x555b8c1214d0 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 10 client.15825 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 20 client.15825 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c100d80 0x555b8c4f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 4 client.15825 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.542+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b8c45e640 con 0x555b8c100d80 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.543+0000 7f16996d5640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c100d80 0x555b8c4f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.545+0000 7f16996d5640 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c100d80 0x555b8c4f8700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b72a2939877b73 server_cookie=2691bd5cba07bd19 in_seq=0 out_seq=0 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.545+0000 7f167cff9640 10 client.15825 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f167cff9640 1 -- 172.21.15.29:0/3279263225 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b8c45e640 con 0x555b8c100d80 2024-09-21T01:50:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f167cff9640 10 client.15825 handle_command_reply: tid=0 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 client.15825 shutdown 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 msgr2=0x555b8c504c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f169000a5b0 tx=0x7f169000a300 comp rx=0 tx=0).stop 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 shutdown_connections 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c100d80 0x555b8c4f8700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c1421b0 0x555b8c5057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1214d0 0x555b8c504c30 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f169000a5b0 tx=0x7f169000a300 comp rx=0 tx=0).stop 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 --2- 172.21.15.29:0/3279263225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c504570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 >> 172.21.15.29:0/3279263225 conn(0x555b8c136300 msgr2=0x555b8c4eaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.547+0000 7f167cff9640 0 client.15825 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.548+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 shutdown_connections 2024-09-21T01:50:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.548+0000 7f16a28a5740 1 -- 172.21.15.29:0/3279263225 wait complete. 2024-09-21T01:50:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.548+0000 7f16a28a5740 20 client.15825 trim_cache size 0 max 16384 2024-09-21T01:50:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.583+0000 7f16a28a5740 1 Processor -- start 2024-09-21T01:50:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.583+0000 7f16a28a5740 1 -- start start 2024-09-21T01:50:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c4f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 0x555b8c4f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16996d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c4f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16996d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c4f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56838/0 (socket says 172.21.15.29:56838) 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16941a8dc0 con 0x555b8c1421b0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16996d5640 1 -- 172.21.15.29:0/761001442 learned_addr learned my addr 172.21.15.29:0/761001442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16941aa9f0 con 0x555b8c0d61c0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16941a9490 con 0x555b8c1192f0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f1698ed4640 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.584+0000 7f1699ed6640 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 0x555b8c4f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.585+0000 7f1699ed6640 1 -- 172.21.15.29:0/761001442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 msgr2=0x555b8c4f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.585+0000 7f1699ed6640 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c4f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.585+0000 7f1699ed6640 1 -- 172.21.15.29:0/761001442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 msgr2=0x555b8c4e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.585+0000 7f1699ed6640 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.585+0000 7f1699ed6640 1 -- 172.21.15.29:0/761001442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8c4f1730 con 0x555b8c0d61c0 2024-09-21T01:50:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.586+0000 7f1699ed6640 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 0x555b8c4f1ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f168c0e5250 tx=0x7f168c0a98e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8c506e21bb4036 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.586+0000 7f167effd640 1 -- 172.21.15.29:0/761001442 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0046c0 con 0x555b8c0d61c0 2024-09-21T01:50:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.586+0000 7f167effd640 1 -- 172.21.15.29:0/761001442 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c0ad420 con 0x555b8c0d61c0 2024-09-21T01:50:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.586+0000 7f167effd640 1 -- 172.21.15.29:0/761001442 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0a9b80 con 0x555b8c0d61c0 2024-09-21T01:50:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 msgr2=0x555b8c4f1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 0x555b8c4f1ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f168c0e5250 tx=0x7f168c0a98e0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 shutdown_connections 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c0d61c0 0x555b8c4f1ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1421b0 0x555b8c4e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 --2- 172.21.15.29:0/761001442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c1192f0 0x555b8c4f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.588+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 >> 172.21.15.29:0/761001442 conn(0x555b8c136300 msgr2=0x555b8c4eac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.590+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 shutdown_connections 2024-09-21T01:50:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.590+0000 7f16a28a5740 1 -- 172.21.15.29:0/761001442 wait complete. 2024-09-21T01:50:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.590+0000 7f16a28a5740 1 -- start start 2024-09-21T01:50:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.591+0000 7f16a28a5740 1 Processor -- start 2024-09-21T01:50:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 0x555b8c50aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1192f0 0x555b8c50b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16996d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 0x555b8c50aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16996d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 0x555b8c50aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56848/0 (socket says 172.21.15.29:56848) 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16941aa260 con 0x555b8c1192f0 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16941a9b90 con 0x555b8c50b950 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1694073950 con 0x555b8c0d61c0 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f16a28a5740 10 client.0 _ll_register_callbacks cb 0x7f169000ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f1699ed6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f1699ed6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:34488/0 (socket says 172.21.15.29:34488) 2024-09-21T01:50:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.592+0000 7f1699ed6640 1 -- 172.21.15.29:0/2614006582 learned_addr learned my addr 172.21.15.29:0/2614006582 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1698ed4640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1192f0 0x555b8c50b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1699ed6640 1 -- 172.21.15.29:0/2614006582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 msgr2=0x555b8c50aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1699ed6640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 0x555b8c50aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1699ed6640 1 -- 172.21.15.29:0/2614006582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1192f0 msgr2=0x555b8c50b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1699ed6640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1192f0 0x555b8c50b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.593+0000 7f1699ed6640 1 -- 172.21.15.29:0/2614006582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8c509fd0 con 0x555b8c50b950 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f1699ed6640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d010 tx=0x7f168c009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f90d1c79b3818f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f167cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0046c0 con 0x555b8c50b950 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f168c0a9260 con 0x555b8c50b950 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168c0ad420 con 0x555b8c50b950 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f168c0b23f0 con 0x555b8c50b950 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f16a28a5740 10 client.25538 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.594+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b8c58d100 con 0x555b8c50b950 2024-09-21T01:50:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c0ad420 con 0x555b8c50b950 2024-09-21T01:50:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f168c0b6870 con 0x555b8c50b950 2024-09-21T01:50:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f16a28a5740 10 client.25538 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f16a28a5740 10 client.25538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f16a28a5740 20 client.25538 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.595+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c58da30 0x555b8c58fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.596+0000 7f16a28a5740 4 client.25538 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:50:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.596+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b8c45a0f0 con 0x555b8c58da30 2024-09-21T01:50:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.596+0000 7f16996d5640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c58da30 0x555b8c58fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.600+0000 7f16996d5640 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c58da30 0x555b8c58fe50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc16b21c7bb7e674 server_cookie=2dd1e7a9a83965cb in_seq=0 out_seq=0 2024-09-21T01:50:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.600+0000 7f167cff9640 10 client.25538 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f167cff9640 1 -- 172.21.15.29:0/2614006582 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b8c45a0f0 con 0x555b8c58da30 2024-09-21T01:50:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f167cff9640 10 client.25538 handle_command_reply: tid=0 2024-09-21T01:50:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 client.25538 shutdown 2024-09-21T01:50:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 msgr2=0x555b8c509a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d010 tx=0x7f168c009720 comp rx=0 tx=0).stop 2024-09-21T01:50:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 shutdown_connections 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b8c58da30 0x555b8c58fe50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8c50b950 0x555b8c509a90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b8c1192f0 0x555b8c50b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 --2- 172.21.15.29:0/2614006582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b8c0d61c0 0x555b8c50aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 >> 172.21.15.29:0/2614006582 conn(0x555b8c136300 msgr2=0x555b8c4eac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.613+0000 7f167cff9640 0 client.25538 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.615+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 shutdown_connections 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.615+0000 7f16a28a5740 1 -- 172.21.15.29:0/2614006582 wait complete. 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.615+0000 7f16a28a5740 20 client.25538 trim_cache size 0 max 16384 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:50:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 msgr2=0x7f16800b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 0x7f16800b8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16900058c0 tx=0x7f1690012040 comp rx=0 tx=0).stop 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 msgr2=0x7f16941a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e970 tx=0x7f168c00ed70 comp rx=0 tx=0).stop 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 shutdown_connections 2024-09-21T01:50:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16800b6650 0x7f16800b8b60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1694079640 0x7f16941a1750 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1694078ca0 0x7f16941a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 --2- 172.21.15.29:0/697967796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1694077c20 0x7f16941a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.616+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 >> 172.21.15.29:0/697967796 conn(0x7f1694101220 msgr2=0x7f169410fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.618+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 shutdown_connections 2024-09-21T01:50:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:22.618+0000 7f169b939640 1 -- 172.21.15.29:0/697967796 wait complete. 2024-09-21T01:50:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 105.0 s, 3911.2 tps, lat 25.565 ms stddev 27.211 2024-09-21T01:50:23.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:23 smithi066 ceph-mon[35001]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-21T01:50:23.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:23 smithi029 ceph-mon[31228]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-21T01:50:23.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:23 smithi190 ceph-mon[34368]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-21T01:50:25.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:25 smithi066 ceph-mon[35001]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-21T01:50:25.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:25 smithi029 ceph-mon[31228]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-21T01:50:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:25 smithi190 ceph-mon[34368]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-21T01:50:27.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:27 smithi066 ceph-mon[35001]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-21T01:50:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:27 smithi029 ceph-mon[31228]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-21T01:50:27.621 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:27 smithi190 ceph-mon[34368]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-21T01:50:27.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 110.0 s, 4064.9 tps, lat 24.585 ms stddev 28.205 2024-09-21T01:50:29.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:29 smithi066 ceph-mon[35001]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T01:50:29.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:29 smithi029 ceph-mon[31228]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T01:50:29.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:29 smithi190 ceph-mon[34368]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T01:50:30.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:30 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:50:30.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:30 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:50:30.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:30 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:50:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:31 smithi066 ceph-mon[35001]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 980 op/s 2024-09-21T01:50:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:50:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:50:31.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:50:31.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:31 smithi029 ceph-mon[31228]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 980 op/s 2024-09-21T01:50:31.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:50:31.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:50:31.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:50:31.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:31 smithi190 ceph-mon[34368]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 980 op/s 2024-09-21T01:50:31.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:50:31.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:50:31.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:50:32.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 115.0 s, 3869.6 tps, lat 25.860 ms stddev 32.092 2024-09-21T01:50:33.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:33 smithi029 ceph-mon[31228]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 795 op/s 2024-09-21T01:50:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:33 smithi190 ceph-mon[34368]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 795 op/s 2024-09-21T01:50:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:33 smithi066 ceph-mon[35001]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 795 op/s 2024-09-21T01:50:35.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:35 smithi029 ceph-mon[31228]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-09-21T01:50:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:35 smithi190 ceph-mon[34368]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-09-21T01:50:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.783+0000 7f5230a5c640 1 -- 172.21.15.29:0/4274744818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10d690 msgr2=0x7f522c10fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.783+0000 7f5230a5c640 1 --2- 172.21.15.29:0/4274744818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10d690 0x7f522c10fb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5220009900 tx=0x7f522002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.784+0000 7f5230a5c640 1 -- 172.21.15.29:0/4274744818 shutdown_connections 2024-09-21T01:50:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.784+0000 7f5230a5c640 1 --2- 172.21.15.29:0/4274744818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c110270 0x7f522c112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.784+0000 7f5230a5c640 1 --2- 172.21.15.29:0/4274744818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10d690 0x7f522c10fb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.784+0000 7f5230a5c640 1 --2- 172.21.15.29:0/4274744818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c10ac50 0x7f522c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.784+0000 7f5230a5c640 1 -- 172.21.15.29:0/4274744818 >> 172.21.15.29:0/4274744818 conn(0x7f522c070c70 msgr2=0x7f522c073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.785+0000 7f5230a5c640 1 -- 172.21.15.29:0/4274744818 shutdown_connections 2024-09-21T01:50:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.785+0000 7f5230a5c640 1 -- 172.21.15.29:0/4274744818 wait complete. 2024-09-21T01:50:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.787+0000 7f5230a5c640 1 Processor -- start 2024-09-21T01:50:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 -- start start 2024-09-21T01:50:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c10d690 0x7f522c1a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c110270 0x7f522c1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f522c115990 con 0x7f522c110270 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f522c115810 con 0x7f522c10ac50 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f5230a5c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f522c115b10 con 0x7f522c10d690 2024-09-21T01:50:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f522a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f522a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40848/0 (socket says 172.21.15.29:40848) 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.788+0000 7f522a59c640 1 -- 172.21.15.29:0/991270081 learned_addr learned my addr 172.21.15.29:0/991270081 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522ad9d640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c110270 0x7f522c1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f5229d9b640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c10d690 0x7f522c1a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 -- 172.21.15.29:0/991270081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c10d690 msgr2=0x7f522c1a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c10d690 0x7f522c1a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 -- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c110270 msgr2=0x7f522c1a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c110270 0x7f522c1a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 -- 172.21.15.29:0/991270081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c1a6400 con 0x7f522c10ac50 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.789+0000 7f522a59c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f521c013f40 tx=0x7f521c013f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.790+0000 7f521b7fe640 1 -- 172.21.15.29:0/991270081 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c014e10 con 0x7f522c10ac50 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.790+0000 7f521b7fe640 1 -- 172.21.15.29:0/991270081 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f521c011580 con 0x7f522c10ac50 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.790+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c1a6690 con 0x7f522c10ac50 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.790+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c19b0f0 con 0x7f522c10ac50 2024-09-21T01:50:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.790+0000 7f521b7fe640 1 -- 172.21.15.29:0/991270081 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c01c630 con 0x7f522c10ac50 2024-09-21T01:50:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.794+0000 7f52379cd740 1 Processor -- start 2024-09-21T01:50:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.794+0000 7f52379cd740 1 -- start start 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac7162f0 0x5567acae7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acae7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5567aca596f0 con 0x5567ac71e4d0 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5567aca48b20 con 0x5567ac7162f0 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f52379cd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5567aca4c930 con 0x5567ac73f1b0 2024-09-21T01:50:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52596/0 (socket says 172.21.15.29:52596) 2024-09-21T01:50:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 -- 172.21.15.29:0/821977348 learned_addr learned my addr 172.21.15.29:0/821977348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f5229d9b640 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acae7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 -- 172.21.15.29:0/821977348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac7162f0 msgr2=0x5567acae7480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac7162f0 0x5567acae7480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 -- 172.21.15.29:0/821977348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 msgr2=0x5567acae7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acae7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.795+0000 7f522ad9d640 1 -- 172.21.15.29:0/821977348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aca0e530 con 0x5567ac73f1b0 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.796+0000 7f521b7fe640 1 -- 172.21.15.29:0/991270081 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f521c01c8f0 con 0x7f522c10ac50 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.796+0000 7f522ad9d640 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f522400e480 tx=0x7f522400a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc26db4b4575d483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.796+0000 7f5218ff9640 1 -- 172.21.15.29:0/821977348 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224004d20 con 0x5567ac73f1b0 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.796+0000 7f5218ff9640 1 -- 172.21.15.29:0/821977348 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522400da70 con 0x5567ac73f1b0 2024-09-21T01:50:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.796+0000 7f5218ff9640 1 -- 172.21.15.29:0/821977348 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224005640 con 0x5567ac73f1b0 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.797+0000 7f521b7fe640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 0x7f52140b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.797+0000 7f521b7fe640 1 -- 172.21.15.29:0/991270081 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f521c0ee8a0 con 0x7f522c10ac50 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.797+0000 7f52379cd740 1 -- 172.21.15.29:0/821977348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 msgr2=0x5567acae6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.797+0000 7f52379cd740 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f522400e480 tx=0x7f522400a610 comp rx=0 tx=0).stop 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 -- 172.21.15.29:0/821977348 shutdown_connections 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acae7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac7162f0 0x5567acae7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 --2- 172.21.15.29:0/821977348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acae6e10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 -- 172.21.15.29:0/821977348 >> 172.21.15.29:0/821977348 conn(0x5567ac733300 msgr2=0x5567acae28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:35 smithi066 ceph-mon[35001]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f522a59c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 0x7f52140b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 -- 172.21.15.29:0/821977348 shutdown_connections 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.798+0000 7f52379cd740 1 -- 172.21.15.29:0/821977348 wait complete. 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.799+0000 7f522a59c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 0x7f52140b8c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f521c014fd0 tx=0x7f521c016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.799+0000 7f52379cd740 1 -- start start 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.799+0000 7f52379cd740 1 Processor -- start 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaf0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acaf1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5567ac980ab0 con 0x5567ac7162f0 2024-09-21T01:50:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47270/0 (socket says 172.21.15.29:47270) 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 -- 172.21.15.29:0/727840242 learned_addr learned my addr 172.21.15.29:0/727840242 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5567ac9ee730 con 0x5567ac71e4d0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5567acae8af0 con 0x5567ac73f1b0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f5229d9b640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaf0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f52379cd740 10 client.0 _ll_register_callbacks cb 0x5567aca426b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 -- 172.21.15.29:0/727840242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 msgr2=0x5567acaf1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acaf1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 -- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 msgr2=0x5567acaf0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaf0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f522ad9d640 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ac7aa790 con 0x5567ac7162f0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.800+0000 7f5229d9b640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaf0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.801+0000 7f522ad9d640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5224006010 tx=0x7f522400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542c9c7da8fad1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.801+0000 7f5211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.801+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224004c40 con 0x5567ac7162f0 2024-09-21T01:50:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.801+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5224004de0 con 0x5567ac7162f0 2024-09-21T01:50:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.801+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522400da70 con 0x5567ac7162f0 2024-09-21T01:50:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.802+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5224033050 con 0x5567ac7162f0 2024-09-21T01:50:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.802+0000 7f52379cd740 10 client.15840 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.802+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5567ac823620 con 0x5567ac7162f0 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522400da70 con 0x5567ac7162f0 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f522400b040 con 0x5567ac7162f0 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 10 client.15840 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 20 client.15840 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac6fdd80 0x5567acafe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 4 client.15840 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567aca5b640 con 0x5567ac6fdd80 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.803+0000 7f5229d9b640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac6fdd80 0x5567acafe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.804+0000 7f5211ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.804+0000 7f5229d9b640 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac6fdd80 0x5567acafe190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55258458cfbdbc5 server_cookie=c149b3f09666b41 in_seq=0 out_seq=0 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.805+0000 7f5211ffb640 1 -- 172.21.15.29:0/727840242 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567aca5b640 con 0x5567ac6fdd80 2024-09-21T01:50:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.805+0000 7f5211ffb640 10 client.15840 handle_command_reply: tid=0 2024-09-21T01:50:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.805+0000 7f52379cd740 1 client.15840 shutdown 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 msgr2=0x5567acaf05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5224006010 tx=0x7f522400df90 comp rx=0 tx=0).stop 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 shutdown_connections 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac6fdd80 0x5567acafe190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac73f1b0 0x5567acaf1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f5211ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaf0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 --2- 172.21.15.29:0/727840242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaf05d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 >> 172.21.15.29:0/727840242 conn(0x5567ac733300 msgr2=0x5567acae0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 shutdown_connections 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.806+0000 7f52379cd740 1 -- 172.21.15.29:0/727840242 wait complete. 2024-09-21T01:50:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.807+0000 7f52379cd740 20 client.15840 trim_cache size 0 max 16384 2024-09-21T01:50:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.839+0000 7f52379cd740 1 Processor -- start 2024-09-21T01:50:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.839+0000 7f52379cd740 1 -- start start 2024-09-21T01:50:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.839+0000 7f52379cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f52379cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f52379cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaeeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52612/0 (socket says 172.21.15.29:52612) 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f522ad9d640 1 -- 172.21.15.29:0/2114082643 learned_addr learned my addr 172.21.15.29:0/2114082643 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b15c0 con 0x5567ac7162f0 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b31f0 con 0x5567ac71e4d0 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.840+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b1c90 con 0x5567ac6fdd80 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaeeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 -- 172.21.15.29:0/2114082643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 msgr2=0x5567acaeeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaeeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 -- 172.21.15.29:0/2114082643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 msgr2=0x5567acaef7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaef7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 -- 172.21.15.29:0/2114082643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567acaf1b70 con 0x5567ac6fdd80 2024-09-21T01:50:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f522400ac30 tx=0x7f52240021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8a1ac969a39bff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.841+0000 7f5213fff640 1 -- 172.21.15.29:0/2114082643 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522400d9c0 con 0x5567ac6fdd80 2024-09-21T01:50:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.842+0000 7f5213fff640 1 -- 172.21.15.29:0/2114082643 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5224018b40 con 0x5567ac6fdd80 2024-09-21T01:50:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.842+0000 7f5213fff640 1 -- 172.21.15.29:0/2114082643 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240186e0 con 0x5567ac6fdd80 2024-09-21T01:50:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.843+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 msgr2=0x5567acafe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.843+0000 7f52379cd740 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f522400ac30 tx=0x7f52240021d0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 shutdown_connections 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac71e4d0 0x5567acaeeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac7162f0 0x5567acaef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 --2- 172.21.15.29:0/2114082643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac6fdd80 0x5567acafe0e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 >> 172.21.15.29:0/2114082643 conn(0x5567ac733300 msgr2=0x5567acadd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 shutdown_connections 2024-09-21T01:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.844+0000 7f52379cd740 1 -- 172.21.15.29:0/2114082643 wait complete. 2024-09-21T01:50:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.845+0000 7f52379cd740 1 -- start start 2024-09-21T01:50:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.845+0000 7f52379cd740 1 Processor -- start 2024-09-21T01:50:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac7162f0 0x5567acaf57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f522ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40878/0 (socket says 172.21.15.29:40878) 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f522ad9d640 1 -- 172.21.15.29:0/2361655978 learned_addr learned my addr 172.21.15.29:0/2361655978 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acaf62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b2a60 con 0x5567ac71e4d0 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b2390 con 0x5567ac6fdd80 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1af5a0 con 0x5567ac7162f0 2024-09-21T01:50:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f52379cd740 10 client.0 _ll_register_callbacks cb 0x7f5220009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.846+0000 7f5229d9b640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac7162f0 0x5567acaf57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acaf62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 -- 172.21.15.29:0/2361655978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac7162f0 msgr2=0x5567acaf57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac7162f0 0x5567acaf57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 -- 172.21.15.29:0/2361655978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 msgr2=0x5567acaf62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acaf62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.847+0000 7f522ad9d640 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567acaf6a00 con 0x5567ac6fdd80 2024-09-21T01:50:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f522ad9d640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5224018ac0 tx=0x7f522400df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff88be42d723640 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f5211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224047060 con 0x5567ac6fdd80 2024-09-21T01:50:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5224003b80 con 0x5567ac6fdd80 2024-09-21T01:50:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5224018b40 con 0x5567ac6fdd80 2024-09-21T01:50:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.848+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f522400a6a0 con 0x5567ac6fdd80 2024-09-21T01:50:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.849+0000 7f52379cd740 10 client.25553 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.849+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5567acaf7cf0 con 0x5567ac6fdd80 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.850+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5224003d80 con 0x5567ac6fdd80 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.850+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5224024570 con 0x5567ac6fdd80 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 10 client.25553 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 20 client.25553 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac73f1b0 0x5567acb6ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 4 client.25553 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:50:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5224018b40 con 0x5567ac73f1b0 2024-09-21T01:50:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.851+0000 7f5229d9b640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac73f1b0 0x5567acb6ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.852+0000 7f5229d9b640 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac73f1b0 0x5567acb6ab60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a575334ecfffd3 server_cookie=294b8d08190289f8 in_seq=0 out_seq=0 2024-09-21T01:50:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.852+0000 7f5211ffb640 10 client.25553 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.863+0000 7f5211ffb640 1 -- 172.21.15.29:0/2361655978 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5224018b40 con 0x5567ac73f1b0 2024-09-21T01:50:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.863+0000 7f5211ffb640 10 client.25553 handle_command_reply: tid=0 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.863+0000 7f52379cd740 1 client.25553 shutdown 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.863+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 msgr2=0x5567acaf50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.863+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5224018ac0 tx=0x7f522400df90 comp rx=0 tx=0).stop 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 shutdown_connections 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5567ac73f1b0 0x5567acb6ab60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5567ac71e4d0 0x5567acaf62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f5211ffb640 0 client.25553 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567ac7162f0 0x5567acaf57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 --2- 172.21.15.29:0/2361655978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567ac6fdd80 0x5567acaf50e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.864+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 >> 172.21.15.29:0/2361655978 conn(0x5567ac733300 msgr2=0x5567acadd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.866+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 shutdown_connections 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.866+0000 7f52379cd740 1 -- 172.21.15.29:0/2361655978 wait complete. 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.866+0000 7f52379cd740 20 client.25553 trim_cache size 0 max 16384 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:50:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:50:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:50:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 msgr2=0x7f52140b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 0x7f52140b8c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f521c014fd0 tx=0x7f521c016040 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 msgr2=0x7f522c1a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f521c013f40 tx=0x7f521c013f70 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 shutdown_connections 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52140b6720 0x7f52140b8c30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f522c110270 0x7f522c1a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c10d690 0x7f522c1a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 --2- 172.21.15.29:0/991270081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c10ac50 0x7f522c1a49c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 >> 172.21.15.29:0/991270081 conn(0x7f522c070c70 msgr2=0x7f522c111110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 shutdown_connections 2024-09-21T01:50:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:35.868+0000 7f5230a5c640 1 -- 172.21.15.29:0/991270081 wait complete. 2024-09-21T01:50:37.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:37 smithi029 ceph-mon[31228]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 856 op/s 2024-09-21T01:50:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:37 smithi190 ceph-mon[34368]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 856 op/s 2024-09-21T01:50:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:37 smithi066 ceph-mon[35001]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 856 op/s 2024-09-21T01:50:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 120.0 s, 4407.2 tps, lat 22.678 ms stddev 23.840 2024-09-21T01:50:39.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:39 smithi029 ceph-mon[31228]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-09-21T01:50:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:39 smithi190 ceph-mon[34368]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-09-21T01:50:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:39 smithi066 ceph-mon[35001]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-09-21T01:50:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:41 smithi029 ceph-mon[31228]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-21T01:50:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:41 smithi190 ceph-mon[34368]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-21T01:50:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:41 smithi066 ceph-mon[35001]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-21T01:50:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 125.0 s, 4900.8 tps, lat 20.440 ms stddev 23.727 2024-09-21T01:50:43.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:43 smithi029 ceph-mon[31228]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-09-21T01:50:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:43 smithi190 ceph-mon[34368]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-09-21T01:50:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:43 smithi066 ceph-mon[35001]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-09-21T01:50:45.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:45 smithi029 ceph-mon[31228]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-21T01:50:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:45 smithi190 ceph-mon[34368]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-21T01:50:45.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:45 smithi066 ceph-mon[35001]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-21T01:50:46.768 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:50:47.181 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:47.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:47 smithi029 ceph-mon[31228]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-21T01:50:47.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:47 smithi190 ceph-mon[34368]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-21T01:50:47.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:47 smithi066 ceph-mon[35001]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-21T01:50:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 130.0 s, 3298.0 tps, lat 30.103 ms stddev 35.279 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:50:48.790 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:50:48.791 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:50:48.792 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 -- 172.21.15.29:0/1920367793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 msgr2=0x7fa62c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 --2- 172.21.15.29:0/1920367793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c10d8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa620009860 tx=0x7fa62002a340 comp rx=0 tx=0).stop 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 -- 172.21.15.29:0/1920367793 shutdown_connections 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 --2- 172.21.15.29:0/1920367793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 0x7fa62c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 --2- 172.21.15.29:0/1920367793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c10d8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 --2- 172.21.15.29:0/1920367793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 -- 172.21.15.29:0/1920367793 >> 172.21.15.29:0/1920367793 conn(0x7fa62c079790 msgr2=0x7fa62c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 -- 172.21.15.29:0/1920367793 shutdown_connections 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.114+0000 7fa630942640 1 -- 172.21.15.29:0/1920367793 wait complete. 2024-09-21T01:50:48.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.115+0000 7fa630942640 1 Processor -- start 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.116+0000 7fa630942640 1 -- start start 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 0x7fa62c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c113aa0 con 0x7fa62c10de60 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c113920 con 0x7fa62c104a30 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa630942640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c113c20 con 0x7fa62c105380 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa62a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa62a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45652/0 (socket says 172.21.15.29:45652) 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 0x7fa62c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa62a59c640 1 -- 172.21.15.29:0/1350015686 learned_addr learned my addr 172.21.15.29:0/1350015686 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.117+0000 7fa629d9b640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 -- 172.21.15.29:0/1350015686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 msgr2=0x7fa62c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 -- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 msgr2=0x7fa62c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 0x7fa62c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 -- 172.21.15.29:0/1350015686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c1034d0 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.118+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9f0 tx=0x7fa61c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.119+0000 7fa61b7fe640 1 -- 172.21.15.29:0/1350015686 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00cd80 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.119+0000 7fa61b7fe640 1 -- 172.21.15.29:0/1350015686 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c0108e0 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.119+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c1aceb0 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.119+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1ad310 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.119+0000 7fa61b7fe640 1 -- 172.21.15.29:0/1350015686 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00cd80 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.124+0000 7fa61b7fe640 1 -- 172.21.15.29:0/1350015686 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa61c010430 con 0x7fa62c104a30 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.124+0000 7fa636eb6740 1 Processor -- start 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.125+0000 7fa636eb6740 1 -- start start 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa61b7fe640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 0x7fa6140b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 0x7fa6140b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d517070 0x558c0d8be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8beb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558c0d834e50 con 0x558c0d517070 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c0d824280 con 0x558c0d4ee1b0 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa636eb6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558c0d828090 con 0x558c0d4f6390 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.126+0000 7fa61b7fe640 1 -- 172.21.15.29:0/1350015686 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa61c0a9670 con 0x7fa62c104a30 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.127+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.127+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59824/0 (socket says 172.21.15.29:59824) 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.127+0000 7fa62ad9d640 1 -- 172.21.15.29:0/2944101801 learned_addr learned my addr 172.21.15.29:0/2944101801 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.127+0000 7fa629d9b640 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.127+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 0x7fa6140b8c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e830 tx=0x7fa624009c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.129+0000 7fa62ad9d640 1 -- 172.21.15.29:0/2944101801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 msgr2=0x558c0d8beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.129+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8beb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.129+0000 7fa62ad9d640 1 -- 172.21.15.29:0/2944101801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d517070 msgr2=0x558c0d8be650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.129+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d517070 0x558c0d8be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.129+0000 7fa62ad9d640 1 -- 172.21.15.29:0/2944101801 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0d582650 con 0x558c0d4f6390 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.130+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d7b0 tx=0x7fa62401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e080090a69a6734e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.132+0000 7fa618ff9640 1 -- 172.21.15.29:0/2944101801 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62401da80 con 0x558c0d4f6390 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.132+0000 7fa618ff9640 1 -- 172.21.15.29:0/2944101801 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa624022040 con 0x558c0d4f6390 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.132+0000 7fa618ff9640 1 -- 172.21.15.29:0/2944101801 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240042a0 con 0x558c0d4f6390 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.134+0000 7fa636eb6740 1 -- 172.21.15.29:0/2944101801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 msgr2=0x558c0d8bf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.134+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d7b0 tx=0x7fa62401d000 comp rx=0 tx=0).stop 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.135+0000 7fa636eb6740 1 -- 172.21.15.29:0/2944101801 shutdown_connections 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.135+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4f6390 0x558c0d8bf0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.135+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.135+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2944101801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d517070 0x558c0d8be650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.135+0000 7fa636eb6740 1 -- 172.21.15.29:0/2944101801 >> 172.21.15.29:0/2944101801 conn(0x558c0d50b460 msgr2=0x558c0d8bc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.136+0000 7fa636eb6740 1 -- 172.21.15.29:0/2944101801 shutdown_connections 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.136+0000 7fa636eb6740 1 -- 172.21.15.29:0/2944101801 wait complete. 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.137+0000 7fa636eb6740 1 -- start start 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.137+0000 7fa636eb6740 1 Processor -- start 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d8e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558c0d685700 con 0x558c0d4f6390 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c0d7bf110 con 0x558c0d517070 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558c0d8ccdd0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa636eb6740 10 client.0 _ll_register_callbacks cb 0x558c0d7d8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d8e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d8e70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45670/0 (socket says 172.21.15.29:45670) 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.138+0000 7fa62ad9d640 1 -- 172.21.15.29:0/876972104 learned_addr learned my addr 172.21.15.29:0/876972104 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.140+0000 7fa629d9b640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.140+0000 7fa62a59c640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 -- 172.21.15.29:0/876972104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 msgr2=0x558c0d8c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 -- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 msgr2=0x558c0d8e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d8e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 -- 172.21.15.29:0/876972104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0d7ecc40 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.141+0000 7fa629d9b640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8ca2e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa62002a860 tx=0x7fa620004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a29e228ded8b41e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.142+0000 7fa611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.142+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620037040 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.142+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa620030450 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.142+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa624022040 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.142+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa620005d30 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.143+0000 7fa636eb6740 10 client.15852 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.143+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c0d8e9b90 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa624022040 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa62003d480 con 0x558c0d4f6390 2024-09-21T01:50:48.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 20 client.15852 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4d59f0 0x558c0d8ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 4 client.15852 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.144+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c0d836da0 con 0x558c0d4d59f0 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.146+0000 7fa62a59c640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4d59f0 0x558c0d8ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.148+0000 7fa62a59c640 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4d59f0 0x558c0d8ed080 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f991c1413a0a3391 server_cookie=b2e83a998d204193 in_seq=0 out_seq=0 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.148+0000 7fa611ffb640 10 client.15852 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.149+0000 7fa611ffb640 1 -- 172.21.15.29:0/876972104 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c0d836da0 con 0x558c0d4d59f0 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.149+0000 7fa611ffb640 10 client.15852 handle_command_reply: tid=0 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.149+0000 7fa636eb6740 1 client.15852 shutdown 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 msgr2=0x558c0d8ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8ca2e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa62002a860 tx=0x7fa620004270 comp rx=0 tx=0).stop 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 shutdown_connections 2024-09-21T01:50:48.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4d59f0 0x558c0d8ed080 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d8e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8ca2e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 --2- 172.21.15.29:0/876972104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa611ffb640 0 client.15852 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 >> 172.21.15.29:0/876972104 conn(0x558c0d50b460 msgr2=0x558c0d8c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 shutdown_connections 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 1 -- 172.21.15.29:0/876972104 wait complete. 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.150+0000 7fa636eb6740 20 client.15852 trim_cache size 0 max 16384 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.176+0000 7fa636eb6740 1 Processor -- start 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.176+0000 7fa636eb6740 1 -- start start 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.176+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.176+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.176+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d517070 0x558c0d8c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa636eb6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1a6b10 con 0x558c0d4f6390 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa636eb6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1aef50 con 0x558c0d4ee1b0 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39824/0 (socket says 172.21.15.29:39824) 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 -- 172.21.15.29:0/2955167285 learned_addr learned my addr 172.21.15.29:0/2955167285 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa62a59c640 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ad9f0 con 0x558c0d517070 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d517070 0x558c0d8c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 -- 172.21.15.29:0/2955167285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d517070 msgr2=0x558c0d8c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d517070 0x558c0d8c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 -- 172.21.15.29:0/2955167285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 msgr2=0x558c0d8bc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8bc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.177+0000 7fa629d9b640 1 -- 172.21.15.29:0/2955167285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0d8cb060 con 0x558c0d4f6390 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.178+0000 7fa629d9b640 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa620009550 tx=0x7fa620009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311bd653f0b9a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.178+0000 7fa613fff640 1 -- 172.21.15.29:0/2955167285 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620055070 con 0x558c0d4f6390 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.178+0000 7fa613fff640 1 -- 172.21.15.29:0/2955167285 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620005d30 con 0x558c0d4f6390 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.178+0000 7fa613fff640 1 -- 172.21.15.29:0/2955167285 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620003920 con 0x558c0d4f6390 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 msgr2=0x558c0d8e7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa620009550 tx=0x7fa620009580 comp rx=0 tx=0).stop 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 shutdown_connections 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d517070 0x558c0d8c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8e7050 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 --2- 172.21.15.29:0/2955167285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d4ee1b0 0x558c0d8bc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.179+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 >> 172.21.15.29:0/2955167285 conn(0x558c0d50b460 msgr2=0x558c0d8bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.180+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 shutdown_connections 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.181+0000 7fa636eb6740 1 -- 172.21.15.29:0/2955167285 wait complete. 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.181+0000 7fa636eb6740 1 -- start start 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.182+0000 7fa636eb6740 1 Processor -- start 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.182+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.182+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8dff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa636eb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d950a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa636eb6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ae7c0 con 0x558c0d4f6390 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa636eb6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ae0f0 con 0x558c0d517070 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa636eb6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1a4af0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa636eb6740 10 client.0 _ll_register_callbacks cb 0x7fa624009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59854/0 (socket says 172.21.15.29:59854) 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 -- 172.21.15.29:0/1026076296 learned_addr learned my addr 172.21.15.29:0/1026076296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62ad9d640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d950a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 -- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 msgr2=0x558c0d950a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d950a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 -- 172.21.15.29:0/1026076296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 msgr2=0x558c0d8dff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.183+0000 7fa62a59c640 1 -- 172.21.15.29:0/1026076296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0d9510c0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.184+0000 7fa62a59c640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009250 tx=0x7fa61c018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8eb34871004934 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.184+0000 7fa611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.185+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0b6d70 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.185+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa61c0be6e0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.185+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa620005d30 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.186+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa61c0be8e0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.186+0000 7fa636eb6740 10 client.25606 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.186+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c0d9583f0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c0c3150 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa61c0a8de0 con 0x558c0d4ee1b0 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 10 client.25606 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 20 client.25606 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:48.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4aada0 0x558c0d95ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 4 client.25606 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa620005d30 con 0x558c0d4aada0 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.187+0000 7fa629d9b640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4aada0 0x558c0d95ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.188+0000 7fa611ffb640 10 client.25606 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.189+0000 7fa629d9b640 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4aada0 0x558c0d95ac70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107822e9bc97e68a server_cookie=c94ab6fceb3fbb19 in_seq=0 out_seq=0 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.190+0000 7fa611ffb640 1 -- 172.21.15.29:0/1026076296 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x7fa620005d30 con 0x558c0d4aada0 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.190+0000 7fa611ffb640 10 client.25606 handle_command_reply: tid=0 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.190+0000 7fa636eb6740 1 client.25606 shutdown 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.190+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 msgr2=0x558c0d8df980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.190+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009250 tx=0x7fa61c018420 comp rx=0 tx=0).stop 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 shutdown_connections 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558c0d4aada0 0x558c0d95ac70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c0d517070 0x558c0d950a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558c0d4f6390 0x558c0d8dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 --2- 172.21.15.29:0/1026076296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558c0d4ee1b0 0x558c0d8df980 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 >> 172.21.15.29:0/1026076296 conn(0x558c0d50b460 msgr2=0x558c0d8c1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.191+0000 7fa611ffb640 0 client.25606 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.192+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 shutdown_connections 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.192+0000 7fa636eb6740 1 -- 172.21.15.29:0/1026076296 wait complete. 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.192+0000 7fa636eb6740 20 client.25606 trim_cache size 0 max 16384 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 msgr2=0x7fa6140b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 0x7fa6140b8c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e830 tx=0x7fa624009c60 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 msgr2=0x7fa62c104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9f0 tx=0x7fa61c00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 shutdown_connections 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa6140b6710 0x7fa6140b8c20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62c10de60 0x7fa62c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa62c105380 0x7fa62c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 --2- 172.21.15.29:0/1350015686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa62c104a30 0x7fa62c104180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:48.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 >> 172.21.15.29:0/1350015686 conn(0x7fa62c079790 msgr2=0x7fa62c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:48.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.193+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 shutdown_connections 2024-09-21T01:50:48.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:48.194+0000 7fa630942640 1 -- 172.21.15.29:0/1350015686 wait complete. 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 -- 172.21.15.29:0/646417967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 msgr2=0x7f49b81050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 --2- 172.21.15.29:0/646417967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b81050a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009900 tx=0x7f49a802a410 comp rx=0 tx=0).stop 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 -- 172.21.15.29:0/646417967 shutdown_connections 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 --2- 172.21.15.29:0/646417967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b8108300 0x7f49b810a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 --2- 172.21.15.29:0/646417967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b81056e0 0x7f49b8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 --2- 172.21.15.29:0/646417967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b81050a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 -- 172.21.15.29:0/646417967 >> 172.21.15.29:0/646417967 conn(0x7f49b80fd690 msgr2=0x7f49b80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 -- 172.21.15.29:0/646417967 shutdown_connections 2024-09-21T01:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.024+0000 7f49bcb89640 1 -- 172.21.15.29:0/646417967 wait complete. 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 Processor -- start 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 -- start start 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b8108300 0x7f49b819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810e1f0 con 0x7f49b8102c80 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810e070 con 0x7f49b8108300 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49bcb89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810e370 con 0x7f49b81056e0 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39834/0 (socket says 172.21.15.29:39834) 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b659c640 1 -- 172.21.15.29:0/2314072818 learned_addr learned my addr 172.21.15.29:0/2314072818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59872/0 (socket says 172.21.15.29:59872) 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.025+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b8108300 0x7f49b819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 -- 172.21.15.29:0/2314072818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b8108300 msgr2=0x7f49b819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b8108300 0x7f49b819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 -- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 msgr2=0x7f49b8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 -- 172.21.15.29:0/2314072818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b819b9b0 con 0x7f49b81056e0 2024-09-21T01:50:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b659c640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49b5d9b640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49a000e9f0 tx=0x7f49a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49af7fe640 1 -- 172.21.15.29:0/2314072818 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a000ccc0 con 0x7f49b81056e0 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b819bca0 con 0x7f49b81056e0 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49af7fe640 1 -- 172.21.15.29:0/2314072818 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a00035f0 con 0x7f49b81056e0 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49af7fe640 1 -- 172.21.15.29:0/2314072818 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a000ce60 con 0x7f49b81056e0 2024-09-21T01:50:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.026+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a2390 con 0x7f49b81056e0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.029+0000 7f49c3bd0740 1 Processor -- start 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 -- start start 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32d61b0 0x562fd3679440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd367a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562fd35f06f0 con 0x562fd32d61b0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562fd35dfb20 con 0x562fd32b54d0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.030+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562fd35e3930 con 0x562fd32ad2f0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32d61b0 0x562fd3679440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59878/0 (socket says 172.21.15.29:59878) 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 -- 172.21.15.29:0/1277728002 learned_addr learned my addr 172.21.15.29:0/1277728002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 -- 172.21.15.29:0/1277728002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 msgr2=0x562fd367a600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd367a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 -- 172.21.15.29:0/1277728002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32d61b0 msgr2=0x562fd3679440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32d61b0 0x562fd3679440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 -- 172.21.15.29:0/1277728002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd35a5530 con 0x562fd32ad2f0 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.031+0000 7f49b659c640 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009a20 tx=0x7f49a8005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f563db73087bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.032+0000 7f49acff9640 1 -- 172.21.15.29:0/1277728002 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8035070 con 0x562fd32ad2f0 2024-09-21T01:50:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.032+0000 7f49acff9640 1 -- 172.21.15.29:0/1277728002 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a80035f0 con 0x562fd32ad2f0 2024-09-21T01:50:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.032+0000 7f49acff9640 1 -- 172.21.15.29:0/1277728002 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a803c680 con 0x562fd32ad2f0 2024-09-21T01:50:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.032+0000 7f49af7fe640 1 -- 172.21.15.29:0/2314072818 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f49a0010690 con 0x7f49b81056e0 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 -- 172.21.15.29:0/1277728002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 msgr2=0x562fd3679ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009a20 tx=0x7f49a8005eb0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49af7fe640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 0x7f49a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 -- 172.21.15.29:0/1277728002 shutdown_connections 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd367a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49af7fe640 1 -- 172.21.15.29:0/2314072818 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f49a00e75b0 con 0x7f49b81056e0 2024-09-21T01:50:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 0x7f49a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32ad2f0 0x562fd3679ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/1277728002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32d61b0 0x562fd3679440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.034+0000 7f49c3bd0740 1 -- 172.21.15.29:0/1277728002 >> 172.21.15.29:0/1277728002 conn(0x562fd32ca300 msgr2=0x562fd367dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.036+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 0x7f49a40b8aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005fd0 tx=0x7f49b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.036+0000 7f49c3bd0740 1 -- 172.21.15.29:0/1277728002 shutdown_connections 2024-09-21T01:50:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.036+0000 7f49c3bd0740 1 -- 172.21.15.29:0/1277728002 wait complete. 2024-09-21T01:50:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.037+0000 7f49c3bd0740 1 -- start start 2024-09-21T01:50:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.037+0000 7f49c3bd0740 1 Processor -- start 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.038+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32ad2f0 0x562fd3687620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd3687ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562fd3517ab0 con 0x562fd32ad2f0 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562fd3585730 con 0x562fd32b54d0 2024-09-21T01:50:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562fd367b0c0 con 0x562fd32d61b0 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49c3bd0740 10 client.0 _ll_register_callbacks cb 0x562fd35d96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49b6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49b659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32ad2f0 0x562fd3687620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49b6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59886/0 (socket says 172.21.15.29:59886) 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.039+0000 7f49b6d9d640 1 -- 172.21.15.29:0/777410758 learned_addr learned my addr 172.21.15.29:0/777410758 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.040+0000 7f49b6d9d640 1 -- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 msgr2=0x562fd3687ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.040+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd3687ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.040+0000 7f49b6d9d640 1 -- 172.21.15.29:0/777410758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32ad2f0 msgr2=0x562fd3687620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.040+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32ad2f0 0x562fd3687620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.040+0000 7f49b6d9d640 1 -- 172.21.15.29:0/777410758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd3341790 con 0x562fd32d61b0 2024-09-21T01:50:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.041+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49b0004030 tx=0x7f49b0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf1d1db124cc9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.041+0000 7f499dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.041+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b000f830 con 0x562fd32d61b0 2024-09-21T01:50:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.041+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f49b000f9d0 con 0x562fd32d61b0 2024-09-21T01:50:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.041+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a80035f0 con 0x562fd32d61b0 2024-09-21T01:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.042+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f49b000fbd0 con 0x562fd32d61b0 2024-09-21T01:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.042+0000 7f49c3bd0740 10 client.25618 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.042+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fd33ba620 con 0x562fd32d61b0 2024-09-21T01:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b000fd70 con 0x562fd32d61b0 2024-09-21T01:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f49b0027070 con 0x562fd32d61b0 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 10 client.25618 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 20 client.25618 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3294d80 0x562fd36a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 4 client.25618 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f49a80035f0 con 0x562fd3294d80 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.043+0000 7f49b659c640 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3294d80 0x562fd36a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.044+0000 7f49b659c640 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3294d80 0x562fd36a5f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848a812730229dc5 server_cookie=bdd15a927fdff5be in_seq=0 out_seq=0 2024-09-21T01:50:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.044+0000 7f499dffb640 10 client.25618 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f499dffb640 1 -- 172.21.15.29:0/777410758 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f49a80035f0 con 0x562fd3294d80 2024-09-21T01:50:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f499dffb640 10 client.25618 handle_command_reply: tid=0 2024-09-21T01:50:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f49c3bd0740 1 client.25618 shutdown 2024-09-21T01:50:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 msgr2=0x562fd368f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49b0004030 tx=0x7f49b0004ab0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.045+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 shutdown_connections 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3294d80 0x562fd36a5f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f830 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32b54d0 0x562fd3687ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/777410758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32ad2f0 0x562fd3687620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f499dffb640 0 client.25618 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.046+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 >> 172.21.15.29:0/777410758 conn(0x562fd32ca300 msgr2=0x562fd3677ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.047+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 shutdown_connections 2024-09-21T01:50:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.047+0000 7f49c3bd0740 1 -- 172.21.15.29:0/777410758 wait complete. 2024-09-21T01:50:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.047+0000 7f49c3bd0740 20 client.25618 trim_cache size 0 max 16384 2024-09-21T01:50:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.072+0000 7f49c3bd0740 1 Processor -- start 2024-09-21T01:50:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.072+0000 7f49c3bd0740 1 -- start start 2024-09-21T01:50:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.072+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd368f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd3685bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81a8d40 con 0x562fd32b54d0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aa970 con 0x562fd32ad2f0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81a9410 con 0x562fd32d61b0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45718/0 (socket says 172.21.15.29:45718) 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 -- 172.21.15.29:0/20742633 learned_addr learned my addr 172.21.15.29:0/20742633 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b5d9b640 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd368f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd3685bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 -- 172.21.15.29:0/20742633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 msgr2=0x562fd3685bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd3685bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 -- 172.21.15.29:0/20742633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 msgr2=0x562fd368f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd368f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.073+0000 7f49b659c640 1 -- 172.21.15.29:0/20742633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd3685850 con 0x562fd32ad2f0 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.074+0000 7f49b659c640 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a80314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88377f7a46489e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.074+0000 7f499ffff640 1 -- 172.21.15.29:0/20742633 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a805c070 con 0x562fd32ad2f0 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.074+0000 7f499ffff640 1 -- 172.21.15.29:0/20742633 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a8004240 con 0x562fd32ad2f0 2024-09-21T01:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.074+0000 7f499ffff640 1 -- 172.21.15.29:0/20742633 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8035040 con 0x562fd32ad2f0 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 -- 172.21.15.29:0/20742633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 msgr2=0x562fd367f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a80314a0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 -- 172.21.15.29:0/20742633 shutdown_connections 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd3685bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd368f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/20742633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd32ad2f0 0x562fd367f140 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a80314a0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 -- 172.21.15.29:0/20742633 >> 172.21.15.29:0/20742633 conn(0x562fd32ca300 msgr2=0x562fd3682810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 -- 172.21.15.29:0/20742633 shutdown_connections 2024-09-21T01:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.076+0000 7f49c3bd0740 1 -- 172.21.15.29:0/20742633 wait complete. 2024-09-21T01:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.077+0000 7f49c3bd0740 1 -- start start 2024-09-21T01:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.077+0000 7f49c3bd0740 1 Processor -- start 2024-09-21T01:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd36abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aa1e0 con 0x562fd32b54d0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81a9b10 con 0x562fd326a1c0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81a6d20 con 0x562fd32d61b0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49b6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49b6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45720/0 (socket says 172.21.15.29:45720) 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49b6d9d640 1 -- 172.21.15.29:0/2915222793 learned_addr learned my addr 172.21.15.29:0/2915222793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49c3bd0740 10 client.0 _ll_register_callbacks cb 0x7f49b0003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.078+0000 7f49b659c640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd36abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 -- 172.21.15.29:0/2915222793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 msgr2=0x562fd368f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 -- 172.21.15.29:0/2915222793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 msgr2=0x562fd36abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd36abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 -- 172.21.15.29:0/2915222793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd370f1b0 con 0x562fd326a1c0 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f49b6d9d640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005ee0 tx=0x7f49b000fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44fa10a866d015e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.079+0000 7f499dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.080+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b001ebc0 con 0x562fd326a1c0 2024-09-21T01:50:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.080+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f49b001ed60 con 0x562fd326a1c0 2024-09-21T01:50:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.080+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a8004240 con 0x562fd326a1c0 2024-09-21T01:50:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.080+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f49b0041050 con 0x562fd326a1c0 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.080+0000 7f49c3bd0740 10 client.25580 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fd3710460 con 0x562fd326a1c0 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a8004240 con 0x562fd326a1c0 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f49b0027070 con 0x562fd326a1c0 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 10 client.25580 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 20 client.25580 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3710e70 0x562fd37132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 4 client.25580 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fd35f2640 con 0x562fd3710e70 2024-09-21T01:50:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.081+0000 7f49b659c640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3710e70 0x562fd37132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.083+0000 7f49b659c640 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3710e70 0x562fd37132e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a56fa491f38526b server_cookie=9055cd22a0c85311 in_seq=0 out_seq=0 2024-09-21T01:50:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.083+0000 7f499dffb640 10 client.25580 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f499dffb640 1 -- 172.21.15.29:0/2915222793 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fd35f2640 con 0x562fd3710e70 2024-09-21T01:50:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f499dffb640 10 client.25580 handle_command_reply: tid=0 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 client.25580 shutdown 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 msgr2=0x562fd370ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005ee0 tx=0x7f49b000fdf0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 shutdown_connections 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562fd3710e70 0x562fd37132e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562fd326a1c0 0x562fd370ea00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562fd32d61b0 0x562fd368f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 --2- 172.21.15.29:0/2915222793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562fd32b54d0 0x562fd36abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 >> 172.21.15.29:0/2915222793 conn(0x562fd32ca300 msgr2=0x562fd367e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.095+0000 7f499dffb640 0 client.25580 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.096+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 shutdown_connections 2024-09-21T01:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.097+0000 7f49c3bd0740 1 -- 172.21.15.29:0/2915222793 wait complete. 2024-09-21T01:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49c3bd0740 20 client.25580 trim_cache size 0 max 16384 2024-09-21T01:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 msgr2=0x7f49a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 0x7f49a40b8aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005fd0 tx=0x7f49b0009500 comp rx=0 tx=0).stop 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 msgr2=0x7f49b819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49a000e9f0 tx=0x7f49a000edf0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 shutdown_connections 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f49a40b6590 0x7f49a40b8aa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49b8108300 0x7f49b819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49b81056e0 0x7f49b819a5e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 --2- 172.21.15.29:0/2314072818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49b8102c80 0x7f49b8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 >> 172.21.15.29:0/2314072818 conn(0x7f49b80fd690 msgr2=0x7f49b80ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 shutdown_connections 2024-09-21T01:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:50:49.098+0000 7f49bcb89640 1 -- 172.21.15.29:0/2314072818 wait complete. 2024-09-21T01:50:49.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:49 smithi029 ceph-mon[31228]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 793 op/s 2024-09-21T01:50:49.601 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:49 smithi066 ceph-mon[35001]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 793 op/s 2024-09-21T01:50:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:49 smithi190 ceph-mon[34368]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 793 op/s 2024-09-21T01:50:49.769 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:50:49.769 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:50:50.203 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:50.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 -- 172.21.15.29:0/4253262999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410e180 msgr2=0x7f188410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:50.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 --2- 172.21.15.29:0/4253262999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410e180 0x7f188410e590 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1874009900 tx=0x7f187402a410 comp rx=0 tx=0).stop 2024-09-21T01:50:50.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 -- 172.21.15.29:0/4253262999 shutdown_connections 2024-09-21T01:50:50.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 --2- 172.21.15.29:0/4253262999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410ead0 0x7f188410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:50.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 --2- 172.21.15.29:0/4253262999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410e180 0x7f188410e590 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 --2- 172.21.15.29:0/4253262999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410aee0 0x7f188410b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 -- 172.21.15.29:0/4253262999 >> 172.21.15.29:0/4253262999 conn(0x7f18840712f0 msgr2=0x7f1884073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 -- 172.21.15.29:0/4253262999 shutdown_connections 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.902+0000 7f188b314640 1 -- 172.21.15.29:0/4253262999 wait complete. 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.903+0000 7f188b314640 1 Processor -- start 2024-09-21T01:50:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.903+0000 7f188b314640 1 -- start start 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 0x7f18841a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410e180 0x7f18841a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 0x7f18841a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f188411a070 con 0x7f188410ead0 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1884119ef0 con 0x7f188410e180 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.904+0000 7f188b314640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f188411a1f0 con 0x7f188410aee0 2024-09-21T01:50:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18898b1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 0x7f18841a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:50.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18898b1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 0x7f18841a62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39912/0 (socket says 172.21.15.29:39912) 2024-09-21T01:50:50.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18898b1640 1 -- 172.21.15.29:0/870865733 learned_addr learned my addr 172.21.15.29:0/870865733 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:50.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 0x7f18841a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18888af640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410e180 0x7f18841a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 -- 172.21.15.29:0/870865733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410e180 msgr2=0x7f18841a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410e180 0x7f18841a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 -- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 msgr2=0x7f18841a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 0x7f18841a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.905+0000 7f18890b0640 1 -- 172.21.15.29:0/870865733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18841a69f0 con 0x7f188410aee0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f18890b0640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 0x7f18841a0a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f187c00da70 tx=0x7f187c00df30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00bbd0 con 0x7f188410aee0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f187c00bd70 con 0x7f188410aee0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c016680 con 0x7f188410aee0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f188b314640 1 -- 172.21.15.29:0/870865733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18841a6ce0 con 0x7f188410aee0 2024-09-21T01:50:50.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.906+0000 7f188b314640 1 -- 172.21.15.29:0/870865733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841ad440 con 0x7f188410aee0 2024-09-21T01:50:50.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.908+0000 7f188b314640 1 -- 172.21.15.29:0/870865733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18680052c0 con 0x7f188410aee0 2024-09-21T01:50:50.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.911+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f187c016820 con 0x7f188410aee0 2024-09-21T01:50:50.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.912+0000 7f187a7fc640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 0x7f18700b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:50.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.912+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f187c0e7d50 con 0x7f188410aee0 2024-09-21T01:50:50.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.913+0000 7f18888af640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 0x7f18700b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:50.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.914+0000 7f18888af640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 0x7f18700b8b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f1874031580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:50.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:50.914+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f187c0ae610 con 0x7f188410aee0 2024-09-21T01:50:51.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.331+0000 7f188b314640 1 -- 172.21.15.29:0/870865733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18680055b0 con 0x7f188410aee0 2024-09-21T01:50:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.334+0000 7f187a7fc640 1 -- 172.21.15.29:0/870865733 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11978 (secure 0 0 0) 0x7f187c0b29f0 con 0x7f188410aee0 2024-09-21T01:50:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:50:51.336 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":31,"btime":"2024-09-21T01:49:01:726666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:49:01.247055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:50:51.336 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 31 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.339+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 msgr2=0x7f18700b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.339+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 0x7f18700b8b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f1874031580 comp rx=0 tx=0).stop 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.339+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 msgr2=0x7f18841a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.339+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 0x7f18841a0a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f187c00da70 tx=0x7f187c00df30 comp rx=0 tx=0).stop 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 shutdown_connections 2024-09-21T01:50:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18700b6660 0x7f18700b8b70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f188410ead0 0x7f18841a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f188410e180 0x7f18841a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 --2- 172.21.15.29:0/870865733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f188410aee0 0x7f18841a0a80 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f187c00da70 tx=0x7f187c00df30 comp rx=0 tx=0).stop 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 >> 172.21.15.29:0/870865733 conn(0x7f18840712f0 msgr2=0x7f188410c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 shutdown_connections 2024-09-21T01:50:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:51.340+0000 7f186ffff640 1 -- 172.21.15.29:0/870865733 wait complete. 2024-09-21T01:50:51.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:51 smithi029 ceph-mon[31228]: pgmap v435: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2024-09-21T01:50:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:51 smithi190 ceph-mon[34368]: pgmap v435: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2024-09-21T01:50:51.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:51 smithi066 ceph-mon[35001]: pgmap v435: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2024-09-21T01:50:52.102 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:50:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:52.632 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:52 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/870865733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:50:52.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:52 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/870865733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:50:52.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:52 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/870865733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:50:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 135.0 s, 3167.4 tps, lat 31.713 ms stddev 39.333 2024-09-21T01:50:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:53 smithi190 ceph-mon[34368]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 947 op/s 2024-09-21T01:50:53.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:53 smithi066 ceph-mon[35001]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 947 op/s 2024-09-21T01:50:53.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:53 smithi029 ceph-mon[31228]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 947 op/s 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 -- 172.21.15.29:0/2417800338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad810de10 msgr2=0x7f3ad8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 --2- 172.21.15.29:0/2417800338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad810de10 0x7f3ad8110300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009860 tx=0x7f3ac802a340 comp rx=0 tx=0).stop 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 -- 172.21.15.29:0/2417800338 shutdown_connections 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 --2- 172.21.15.29:0/2417800338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad810de10 0x7f3ad8110300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 --2- 172.21.15.29:0/2417800338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ad8105300 0x7f3ad810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 --2- 172.21.15.29:0/2417800338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad81049b0 0x7f3ad8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.705+0000 7f3adecbe640 1 -- 172.21.15.29:0/2417800338 >> 172.21.15.29:0/2417800338 conn(0x7f3ad8079730 msgr2=0x7f3ad8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.706+0000 7f3adecbe640 1 -- 172.21.15.29:0/2417800338 shutdown_connections 2024-09-21T01:50:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.706+0000 7f3adecbe640 1 -- 172.21.15.29:0/2417800338 wait complete. 2024-09-21T01:50:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 Processor -- start 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 -- start start 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ad81049b0 0x7f3ad819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 0x7f3ad819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 0x7f3ad81a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113a70 con 0x7f3ad81049b0 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81138f0 con 0x7f3ad810de10 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.707+0000 7f3adecbe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113bf0 con 0x7f3ad8105300 2024-09-21T01:50:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 0x7f3ad819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 0x7f3ad819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33150/0 (socket says 172.21.15.29:33150) 2024-09-21T01:50:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3ad7fff640 1 -- 172.21.15.29:0/1759381229 learned_addr learned my addr 172.21.15.29:0/1759381229 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 0x7f3ad81a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 -- 172.21.15.29:0/1759381229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 msgr2=0x7f3ad819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 0x7f3ad819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 -- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ad81049b0 msgr2=0x7f3ad819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ad81049b0 0x7f3ad819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.708+0000 7f3add25b640 1 -- 172.21.15.29:0/1759381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad81a2610 con 0x7f3ad810de10 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.709+0000 7f3add25b640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 0x7f3ad81a1ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802a930 tx=0x7f3ac8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.711+0000 7f3ad5ffb640 1 -- 172.21.15.29:0/1759381229 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8042070 con 0x7f3ad810de10 2024-09-21T01:50:54.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.711+0000 7f3ad5ffb640 1 -- 172.21.15.29:0/1759381229 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac8005520 con 0x7f3ad810de10 2024-09-21T01:50:54.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.711+0000 7f3ad5ffb640 1 -- 172.21.15.29:0/1759381229 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8037040 con 0x7f3ad810de10 2024-09-21T01:50:54.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.712+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad81a28a0 con 0x7f3ad810de10 2024-09-21T01:50:54.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.712+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81a2d00 con 0x7f3ad810de10 2024-09-21T01:50:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.720+0000 7f3ae5232740 1 Processor -- start 2024-09-21T01:50:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.720+0000 7f3ae5232740 1 -- start start 2024-09-21T01:50:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.720+0000 7f3ad5ffb640 1 -- 172.21.15.29:0/1759381229 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3ac8004d70 con 0x7f3ad810de10 2024-09-21T01:50:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36454/0 (socket says 172.21.15.29:36454) 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 0x55c08d803be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ad7fff640 1 -- 172.21.15.29:0/2343433633 learned_addr learned my addr 172.21.15.29:0/2343433633 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d804120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c08d779ea0 con 0x55c08d45c070 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c08d7692d0 con 0x55c08d4331b0 2024-09-21T01:50:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.721+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c08d76d0e0 con 0x55c08d43b390 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d804120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3adca5a640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 0x55c08d803be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 -- 172.21.15.29:0/2343433633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 msgr2=0x55c08d804120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d804120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 -- 172.21.15.29:0/2343433633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 msgr2=0x55c08d803be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 0x55c08d803be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad7fff640 1 -- 172.21.15.29:0/2343433633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08d4c7650 con 0x55c08d45c070 2024-09-21T01:50:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3adca5a640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 0x55c08d803be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:50:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.722+0000 7f3ad5ffb640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 0x7f3ac00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.723+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c400 tx=0x7f3acc017d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0281d5d52a59e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.723+0000 7f3ad5ffb640 1 -- 172.21.15.29:0/1759381229 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3ac8106890 con 0x7f3ad810de10 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.723+0000 7f3abeffd640 1 -- 172.21.15.29:0/2343433633 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc01b020 con 0x55c08d45c070 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.724+0000 7f3abeffd640 1 -- 172.21.15.29:0/2343433633 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc021040 con 0x55c08d45c070 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.724+0000 7f3abeffd640 1 -- 172.21.15.29:0/2343433633 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc017a00 con 0x55c08d45c070 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.724+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 0x7f3ac00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 msgr2=0x55c08d8036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c400 tx=0x7f3acc017d90 comp rx=0 tx=0).stop 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 shutdown_connections 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d804120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d4331b0 0x55c08d803be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.725+0000 7f3ae5232740 1 --2- 172.21.15.29:0/2343433633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d45c070 0x55c08d8036a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.726+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 >> 172.21.15.29:0/2343433633 conn(0x55c08d450460 msgr2=0x55c08d801590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.726+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 0x7f3ac00b8c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c08d802320 tx=0x7f3ad0009b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.726+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 shutdown_connections 2024-09-21T01:50:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.726+0000 7f3ae5232740 1 -- 172.21.15.29:0/2343433633 wait complete. 2024-09-21T01:50:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.727+0000 7f3ae5232740 1 -- start start 2024-09-21T01:50:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.727+0000 7f3ae5232740 1 Processor -- start 2024-09-21T01:50:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d80ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 0x55c08d81b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c08d5ca700 con 0x55c08d4331b0 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c08d704160 con 0x55c08d45c070 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c08d811e20 con 0x55c08d43b390 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ae5232740 10 client.0 _ll_register_callbacks cb 0x55c08d71d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3add25b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 0x55c08d81b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33172/0 (socket says 172.21.15.29:33172) 2024-09-21T01:50:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.728+0000 7f3ad7fff640 1 -- 172.21.15.29:0/1829084034 learned_addr learned my addr 172.21.15.29:0/1829084034 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 -- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 msgr2=0x55c08d81b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 0x55c08d81b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 -- 172.21.15.29:0/1829084034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 msgr2=0x55c08d80ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d80ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 -- 172.21.15.29:0/1829084034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08d731c90 con 0x55c08d43b390 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3add25b640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 0x55c08d81b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006010 tx=0x7f3acc017e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a709e220579db12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc035060 con 0x55c08d43b390 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3acc00ce40 con 0x55c08d43b390 2024-09-21T01:50:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.729+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc021040 con 0x55c08d43b390 2024-09-21T01:50:54.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.730+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3acc01b030 con 0x55c08d43b390 2024-09-21T01:50:54.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.730+0000 7f3ae5232740 10 client.25639 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:54.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.730+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c08d81db30 con 0x55c08d43b390 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc015d60 con 0x55c08d43b390 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f3acc0035f0 con 0x55c08d43b390 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 10 client.25639 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 10 client.25639 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 20 client.25639 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d41a9f0 0x55c08d820fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 4 client.25639 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3acc021040 con 0x55c08d41a9f0 2024-09-21T01:50:54.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.731+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d41a9f0 0x55c08d820fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.732+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d41a9f0 0x55c08d820fa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a2485049ea21a0b server_cookie=a377d5ba9304ec47 in_seq=0 out_seq=0 2024-09-21T01:50:54.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.732+0000 7f3ab7fff640 10 client.25639 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.733+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1829084034 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3acc021040 con 0x55c08d41a9f0 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.733+0000 7f3ab7fff640 10 client.25639 handle_command_reply: tid=0 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.733+0000 7f3ae5232740 1 client.25639 shutdown 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 msgr2=0x55c08d80f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006010 tx=0x7f3acc017e90 comp rx=0 tx=0).stop 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 shutdown_connections 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d41a9f0 0x55c08d820fa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ab7fff640 0 client.25639 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d45c070 0x55c08d81b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d43b390 0x55c08d80f340 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1829084034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d80ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 >> 172.21.15.29:0/1829084034 conn(0x55c08d450460 msgr2=0x55c08d806880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 shutdown_connections 2024-09-21T01:50:54.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.734+0000 7f3ae5232740 1 -- 172.21.15.29:0/1829084034 wait complete. 2024-09-21T01:50:54.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.735+0000 7f3ae5232740 20 client.25639 trim_cache size 0 max 16384 2024-09-21T01:50:54.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.757+0000 7f3ae5232740 1 Processor -- start 2024-09-21T01:50:54.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.757+0000 7f3ae5232740 1 -- start start 2024-09-21T01:50:54.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 0x55c08d801600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d81b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 0x55c08d8067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81ad0f0 con 0x55c08d4331b0 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81aed20 con 0x55c08d43b390 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ae5232740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81ad7c0 con 0x55c08d41a9f0 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3add25b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 0x55c08d8067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3adca5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 0x55c08d801600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3add25b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 0x55c08d8067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50914/0 (socket says 172.21.15.29:50914) 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3adca5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 0x55c08d801600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33176/0 (socket says 172.21.15.29:33176) 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3add25b640 1 -- 172.21.15.29:0/3419401226 learned_addr learned my addr 172.21.15.29:0/3419401226 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:54.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d81b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ad7fff640 1 -- 172.21.15.29:0/3419401226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 msgr2=0x55c08d801600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 0x55c08d801600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ad7fff640 1 -- 172.21.15.29:0/3419401226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 msgr2=0x55c08d8067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.758+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 0x55c08d8067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.759+0000 7f3ad7fff640 1 -- 172.21.15.29:0/3419401226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08d806500 con 0x55c08d4331b0 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.759+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d81b0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0039c0 tx=0x7f3acc009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4431d15534adcc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.759+0000 7f3abe7fc640 1 -- 172.21.15.29:0/3419401226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc040070 con 0x55c08d4331b0 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.760+0000 7f3abe7fc640 1 -- 172.21.15.29:0/3419401226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc00ce40 con 0x55c08d4331b0 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.760+0000 7f3abe7fc640 1 -- 172.21.15.29:0/3419401226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc035040 con 0x55c08d4331b0 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.761+0000 7f3ae5232740 1 -- 172.21.15.29:0/3419401226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 msgr2=0x55c08d81b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.761+0000 7f3ae5232740 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d81b0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0039c0 tx=0x7f3acc009e50 comp rx=0 tx=0).stop 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 -- 172.21.15.29:0/3419401226 shutdown_connections 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d43b390 0x55c08d8067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d4331b0 0x55c08d81b0b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 --2- 172.21.15.29:0/3419401226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d41a9f0 0x55c08d801600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 -- 172.21.15.29:0/3419401226 >> 172.21.15.29:0/3419401226 conn(0x55c08d450460 msgr2=0x55c08d800ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 -- 172.21.15.29:0/3419401226 shutdown_connections 2024-09-21T01:50:54.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.762+0000 7f3ae5232740 1 -- 172.21.15.29:0/3419401226 wait complete. 2024-09-21T01:50:54.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.763+0000 7f3ae5232740 1 -- start start 2024-09-21T01:50:54.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.763+0000 7f3ae5232740 1 Processor -- start 2024-09-21T01:50:54.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.764+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.764+0000 7f3ae5232740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d4331b0 0x55c08d821dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.764+0000 7f3adca5a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.764+0000 7f3adca5a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50930/0 (socket says 172.21.15.29:50930) 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.764+0000 7f3adca5a640 1 -- 172.21.15.29:0/1710651916 learned_addr learned my addr 172.21.15.29:0/1710651916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d43b390 0x55c08d822460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d4331b0 0x55c08d821dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81ae590 con 0x55c08d43b390 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81adec0 con 0x55c08d41a9f0 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81ab110 con 0x55c08d4331b0 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3add25b640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d43b390 0x55c08d822460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3ae5232740 10 client.0 _ll_register_callbacks cb 0x7f3acc009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3adca5a640 1 -- 172.21.15.29:0/1710651916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d4331b0 msgr2=0x55c08d821dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.765+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d4331b0 0x55c08d821dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3adca5a640 1 -- 172.21.15.29:0/1710651916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d43b390 msgr2=0x55c08d822460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d43b390 0x55c08d822460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3adca5a640 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08d822bd0 con 0x55c08d41a9f0 2024-09-21T01:50:54.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3adca5a640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06a46c8e0f519d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:54.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3ab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0030060 con 0x55c08d41a9f0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3ad001a6b0 con 0x55c08d41a9f0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.766+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc00ce40 con 0x55c08d41a9f0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.767+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3ad001a8b0 con 0x55c08d41a9f0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.767+0000 7f3ae5232740 10 client.25601 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.767+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c08d88b7c0 con 0x55c08d41a9f0 2024-09-21T01:50:54.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc00ce40 con 0x55c08d41a9f0 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f3ad0017040 con 0x55c08d41a9f0 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 10 client.25601 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 10 client.25601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 20 client.25601 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d45c070 0x55c08d88dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 4 client.25601 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c08d77bdf0 con 0x55c08d45c070 2024-09-21T01:50:54.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.768+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d45c070 0x55c08d88dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.769+0000 7f3ab7fff640 10 client.25601 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.770+0000 7f3ad7fff640 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d45c070 0x55c08d88dfb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9c37fc75ce9659e server_cookie=7336005defcdb20 in_seq=0 out_seq=0 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.770+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1710651916 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c08d77bdf0 con 0x55c08d45c070 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.770+0000 7f3ab7fff640 10 client.25601 handle_command_reply: tid=0 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.770+0000 7f3ae5232740 1 client.25601 shutdown 2024-09-21T01:50:54.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 msgr2=0x55c08d8248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad0004a40 comp rx=0 tx=0).stop 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 shutdown_connections 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c08d45c070 0x55c08d88dfb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c08d43b390 0x55c08d822460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08d4331b0 0x55c08d821dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 --2- 172.21.15.29:0/1710651916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c08d41a9f0 0x55c08d8248f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 >> 172.21.15.29:0/1710651916 conn(0x55c08d450460 msgr2=0x55c08d8067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ab7fff640 0 client.25601 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 shutdown_connections 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 1 -- 172.21.15.29:0/1710651916 wait complete. 2024-09-21T01:50:54.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.771+0000 7f3ae5232740 20 client.25601 trim_cache size 0 max 16384 2024-09-21T01:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:54.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.772+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 msgr2=0x7f3ac00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.772+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 0x7f3ac00b8c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c08d802320 tx=0x7f3ad0009b30 comp rx=0 tx=0).stop 2024-09-21T01:50:54.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.772+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 msgr2=0x7f3ad81a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.772+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 0x7f3ad81a1ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802a930 tx=0x7f3ac8005770 comp rx=0 tx=0).stop 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 shutdown_connections 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3ac00b6720 0x7f3ac00b8c30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad810de10 0x7f3ad81a1ee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad8105300 0x7f3ad819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 --2- 172.21.15.29:0/1759381229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ad81049b0 0x7f3ad819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 >> 172.21.15.29:0/1759381229 conn(0x7f3ad8079730 msgr2=0x7f3ad8109c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 shutdown_connections 2024-09-21T01:50:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:54.773+0000 7f3adecbe640 1 -- 172.21.15.29:0/1759381229 wait complete. 2024-09-21T01:50:55.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:55 smithi029 ceph-mon[31228]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 876 op/s 2024-09-21T01:50:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:55 smithi190 ceph-mon[34368]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 876 op/s 2024-09-21T01:50:55.768 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:50:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:55 smithi066 ceph-mon[35001]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 876 op/s 2024-09-21T01:50:56.202 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:56.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.817+0000 7f852501f640 1 -- 172.21.15.29:0/3522892798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f851809a1a0 msgr2=0x7f851809a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.817+0000 7f852501f640 1 --2- 172.21.15.29:0/3522892798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f851809a1a0 0x7f851809a680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8520066480 tx=0x7f8520098180 comp rx=0 tx=0).stop 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 -- 172.21.15.29:0/3522892798 shutdown_connections 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 --2- 172.21.15.29:0/3522892798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f851809a1a0 0x7f851809a680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 --2- 172.21.15.29:0/3522892798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85180956b0 0x7f8518099be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 --2- 172.21.15.29:0/3522892798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518094d60 0x7f8518095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 -- 172.21.15.29:0/3522892798 >> 172.21.15.29:0/3522892798 conn(0x7f851800cd70 msgr2=0x7f851800d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 -- 172.21.15.29:0/3522892798 shutdown_connections 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.818+0000 7f852501f640 1 -- 172.21.15.29:0/3522892798 wait complete. 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.819+0000 7f852501f640 1 Processor -- start 2024-09-21T01:50:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.819+0000 7f852501f640 1 -- start start 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 0x7f8518132f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851809a1a0 0x7f8518138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f85180ac580 con 0x7f85180956b0 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85180ac400 con 0x7f851809a1a0 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f852501f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85180ac700 con 0x7f8518094d60 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36482/0 (socket says 172.21.15.29:36482) 2024-09-21T01:50:56.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.820+0000 7f851f7fe640 1 -- 172.21.15.29:0/3011018727 learned_addr learned my addr 172.21.15.29:0/3011018727 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851ffff640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 0x7f8518132f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f852481e640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851809a1a0 0x7f8518138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851f7fe640 1 -- 172.21.15.29:0/3011018727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 msgr2=0x7f8518132f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851f7fe640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 0x7f8518132f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851f7fe640 1 -- 172.21.15.29:0/3011018727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851809a1a0 msgr2=0x7f8518138790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851f7fe640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851809a1a0 0x7f8518138790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851f7fe640 1 -- 172.21.15.29:0/3011018727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8518138ec0 con 0x7f85180956b0 2024-09-21T01:50:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.821+0000 7f851ffff640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 0x7f8518132f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.822+0000 7f851f7fe640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f851400d740 tx=0x7f851400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.822+0000 7f851d7fa640 1 -- 172.21.15.29:0/3011018727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514017070 con 0x7f85180956b0 2024-09-21T01:50:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.822+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181391b0 con 0x7f85180956b0 2024-09-21T01:50:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.822+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851813f910 con 0x7f85180956b0 2024-09-21T01:50:56.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.823+0000 7f851d7fa640 1 -- 172.21.15.29:0/3011018727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851400de50 con 0x7f85180956b0 2024-09-21T01:50:56.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.823+0000 7f851d7fa640 1 -- 172.21.15.29:0/3011018727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514021060 con 0x7f85180956b0 2024-09-21T01:50:56.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.826+0000 7f852c7f5740 1 Processor -- start 2024-09-21T01:50:56.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 -- start start 2024-09-21T01:50:56.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf0213070 0x55ccf05ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f851ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ccf0530ea0 con 0x55ccf01ea1b0 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f851ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36486/0 (socket says 172.21.15.29:36486) 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ccf05202d0 con 0x55ccf0213070 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f851ffff640 1 -- 172.21.15.29:0/4274338626 learned_addr learned my addr 172.21.15.29:0/4274338626 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.827+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ccf05240e0 con 0x55ccf01f2390 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f852481e640 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf0213070 0x55ccf05ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 -- 172.21.15.29:0/4274338626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 msgr2=0x55ccf05bb2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bb2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 -- 172.21.15.29:0/4274338626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf0213070 msgr2=0x55ccf05ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf0213070 0x55ccf05ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 -- 172.21.15.29:0/4274338626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf027e650 con 0x55ccf01ea1b0 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.828+0000 7f851ffff640 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f850c0104d0 tx=0x7f850c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0d6f42dbd4d42a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.829+0000 7f850a7fc640 1 -- 172.21.15.29:0/4274338626 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c004220 con 0x55ccf01ea1b0 2024-09-21T01:50:56.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.829+0000 7f850a7fc640 1 -- 172.21.15.29:0/4274338626 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c0043c0 con 0x55ccf01ea1b0 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.829+0000 7f851d7fa640 1 -- 172.21.15.29:0/3011018727 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f851401a070 con 0x7f85180956b0 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 msgr2=0x55ccf05bada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f850c0104d0 tx=0x7f850c010990 comp rx=0 tx=0).stop 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 shutdown_connections 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01ea1b0 0x55ccf05bada0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 --2- 172.21.15.29:0/4274338626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf0213070 0x55ccf05ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.830+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 >> 172.21.15.29:0/4274338626 conn(0x55ccf0207460 msgr2=0x55ccf05b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.831+0000 7f851d7fa640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 0x7f85100b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.831+0000 7f851d7fa640 1 -- 172.21.15.29:0/3011018727 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f85140e6ac0 con 0x7f85180956b0 2024-09-21T01:50:56.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.831+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 shutdown_connections 2024-09-21T01:50:56.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.831+0000 7f852c7f5740 1 -- 172.21.15.29:0/4274338626 wait complete. 2024-09-21T01:50:56.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.832+0000 7f851ffff640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 0x7f85100b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.832+0000 7f852c7f5740 1 -- start start 2024-09-21T01:50:56.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.832+0000 7f852c7f5740 1 Processor -- start 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f851ffff640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 0x7f85100b8aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f850c010e90 tx=0x7f850c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01f2390 0x55ccf05c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ccf0381700 con 0x55ccf01f2390 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ccf04bb160 con 0x55ccf01d19f0 2024-09-21T01:50:56.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852c7f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ccf05c9090 con 0x55ccf0213070 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.833+0000 7f852481e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f851ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01f2390 0x55ccf05c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33200/0 (socket says 172.21.15.29:33200) 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f851f7fe640 1 -- 172.21.15.29:0/720106672 learned_addr learned my addr 172.21.15.29:0/720106672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50968/0 (socket says 172.21.15.29:50968) 2024-09-21T01:50:56.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852c7f5740 10 client.0 _ll_register_callbacks cb 0x55ccf04d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:56.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 -- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 msgr2=0x55ccf05c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 -- 172.21.15.29:0/720106672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01f2390 msgr2=0x55ccf05c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01f2390 0x55ccf05c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.834+0000 7f852481e640 1 -- 172.21.15.29:0/720106672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf04e8c90 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.835+0000 7f852481e640 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8520052910 tx=0x7f8520069070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e84b1ca998127e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.835+0000 7f84ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.835+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852006c320 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.835+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f85200a06d0 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.835+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccf0532df0 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.836+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f85200a08d0 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.836+0000 7f852c7f5740 10 client.25613 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.836+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ccf05d4ad0 con 0x55ccf01d19f0 2024-09-21T01:50:56.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.836+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852009c040 con 0x55ccf01d19f0 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f852006abe0 con 0x55ccf01d19f0 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 10 client.25613 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 10 client.25613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 20 client.25613 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf01a6da0 0x55ccf05d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 4 client.25613 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf0532df0 con 0x55ccf01a6da0 2024-09-21T01:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.837+0000 7f851ffff640 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf01a6da0 0x55ccf05d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.840+0000 7f851ffff640 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf01a6da0 0x55ccf05d8590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43a01dfc7ba22c74 server_cookie=dcb4be62c9af247f in_seq=0 out_seq=0 2024-09-21T01:50:56.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.840+0000 7f84ff7fe640 10 client.25613 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:56.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.841+0000 7f84ff7fe640 1 -- 172.21.15.29:0/720106672 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccf0532df0 con 0x55ccf01a6da0 2024-09-21T01:50:56.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.841+0000 7f84ff7fe640 10 client.25613 handle_command_reply: tid=0 2024-09-21T01:50:56.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.841+0000 7f852c7f5740 1 client.25613 shutdown 2024-09-21T01:50:56.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 msgr2=0x55ccf05c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8520052910 tx=0x7f8520069070 comp rx=0 tx=0).stop 2024-09-21T01:50:56.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 shutdown_connections 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05c6250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf01a6da0 0x55ccf05d8590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 --2- 172.21.15.29:0/720106672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01f2390 0x55ccf05c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 >> 172.21.15.29:0/720106672 conn(0x55ccf0207460 msgr2=0x55ccf05bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f84ff7fe640 0 client.25613 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:56.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 shutdown_connections 2024-09-21T01:50:56.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.842+0000 7f852c7f5740 1 -- 172.21.15.29:0/720106672 wait complete. 2024-09-21T01:50:56.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.843+0000 7f852c7f5740 20 client.25613 trim_cache size 0 max 16384 2024-09-21T01:50:56.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 Processor -- start 2024-09-21T01:50:56.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 -- start start 2024-09-21T01:50:56.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f850400e460 con 0x55ccf0213070 2024-09-21T01:50:56.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8504010090 con 0x55ccf01d19f0 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.867+0000 7f852c7f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f850400eb30 con 0x55ccf01f2390 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f852481e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f852481e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36506/0 (socket says 172.21.15.29:36506) 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f852481e640 1 -- 172.21.15.29:0/2910484357 learned_addr learned my addr 172.21.15.29:0/2910484357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851f7fe640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 -- 172.21.15.29:0/2910484357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 msgr2=0x55ccf05bdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 -- 172.21.15.29:0/2910484357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 msgr2=0x55ccf05c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 -- 172.21.15.29:0/2910484357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf05c89e0 con 0x55ccf01d19f0 2024-09-21T01:50:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f852481e640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:50:56.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.868+0000 7f851ffff640 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05bd960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ccf05d8d30 tx=0x7f850c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb7ccae00de718d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.870+0000 7f8509ffb640 1 -- 172.21.15.29:0/2910484357 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c037070 con 0x55ccf01d19f0 2024-09-21T01:50:56.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.870+0000 7f8509ffb640 1 -- 172.21.15.29:0/2910484357 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c00bbe0 con 0x55ccf01d19f0 2024-09-21T01:50:56.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.870+0000 7f8509ffb640 1 -- 172.21.15.29:0/2910484357 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c01c6a0 con 0x55ccf01d19f0 2024-09-21T01:50:56.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 -- 172.21.15.29:0/2910484357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 msgr2=0x55ccf05bd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05bd960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ccf05d8d30 tx=0x7f850c01f000 comp rx=0 tx=0).stop 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 -- 172.21.15.29:0/2910484357 shutdown_connections 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf0213070 0x55ccf05c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf01f2390 0x55ccf05bdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 --2- 172.21.15.29:0/2910484357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01d19f0 0x55ccf05bd960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.871+0000 7f852c7f5740 1 -- 172.21.15.29:0/2910484357 >> 172.21.15.29:0/2910484357 conn(0x55ccf0207460 msgr2=0x55ccf05c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.872+0000 7f852c7f5740 1 -- 172.21.15.29:0/2910484357 shutdown_connections 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.872+0000 7f852c7f5740 1 -- 172.21.15.29:0/2910484357 wait complete. 2024-09-21T01:50:56.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.872+0000 7f852c7f5740 1 -- start start 2024-09-21T01:50:56.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.873+0000 7f852c7f5740 1 Processor -- start 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.873+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 0x55ccf05ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.873+0000 7f852c7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01f2390 0x55ccf05ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f851ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 0x55ccf05ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f851ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 0x55ccf05ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36512/0 (socket says 172.21.15.29:36512) 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01f2390 0x55ccf05ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f851ffff640 1 -- 172.21.15.29:0/3824576422 learned_addr learned my addr 172.21.15.29:0/3824576422 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f850400f900 con 0x55ccf01d19f0 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f850400f230 con 0x55ccf01f2390 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f850400c440 con 0x55ccf0213070 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852481e640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.874+0000 7f852c7f5740 10 client.0 _ll_register_callbacks cb 0x7f8520064810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:56.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 -- 172.21.15.29:0/3824576422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01f2390 msgr2=0x55ccf05ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01f2390 0x55ccf05ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 -- 172.21.15.29:0/3824576422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 msgr2=0x55ccf05ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 0x55ccf05ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf0641f90 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852481e640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05cae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85200521d0 tx=0x7f852006c550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3453e7458f9a2390 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f84ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520098e60 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c00bbe0 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f85200a06d0 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f85200a0c80 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.875+0000 7f852c7f5740 10 client.25660 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ccf06431c0 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c00bbe0 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f852006a680 con 0x55ccf0213070 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 10 client.25660 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 10 client.25660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:50:56.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 20 client.25660 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf0643940 0x55ccf0645d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 4 client.25660 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccf052e8a0 con 0x55ccf0643940 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.876+0000 7f851ffff640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf0643940 0x55ccf0645d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f84ff7fe640 10 client.25660 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f851ffff640 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf0643940 0x55ccf0645d60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bf8ba69fd4bd7a3 server_cookie=83f8967e5ae3a2d in_seq=0 out_seq=0 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f84ff7fe640 1 -- 172.21.15.29:0/3824576422 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccf052e8a0 con 0x55ccf0643940 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f84ff7fe640 10 client.25660 handle_command_reply: tid=0 2024-09-21T01:50:56.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f852c7f5740 1 client.25660 shutdown 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 msgr2=0x55ccf05cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.877+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05cae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85200521d0 tx=0x7f852006c550 comp rx=0 tx=0).stop 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 shutdown_connections 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55ccf0643940 0x55ccf0645d60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f84ff7fe640 0 client.25660 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccf0213070 0x55ccf05cae40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ccf01f2390 0x55ccf05ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 --2- 172.21.15.29:0/3824576422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ccf01d19f0 0x55ccf05ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 >> 172.21.15.29:0/3824576422 conn(0x55ccf0207460 msgr2=0x55ccf05bd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 shutdown_connections 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 1 -- 172.21.15.29:0/3824576422 wait complete. 2024-09-21T01:50:56.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.878+0000 7f852c7f5740 20 client.25660 trim_cache size 0 max 16384 2024-09-21T01:50:56.882 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 msgr2=0x7f85100b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 0x7f85100b8aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f850c010e90 tx=0x7f850c000bc0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 msgr2=0x7f85181334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f851400d740 tx=0x7f851400dc00 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 shutdown_connections 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85100b6590 0x7f85100b8aa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851809a1a0 0x7f8518138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.879+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180956b0 0x7f85181334f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.880+0000 7f852501f640 1 --2- 172.21.15.29:0/3011018727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518094d60 0x7f8518132f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.880+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 >> 172.21.15.29:0/3011018727 conn(0x7f851800cd70 msgr2=0x7f8518097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.880+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 shutdown_connections 2024-09-21T01:50:56.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:56.880+0000 7f852501f640 1 -- 172.21.15.29:0/3011018727 wait complete. 2024-09-21T01:50:57.622 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:57 smithi029 ceph-mon[31228]: pgmap v438: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 46 MiB/s wr, 984 op/s 2024-09-21T01:50:57.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:50:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:57 smithi190 ceph-mon[34368]: pgmap v438: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 46 MiB/s wr, 984 op/s 2024-09-21T01:50:57.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:57 smithi066 ceph-mon[35001]: pgmap v438: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 46 MiB/s wr, 984 op/s 2024-09-21T01:50:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 140.0 s, 3190.3 tps, lat 31.405 ms stddev 39.556 2024-09-21T01:50:58.041 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:50:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 -- 172.21.15.29:0/3291730934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01039b0 msgr2=0x7fb3d0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 --2- 172.21.15.29:0/3291730934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01039b0 0x7fb3d0103dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009860 tx=0x7fb3c002a380 comp rx=0 tx=0).stop 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 -- 172.21.15.29:0/3291730934 shutdown_connections 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 --2- 172.21.15.29:0/3291730934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d0109c40 0x7fb3d0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 --2- 172.21.15.29:0/3291730934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d01092f0 0x7fb3d0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 --2- 172.21.15.29:0/3291730934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01039b0 0x7fb3d0103dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.688+0000 7fb3cffff640 1 -- 172.21.15.29:0/3291730934 >> 172.21.15.29:0/3291730934 conn(0x7fb3d00ff440 msgr2=0x7fb3d0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.689+0000 7fb3cffff640 1 -- 172.21.15.29:0/3291730934 shutdown_connections 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.689+0000 7fb3cffff640 1 -- 172.21.15.29:0/3291730934 wait complete. 2024-09-21T01:50:58.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.690+0000 7fb3cffff640 1 Processor -- start 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.690+0000 7fb3cffff640 1 -- start start 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d01039b0 0x7fb3d019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 0x7fb3d019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 0x7fb3d019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0113950 con 0x7fb3d0109c40 2024-09-21T01:50:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d01137d0 con 0x7fb3d01039b0 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3cffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0113ad0 con 0x7fb3d01092f0 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.691+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 0x7fb3d019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 0x7fb3d019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 0x7fb3d019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36526/0 (socket says 172.21.15.29:36526) 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/2009612 learned_addr learned my addr 172.21.15.29:0/2009612 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d01039b0 0x7fb3d019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ce7fc640 1 -- 172.21.15.29:0/2009612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d01039b0 msgr2=0x7fb3d019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d01039b0 0x7fb3d019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ce7fc640 1 -- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 msgr2=0x7fb3d019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 0x7fb3d019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.692+0000 7fb3ce7fc640 1 -- 172.21.15.29:0/2009612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d01a0260 con 0x7fb3d01092f0 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.693+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 0x7fb3d019a8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3d006b990 tx=0x7fb3c400db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.695+0000 7fb3d48ff640 1 -- 172.21.15.29:0/2009612 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4017070 con 0x7fb3d01092f0 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.695+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01a0550 con 0x7fb3d01092f0 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.695+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d00755f0 con 0x7fb3d01092f0 2024-09-21T01:50:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.695+0000 7fb3d48ff640 1 -- 172.21.15.29:0/2009612 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c4010460 con 0x7fb3d01092f0 2024-09-21T01:50:58.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.695+0000 7fb3d48ff640 1 -- 172.21.15.29:0/2009612 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f620 con 0x7fb3d01092f0 2024-09-21T01:50:58.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.705+0000 7fb3d48ff640 1 -- 172.21.15.29:0/2009612 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb3c401a070 con 0x7fb3d01092f0 2024-09-21T01:50:58.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.706+0000 7fb3d48ff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 0x7fb3b80b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.707+0000 7fb3d48ff640 1 -- 172.21.15.29:0/2009612 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb3c40e6ac0 con 0x7fb3d01092f0 2024-09-21T01:50:58.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.707+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 0x7fb3b80b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.708+0000 7fb3dbe75740 1 Processor -- start 2024-09-21T01:50:58.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.708+0000 7fb3dbe75740 1 -- start start 2024-09-21T01:50:58.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x561471267460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e971b0 0x5614712679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614711ddea0 con 0x561470ec0070 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614711cd2d0 con 0x561470e971b0 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5614711d10e0 con 0x561470e9f390 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33224/0 (socket says 172.21.15.29:33224) 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/2454323449 learned_addr learned my addr 172.21.15.29:0/2454323449 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.709+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 0x7fb3b80b8c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00095f0 tx=0x7fb3c0030980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/2454323449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e971b0 msgr2=0x5614712679a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e971b0 0x5614712679a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/2454323449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 msgr2=0x561471267460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x561471267460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/2454323449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561470f2b650 con 0x561470e9f390 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fa0 tx=0x7fb3c8011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ba1e5365528b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3b5ffb640 1 -- 172.21.15.29:0/2454323449 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8012710 con 0x561470e9f390 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.710+0000 7fb3b5ffb640 1 -- 172.21.15.29:0/2454323449 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c8012d60 con 0x561470e9f390 2024-09-21T01:50:58.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.711+0000 7fb3b5ffb640 1 -- 172.21.15.29:0/2454323449 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800fe10 con 0x561470e9f390 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 -- 172.21.15.29:0/2454323449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 msgr2=0x561471267ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fa0 tx=0x7fb3c8011870 comp rx=0 tx=0).stop 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 -- 172.21.15.29:0/2454323449 shutdown_connections 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e9f390 0x561471267ee0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e971b0 0x5614712679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/2454323449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x561471267460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 -- 172.21.15.29:0/2454323449 >> 172.21.15.29:0/2454323449 conn(0x561470eb4460 msgr2=0x561471265350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.712+0000 7fb3dbe75740 1 -- 172.21.15.29:0/2454323449 shutdown_connections 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.713+0000 7fb3dbe75740 1 -- 172.21.15.29:0/2454323449 wait complete. 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.713+0000 7fb3dbe75740 1 -- start start 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 Processor -- start 2024-09-21T01:50:58.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471273030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x5614712735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56147102e700 con 0x561470e971b0 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561471168160 con 0x561470e9f390 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561471275c90 con 0x561470ec0070 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3dbe75740 10 client.0 _ll_register_callbacks cb 0x5614711819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36536/0 (socket says 172.21.15.29:36536) 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3ceffd640 1 -- 172.21.15.29:0/254241385 learned_addr learned my addr 172.21.15.29:0/254241385 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:58.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.714+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x5614712735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.715+0000 7fb3ceffd640 1 -- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 msgr2=0x5614712735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.715+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x5614712735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.715+0000 7fb3ceffd640 1 -- 172.21.15.29:0/254241385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 msgr2=0x561471273030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.715+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471273030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.715+0000 7fb3ceffd640 1 -- 172.21.15.29:0/254241385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561471195c90 con 0x561470e971b0 2024-09-21T01:50:58.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.716+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0034000 tx=0x7fb3c0030e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33442f98b64d439 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.717+0000 7fb3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.717+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c004e060 con 0x561470e971b0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.717+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb3c0005450 con 0x561470e971b0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.717+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c8012d60 con 0x561470e971b0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.717+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3c0037040 con 0x561470e971b0 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 10 client.15918 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:58.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614712819c0 con 0x561470e971b0 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c0004b70 con 0x561470e971b0 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb3c00039b0 con 0x561470e971b0 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 10 client.15918 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 10 client.15918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 20 client.15918 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561470e7e9f0 0x5614712853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 4 client.15918 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.718+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb3c8012d60 con 0x561470e7e9f0 2024-09-21T01:50:58.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.719+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561470e7e9f0 0x5614712853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.720+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561470e7e9f0 0x5614712853a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6ac61dd83ec71b0 server_cookie=9233b767b9cf9d93 in_seq=0 out_seq=0 2024-09-21T01:50:58.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.721+0000 7fb3aeffd640 10 client.15918 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.722+0000 7fb3aeffd640 1 -- 172.21.15.29:0/254241385 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb3c8012d60 con 0x561470e7e9f0 2024-09-21T01:50:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.722+0000 7fb3aeffd640 10 client.15918 handle_command_reply: tid=0 2024-09-21T01:50:58.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.722+0000 7fb3dbe75740 1 client.15918 shutdown 2024-09-21T01:50:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.722+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 msgr2=0x561471272af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.722+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0034000 tx=0x7fb3c0030e20 comp rx=0 tx=0).stop 2024-09-21T01:50:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 shutdown_connections 2024-09-21T01:50:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561470e7e9f0 0x5614712853a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x5614712735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471273030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/254241385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e971b0 0x561471272af0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3aeffd640 0 client.15918 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 >> 172.21.15.29:0/254241385 conn(0x561470eb4460 msgr2=0x56147126a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 shutdown_connections 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 1 -- 172.21.15.29:0/254241385 wait complete. 2024-09-21T01:50:58.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.724+0000 7fb3dbe75740 20 client.15918 trim_cache size 0 max 16384 2024-09-21T01:50:58.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.748+0000 7fb3dbe75740 1 Processor -- start 2024-09-21T01:50:58.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.748+0000 7fb3dbe75740 1 -- start start 2024-09-21T01:50:58.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.748+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.748+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471264ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.748+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x56147126fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc00e4e0 con 0x561470ec0070 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc010110 con 0x561470e9f390 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc00ebb0 con 0x561470e971b0 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33242/0 (socket says 172.21.15.29:33242) 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 -- 172.21.15.29:0/203161744 learned_addr learned my addr 172.21.15.29:0/203161744 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ce7fc640 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471264ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x56147126fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 -- 172.21.15.29:0/203161744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 msgr2=0x561471264ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471264ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 -- 172.21.15.29:0/203161744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 msgr2=0x56147126fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x56147126fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.749+0000 7fb3ceffd640 1 -- 172.21.15.29:0/203161744 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561471273b90 con 0x561470e971b0 2024-09-21T01:50:58.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.750+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3c003d040 tx=0x7fb3c002aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1bf1fcd6ec2f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.750+0000 7fb3b57fa640 1 -- 172.21.15.29:0/203161744 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0058070 con 0x561470e971b0 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.750+0000 7fb3b57fa640 1 -- 172.21.15.29:0/203161744 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c0004490 con 0x561470e971b0 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.750+0000 7fb3b57fa640 1 -- 172.21.15.29:0/203161744 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c004e070 con 0x561470e971b0 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 -- 172.21.15.29:0/203161744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 msgr2=0x561471264980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3c003d040 tx=0x7fb3c002aa00 comp rx=0 tx=0).stop 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 -- 172.21.15.29:0/203161744 shutdown_connections 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470ec0070 0x56147126fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e9f390 0x561471264ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/203161744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470e971b0 0x561471264980 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3c003d040 tx=0x7fb3c002aa00 comp rx=0 tx=0).stop 2024-09-21T01:50:58.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.751+0000 7fb3dbe75740 1 -- 172.21.15.29:0/203161744 >> 172.21.15.29:0/203161744 conn(0x561470eb4460 msgr2=0x561471271370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.753+0000 7fb3dbe75740 1 -- 172.21.15.29:0/203161744 shutdown_connections 2024-09-21T01:50:58.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.753+0000 7fb3dbe75740 1 -- 172.21.15.29:0/203161744 wait complete. 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.753+0000 7fb3dbe75740 1 -- start start 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 Processor -- start 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e9f390 0x561471277730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x561471277c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc00f980 con 0x561470e9f390 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc00f2b0 con 0x561470e53da0 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc00c4c0 con 0x561470ec0070 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.754+0000 7fb3dbe75740 10 client.0 _ll_register_callbacks cb 0x7fb3c0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:50:58.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x561471277c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51050/0 (socket says 172.21.15.29:51050) 2024-09-21T01:50:58.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/1023859294 learned_addr learned my addr 172.21.15.29:0/1023859294 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 msgr2=0x561471277c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x561471277c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/1023859294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e9f390 msgr2=0x561471277730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e9f390 0x561471277730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 -- 172.21.15.29:0/1023859294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614713003a0 con 0x561470e53da0 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.755+0000 7fb3cf7fe640 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fd0 tx=0x7fb3c8012740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6390092e7c572150 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:50:58.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8012cc0 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb3c802f410 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c0004490 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3c802f650 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3dbe75740 10 client.25631 fetch_fsmap learned FSMap version 31 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.756+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561471301620 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c0004490 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb3c8025660 con 0x561470e53da0 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 10 client.25631 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:50:58.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 10 client.25631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:50:58.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 20 client.25631 populate_metadata read hostname 'smithi029' 2024-09-21T01:50:58.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561471301da0 0x5614713041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:50:58.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 4 client.25631 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:50:58.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614711dfdf0 con 0x561471301da0 2024-09-21T01:50:58.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.757+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561471301da0 0x5614713041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.758+0000 7fb3ceffd640 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561471301da0 0x5614713041c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a63c85c005f2a135 server_cookie=ec5aa97336ee98d1 in_seq=0 out_seq=0 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.759+0000 7fb3aeffd640 10 client.25631 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.759+0000 7fb3aeffd640 1 -- 172.21.15.29:0/1023859294 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614711dfdf0 con 0x561471301da0 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.759+0000 7fb3aeffd640 10 client.25631 handle_command_reply: tid=0 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.759+0000 7fb3dbe75740 1 client.25631 shutdown 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 msgr2=0x5614712eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fd0 tx=0x7fb3c8012740 comp rx=0 tx=0).stop 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 shutdown_connections 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x561471301da0 0x5614713041c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561470e53da0 0x5614712eeda0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561470ec0070 0x561471277c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3aeffd640 0 client.25631 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 --2- 172.21.15.29:0/1023859294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561470e9f390 0x561471277730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 >> 172.21.15.29:0/1023859294 conn(0x561470eb4460 msgr2=0x561471268490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 shutdown_connections 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 1 -- 172.21.15.29:0/1023859294 wait complete. 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.760+0000 7fb3dbe75740 20 client.25631 trim_cache size 0 max 16384 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.761+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 msgr2=0x7fb3b80b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.761+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 0x7fb3b80b8c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00095f0 tx=0x7fb3c0030980 comp rx=0 tx=0).stop 2024-09-21T01:50:58.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.761+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 msgr2=0x7fb3d019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.761+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 0x7fb3d019a8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3d006b990 tx=0x7fb3c400db30 comp rx=0 tx=0).stop 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 shutdown_connections 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb3b80b6740 0x7fb3b80b8c50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d0109c40 0x7fb3d019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01092f0 0x7fb3d019a8c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 --2- 172.21.15.29:0/2009612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3d01039b0 0x7fb3d019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 >> 172.21.15.29:0/2009612 conn(0x7fb3d00ff440 msgr2=0x7fb3d0108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 shutdown_connections 2024-09-21T01:50:58.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:50:58.762+0000 7fb3cffff640 1 -- 172.21.15.29:0/2009612 wait complete. 2024-09-21T01:50:59.677 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:50:59 smithi029 ceph-mon[31228]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 723 op/s 2024-09-21T01:50:59.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:50:59.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:50:59 smithi190 ceph-mon[34368]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 723 op/s 2024-09-21T01:50:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:50:59 smithi066 ceph-mon[35001]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 723 op/s 2024-09-21T01:51:00.138 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 -- 172.21.15.29:0/2418532877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae841083d0 msgr2=0x7fae841087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 --2- 172.21.15.29:0/2418532877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae841083d0 0x7fae841087e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae78009900 tx=0x7fae7802a400 comp rx=0 tx=0).stop 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 -- 172.21.15.29:0/2418532877 shutdown_connections 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 --2- 172.21.15.29:0/2418532877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84108d20 0x7fae8410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 --2- 172.21.15.29:0/2418532877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae841083d0 0x7fae841087e0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae78009900 tx=0x7fae7802a400 comp rx=0 tx=0).stop 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 --2- 172.21.15.29:0/2418532877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.777+0000 7fae890b4640 1 -- 172.21.15.29:0/2418532877 >> 172.21.15.29:0/2418532877 conn(0x7fae84079790 msgr2=0x7fae84079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.778+0000 7fae890b4640 1 -- 172.21.15.29:0/2418532877 shutdown_connections 2024-09-21T01:51:00.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.779+0000 7fae890b4640 1 -- 172.21.15.29:0/2418532877 wait complete. 2024-09-21T01:51:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.781+0000 7fae890b4640 1 Processor -- start 2024-09-21T01:51:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.781+0000 7fae890b4640 1 -- start start 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84199be0 0x7fae841a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae8259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae8259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51078/0 (socket says 172.21.15.29:51078) 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae8259c640 1 -- 172.21.15.29:0/2171222664 learned_addr learned my addr 172.21.15.29:0/2171222664 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae841129e0 con 0x7fae84199be0 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112860 con 0x7fae84108d20 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112b60 con 0x7fae84102ae0 2024-09-21T01:51:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.782+0000 7fae8359e640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84199be0 0x7fae841a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae8259c640 1 -- 172.21.15.29:0/2171222664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 msgr2=0x7fae84199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae8259c640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae8259c640 1 -- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84199be0 msgr2=0x7fae841a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae8259c640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84199be0 0x7fae841a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae8259c640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841a8d20 con 0x7fae84108d20 2024-09-21T01:51:00.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.783+0000 7fae82d9d640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:00.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae8259c640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae78006010 tx=0x7fae780040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae6bfff640 1 -- 172.21.15.29:0/2171222664 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7803f070 con 0x7fae84108d20 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae6bfff640 1 -- 172.21.15.29:0/2171222664 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae780044c0 con 0x7fae84108d20 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae6bfff640 1 -- 172.21.15.29:0/2171222664 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78035070 con 0x7fae84108d20 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841a8fb0 con 0x7fae84108d20 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.784+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a9350 con 0x7fae84108d20 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.788+0000 7fae8f628740 1 Processor -- start 2024-09-21T01:51:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.788+0000 7fae8f628740 1 -- start start 2024-09-21T01:51:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.789+0000 7fae6bfff640 1 -- 172.21.15.29:0/2171222664 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fae78043400 con 0x7fae84108d20 2024-09-21T01:51:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edf52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edf5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56227ed5aea0 con 0x56227ea1c390 2024-09-21T01:51:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56227ed4a2d0 con 0x56227ea141b0 2024-09-21T01:51:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.790+0000 7fae8f628740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56227ed4e0e0 con 0x56227ea3d070 2024-09-21T01:51:00.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.791+0000 7fae8359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edf52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.791+0000 7fae8359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edf52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33282/0 (socket says 172.21.15.29:33282) 2024-09-21T01:51:00.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.791+0000 7fae8359e640 1 -- 172.21.15.29:0/2610502587 learned_addr learned my addr 172.21.15.29:0/2610502587 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:00.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.791+0000 7fae6bfff640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 0x7fae6c0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.792+0000 7fae6bfff640 1 -- 172.21.15.29:0/2171222664 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fae781085f0 con 0x7fae84108d20 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.793+0000 7fae8259c640 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edf5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 -- 172.21.15.29:0/2610502587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 msgr2=0x56227edf52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edf52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 -- 172.21.15.29:0/2610502587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 msgr2=0x56227edf5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 -- 172.21.15.29:0/2610502587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227eaa8650 con 0x56227ea1c390 2024-09-21T01:51:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.794+0000 7fae8259c640 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edf5d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae78033000 tx=0x7fae780ce130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40541f34a44a2b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.796+0000 7fae697fa640 1 -- 172.21.15.29:0/2610502587 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78048050 con 0x56227ea1c390 2024-09-21T01:51:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.796+0000 7fae697fa640 1 -- 172.21.15.29:0/2610502587 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae780d3210 con 0x56227ea1c390 2024-09-21T01:51:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.796+0000 7fae697fa640 1 -- 172.21.15.29:0/2610502587 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7803b6c0 con 0x56227ea1c390 2024-09-21T01:51:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.797+0000 7fae8f628740 1 -- 172.21.15.29:0/2610502587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 msgr2=0x56227edf5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.797+0000 7fae8f628740 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edf5d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae78033000 tx=0x7fae780ce130 comp rx=0 tx=0).stop 2024-09-21T01:51:00.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.799+0000 7fae8359e640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 0x7fae6c0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.801+0000 7fae8f628740 1 -- 172.21.15.29:0/2610502587 shutdown_connections 2024-09-21T01:51:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.801+0000 7fae8f628740 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edf5d50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.801+0000 7fae8f628740 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.801+0000 7fae8f628740 1 --2- 172.21.15.29:0/2610502587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edf52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.801+0000 7fae8f628740 1 -- 172.21.15.29:0/2610502587 >> 172.21.15.29:0/2610502587 conn(0x56227ea31460 msgr2=0x56227edf31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.803+0000 7fae8f628740 1 -- 172.21.15.29:0/2610502587 shutdown_connections 2024-09-21T01:51:00.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.803+0000 7fae8f628740 1 -- 172.21.15.29:0/2610502587 wait complete. 2024-09-21T01:51:00.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.803+0000 7fae8f628740 1 -- start start 2024-09-21T01:51:00.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.803+0000 7fae8f628740 1 Processor -- start 2024-09-21T01:51:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 0x56227ee00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee00e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea3d070 0x56227ee01400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56227ebab700 con 0x56227ea3d070 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56227ece5160 con 0x56227ea1c390 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56227ee03870 con 0x56227ea141b0 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.805+0000 7fae8f628740 10 client.0 _ll_register_callbacks cb 0x56227ecfe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.806+0000 7fae8359e640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 0x7fae6c0b8bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005f40 tx=0x7fae7c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.806+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 0x56227ee00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.806+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 0x56227ee00940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33298/0 (socket says 172.21.15.29:33298) 2024-09-21T01:51:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.806+0000 7fae82d9d640 1 -- 172.21.15.29:0/1163511746 learned_addr learned my addr 172.21.15.29:0/1163511746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.807+0000 7fae8259c640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee00e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.807+0000 7fae8259c640 1 -- 172.21.15.29:0/1163511746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 msgr2=0x56227ee00940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.807+0000 7fae8259c640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 0x56227ee00940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae8259c640 1 -- 172.21.15.29:0/1163511746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea3d070 msgr2=0x56227ee01400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae8259c640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea3d070 0x56227ee01400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae8259c640 1 -- 172.21.15.29:0/1163511746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227ed12c90 con 0x56227ea1c390 2024-09-21T01:51:00.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae8259c640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee00e80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae78005940 tx=0x7fae780ce260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a9a4481e1d09e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:00.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.808+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae780cdc40 con 0x56227ea1c390 2024-09-21T01:51:00.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.809+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae780d3210 con 0x56227ea1c390 2024-09-21T01:51:00.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.809+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fae7803b460 con 0x56227ea1c390 2024-09-21T01:51:00.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.809+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae7807b050 con 0x56227ea1c390 2024-09-21T01:51:00.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.809+0000 7fae8f628740 10 client.25640 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:00.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.809+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56227ee06290 con 0x56227ea1c390 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae780d3210 con 0x56227ea1c390 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fae7804b060 con 0x56227ea1c390 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 10 client.25640 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 10 client.25640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 20 client.25640 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9fb9f0 0x56227ee09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 4 client.25640 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56227ed5cdf0 con 0x56227e9fb9f0 2024-09-21T01:51:00.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.812+0000 7fae82d9d640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9fb9f0 0x56227ee09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.813+0000 7fae66ff5640 10 client.25640 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:00.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.814+0000 7fae82d9d640 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9fb9f0 0x56227ee09810 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f743c205a5e2f781 server_cookie=66711929fc937577 in_seq=0 out_seq=0 2024-09-21T01:51:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.815+0000 7fae66ff5640 1 -- 172.21.15.29:0/1163511746 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56227ed5cdf0 con 0x56227e9fb9f0 2024-09-21T01:51:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.815+0000 7fae66ff5640 10 client.25640 handle_command_reply: tid=0 2024-09-21T01:51:00.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.815+0000 7fae8f628740 1 client.25640 shutdown 2024-09-21T01:51:00.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.815+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 msgr2=0x56227ee00e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.815+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee00e80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae78005940 tx=0x7fae780ce260 comp rx=0 tx=0).stop 2024-09-21T01:51:00.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 shutdown_connections 2024-09-21T01:51:00.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9fb9f0 0x56227ee09810 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea3d070 0x56227ee01400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee00e80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 --2- 172.21.15.29:0/1163511746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea141b0 0x56227ee00940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 >> 172.21.15.29:0/1163511746 conn(0x56227ea31460 msgr2=0x56227edf8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae66ff5640 0 client.25640 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:00.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 shutdown_connections 2024-09-21T01:51:00.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 1 -- 172.21.15.29:0/1163511746 wait complete. 2024-09-21T01:51:00.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.818+0000 7fae8f628740 20 client.25640 trim_cache size 0 max 16384 2024-09-21T01:51:00.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 Processor -- start 2024-09-21T01:51:00.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 -- start start 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae7000e420 con 0x56227ea1c390 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae70010050 con 0x56227ea141b0 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8f628740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae7000eaf0 con 0x56227ea3d070 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51122/0 (socket says 172.21.15.29:51122) 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae82d9d640 1 -- 172.21.15.29:0/3059731286 learned_addr learned my addr 172.21.15.29:0/3059731286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:00.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8359e640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.845+0000 7fae8259c640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 -- 172.21.15.29:0/3059731286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 msgr2=0x56227edff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 -- 172.21.15.29:0/3059731286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 msgr2=0x56227edff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 -- 172.21.15.29:0/3059731286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227ee01c00 con 0x56227ea141b0 2024-09-21T01:51:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae8359e640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.846+0000 7fae82d9d640 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae74005fd0 tx=0x7fae74012420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503267cdf9b1c468 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.848+0000 7fae68ff9640 1 -- 172.21.15.29:0/3059731286 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74013d40 con 0x56227ea141b0 2024-09-21T01:51:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.848+0000 7fae68ff9640 1 -- 172.21.15.29:0/3059731286 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae740035f0 con 0x56227ea141b0 2024-09-21T01:51:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 -- 172.21.15.29:0/3059731286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 msgr2=0x56227edf6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae74005fd0 tx=0x7fae74012420 comp rx=0 tx=0).stop 2024-09-21T01:51:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 -- 172.21.15.29:0/3059731286 shutdown_connections 2024-09-21T01:51:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227edff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea1c390 0x56227edff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 --2- 172.21.15.29:0/3059731286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea141b0 0x56227edf6de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 -- 172.21.15.29:0/3059731286 >> 172.21.15.29:0/3059731286 conn(0x56227ea31460 msgr2=0x56227edfbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.849+0000 7fae8f628740 1 -- 172.21.15.29:0/3059731286 shutdown_connections 2024-09-21T01:51:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 -- 172.21.15.29:0/3059731286 wait complete. 2024-09-21T01:51:00.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 -- start start 2024-09-21T01:51:00.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 Processor -- start 2024-09-21T01:51:00.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 0x56227ee0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227ee0b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae7000f8c0 con 0x56227ea141b0 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae7000f1f0 con 0x56227ea1c390 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.850+0000 7fae8f628740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae7000c400 con 0x56227ea3d070 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae8f628740 10 client.0 _ll_register_callbacks cb 0x7fae780d8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 0x56227ee0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 0x56227ee0f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36622/0 (socket says 172.21.15.29:36622) 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae82d9d640 1 -- 172.21.15.29:0/3837678678 learned_addr learned my addr 172.21.15.29:0/3837678678 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae8359e640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227ee0b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.851+0000 7fae8259c640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 -- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 msgr2=0x56227ee0ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee0ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 -- 172.21.15.29:0/3837678678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 msgr2=0x56227ee0f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 0x56227ee0f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 -- 172.21.15.29:0/3837678678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227ee0b830 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8259c640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee0ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8359e640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227ee0b140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005fd0 tx=0x7fae7c009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9335183eb8f7b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c01fb70 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae740035f0 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.852+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fae7c01fd10 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.853+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae7c01c440 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.853+0000 7fae8f628740 10 client.25696 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.853+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56227ee8e5a0 con 0x56227ea3d070 2024-09-21T01:51:00.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.853+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae740035f0 con 0x56227ea3d070 2024-09-21T01:51:00.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.853+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fae7c00f040 con 0x56227ea3d070 2024-09-21T01:51:00.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 10 client.25696 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 10 client.25696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 20 client.25696 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9d0da0 0x56227ee90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 4 client.25696 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56227ed588a0 con 0x56227e9d0da0 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.856+0000 7fae82d9d640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9d0da0 0x56227ee90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae66ff5640 10 client.25696 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae82d9d640 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9d0da0 0x56227ee90ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2aae9011aa72bb server_cookie=ceff8f9b9c5f7a44 in_seq=0 out_seq=0 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae66ff5640 1 -- 172.21.15.29:0/3837678678 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56227ed588a0 con 0x56227e9d0da0 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae66ff5640 10 client.25696 handle_command_reply: tid=0 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae8f628740 1 client.25696 shutdown 2024-09-21T01:51:00.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.857+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 msgr2=0x56227ee0b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227ee0b140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005fd0 tx=0x7fae7c009600 comp rx=0 tx=0).stop 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 shutdown_connections 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56227e9d0da0 0x56227ee90ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56227ea3d070 0x56227ee0b140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56227ea1c390 0x56227ee0ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 --2- 172.21.15.29:0/3837678678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56227ea141b0 0x56227ee0f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 >> 172.21.15.29:0/3837678678 conn(0x56227ea31460 msgr2=0x56227edfbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae66ff5640 0 client.25696 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 shutdown_connections 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 1 -- 172.21.15.29:0/3837678678 wait complete. 2024-09-21T01:51:00.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.858+0000 7fae8f628740 20 client.25696 trim_cache size 0 max 16384 2024-09-21T01:51:00.861 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.859+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 msgr2=0x7fae6c0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.859+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 0x7fae6c0b8bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005f40 tx=0x7fae7c004a40 comp rx=0 tx=0).stop 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.859+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 msgr2=0x7fae841996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.859+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae78006010 tx=0x7fae780040c0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 shutdown_connections 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fae6c0b66e0 0x7fae6c0b8bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae84199be0 0x7fae841a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae84108d20 0x7fae841996a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 --2- 172.21.15.29:0/2171222664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84102ae0 0x7fae84199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:00.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 >> 172.21.15.29:0/2171222664 conn(0x7fae84079790 msgr2=0x7fae84106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:00.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 shutdown_connections 2024-09-21T01:51:00.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:00.860+0000 7fae890b4640 1 -- 172.21.15.29:0/2171222664 wait complete. 2024-09-21T01:51:01.456 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:01 smithi029 ceph-mon[31228]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 MiB/s wr, 938 op/s 2024-09-21T01:51:01.630 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:51:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:01 smithi190 ceph-mon[34368]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 MiB/s wr, 938 op/s 2024-09-21T01:51:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:01 smithi066 ceph-mon[35001]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 MiB/s wr, 938 op/s 2024-09-21T01:51:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.253+0000 7faa39dc3640 1 -- 172.21.15.29:0/3891462287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 msgr2=0x7faa34102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.253+0000 7faa39dc3640 1 --2- 172.21.15.29:0/3891462287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34102a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa24009900 tx=0x7faa2402a410 comp rx=0 tx=0).stop 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 -- 172.21.15.29:0/3891462287 shutdown_connections 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 --2- 172.21.15.29:0/3891462287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 0x7faa3410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 --2- 172.21.15.29:0/3891462287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 0x7faa34103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 --2- 172.21.15.29:0/3891462287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34102a20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 -- 172.21.15.29:0/3891462287 >> 172.21.15.29:0/3891462287 conn(0x7faa340fdf30 msgr2=0x7faa341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 -- 172.21.15.29:0/3891462287 shutdown_connections 2024-09-21T01:51:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.254+0000 7faa39dc3640 1 -- 172.21.15.29:0/3891462287 wait complete. 2024-09-21T01:51:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.255+0000 7faa39dc3640 1 Processor -- start 2024-09-21T01:51:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 -- start start 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 0x7faa341983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 0x7faa34198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa34111530 con 0x7faa34103fd0 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa341113b0 con 0x7faa34103630 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa39dc3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa341116b0 con 0x7faa341025b0 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 0x7faa34198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34162/0 (socket says 172.21.15.29:34162) 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa337fe640 1 -- 172.21.15.29:0/2744897253 learned_addr learned my addr 172.21.15.29:0/2744897253 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.256+0000 7faa32ffd640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 0x7faa341983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 -- 172.21.15.29:0/2744897253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 msgr2=0x7faa341983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 0x7faa341983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 -- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 msgr2=0x7faa34198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 0x7faa34198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 -- 172.21.15.29:0/2744897253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa341997b0 con 0x7faa341025b0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa337fe640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa24031440 tx=0x7faa24003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa30ff9640 1 -- 172.21.15.29:0/2744897253 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2403f070 con 0x7faa341025b0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa34199a40 con 0x7faa341025b0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa30ff9640 1 -- 172.21.15.29:0/2744897253 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa24031c50 con 0x7faa341025b0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.257+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3419e180 con 0x7faa341025b0 2024-09-21T01:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.258+0000 7faa30ff9640 1 -- 172.21.15.29:0/2744897253 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24035030 con 0x7faa341025b0 2024-09-21T01:51:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.261+0000 7faa30ff9640 1 -- 172.21.15.29:0/2744897253 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7faa24042070 con 0x7faa341025b0 2024-09-21T01:51:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.264+0000 7faa40e0b740 1 Processor -- start 2024-09-21T01:51:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.264+0000 7faa40e0b740 1 -- start start 2024-09-21T01:51:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 0x55e15305fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 0x55e1530604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153061020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e152fc16f0 con 0x55e152c7e2f0 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e152fb0b20 con 0x55e152ca71b0 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa40e0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e152fb4930 con 0x55e152c864d0 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa33fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 0x55e15305fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa33fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 0x55e15305fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35344/0 (socket says 172.21.15.29:35344) 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa33fff640 1 -- 172.21.15.29:0/4282823373 learned_addr learned my addr 172.21.15.29:0/4282823373 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa32ffd640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153061020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.265+0000 7faa337fe640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 0x55e1530604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 -- 172.21.15.29:0/4282823373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 msgr2=0x55e15305fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 0x55e15305fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 -- 172.21.15.29:0/4282823373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 msgr2=0x55e1530604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 0x55e1530604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 -- 172.21.15.29:0/4282823373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e152f76530 con 0x55e152c864d0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa337fe640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 0x55e1530604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa32ffd640 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153061020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa28009c20 tx=0x7faa28011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e3e9306c36a487 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa30ff9640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 0x7faa1c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa19ffb640 1 -- 172.21.15.29:0/4282823373 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004050 con 0x55e152c864d0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa19ffb640 1 -- 172.21.15.29:0/4282823373 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa280054d0 con 0x55e152c864d0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.266+0000 7faa19ffb640 1 -- 172.21.15.29:0/4282823373 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004d00 con 0x55e152c864d0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.267+0000 7faa33fff640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 0x7faa1c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.267+0000 7faa30ff9640 1 -- 172.21.15.29:0/2744897253 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faa241089a0 con 0x7faa341025b0 2024-09-21T01:51:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 -- 172.21.15.29:0/4282823373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 msgr2=0x55e153061020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153061020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa28009c20 tx=0x7faa28011810 comp rx=0 tx=0).stop 2024-09-21T01:51:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 -- 172.21.15.29:0/4282823373 shutdown_connections 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153061020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c7e2f0 0x55e1530604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 --2- 172.21.15.29:0/4282823373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152ca71b0 0x55e15305fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa33fff640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 0x7faa1c0b8a10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0116b0 tx=0x7faa2c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.268+0000 7faa40e0b740 1 -- 172.21.15.29:0/4282823373 >> 172.21.15.29:0/4282823373 conn(0x55e152c9b300 msgr2=0x55e15304a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.269+0000 7faa40e0b740 1 -- 172.21.15.29:0/4282823373 shutdown_connections 2024-09-21T01:51:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.269+0000 7faa40e0b740 1 -- 172.21.15.29:0/4282823373 wait complete. 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.269+0000 7faa40e0b740 1 -- start start 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.270+0000 7faa40e0b740 1 Processor -- start 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.270+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 0x55e153069660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.270+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.270+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38218/0 (socket says 172.21.15.29:38218) 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa33fff640 1 -- 172.21.15.29:0/1224922129 learned_addr learned my addr 172.21.15.29:0/1224922129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e152ee8ab0 con 0x55e152ca71b0 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e152f56730 con 0x55e152c7e2f0 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.271+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e153061b40 con 0x55e152c864d0 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.272+0000 7faa32ffd640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.272+0000 7faa337fe640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 0x55e153069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.272+0000 7faa40e0b740 10 client.0 _ll_register_callbacks cb 0x55e152faa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 -- 172.21.15.29:0/1224922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 msgr2=0x55e153069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 -- 172.21.15.29:0/1224922129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 msgr2=0x55e15306a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e152d12790 con 0x55e152c7e2f0 2024-09-21T01:51:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa337fe640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 0x55e153069660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faa24106820 tx=0x7faa24038000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ce965418a4ca5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.273+0000 7faa12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.274+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24049040 con 0x55e152c7e2f0 2024-09-21T01:51:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.274+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faa240cae60 con 0x55e152c7e2f0 2024-09-21T01:51:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.274+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa280054d0 con 0x55e152c7e2f0 2024-09-21T01:51:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.275+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa24031e00 con 0x55e152c7e2f0 2024-09-21T01:51:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.275+0000 7faa40e0b740 10 client.25655 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.275+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e152d8b620 con 0x55e152c7e2f0 2024-09-21T01:51:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.275+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa280054d0 con 0x55e152c7e2f0 2024-09-21T01:51:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7faa240ca050 con 0x55e152c7e2f0 2024-09-21T01:51:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 10 client.25655 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 20 client.25655 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e152c65d80 0x55e153076e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 4 client.25655 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.276+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e152fc3640 con 0x55e152c65d80 2024-09-21T01:51:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.277+0000 7faa32ffd640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e152c65d80 0x55e153076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.288+0000 7faa32ffd640 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e152c65d80 0x55e153076e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14149720577438d server_cookie=11c9e4a5f45d5de2 in_seq=0 out_seq=0 2024-09-21T01:51:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.288+0000 7faa12ffd640 10 client.25655 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.292+0000 7faa12ffd640 1 -- 172.21.15.29:0/1224922129 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e152fc3640 con 0x55e152c65d80 2024-09-21T01:51:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.292+0000 7faa12ffd640 10 client.25655 handle_command_reply: tid=0 2024-09-21T01:51:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.292+0000 7faa40e0b740 1 client.25655 shutdown 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 msgr2=0x55e153069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 0x55e153069660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faa24106820 tx=0x7faa24038000 comp rx=0 tx=0).stop 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 shutdown_connections 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e152c65d80 0x55e153076e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e153069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 --2- 172.21.15.29:0/1224922129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c7e2f0 0x55e153069660 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 >> 172.21.15.29:0/1224922129 conn(0x55e152c9b300 msgr2=0x55e153048ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.296+0000 7faa12ffd640 0 client.25655 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.298+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 shutdown_connections 2024-09-21T01:51:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.298+0000 7faa40e0b740 1 -- 172.21.15.29:0/1224922129 wait complete. 2024-09-21T01:51:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.298+0000 7faa40e0b740 20 client.25655 trim_cache size 0 max 16384 2024-09-21T01:51:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.331+0000 7faa40e0b740 1 Processor -- start 2024-09-21T01:51:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.332+0000 7faa40e0b740 1 -- start start 2024-09-21T01:51:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.332+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 0x55e15304c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.332+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 0x55e15304a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34178/0 (socket says 172.21.15.29:34178) 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa32ffd640 1 -- 172.21.15.29:0/935573611 learned_addr learned my addr 172.21.15.29:0/935573611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa337fe640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 0x55e15304c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa33fff640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 0x55e15304a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa2000e4b0 con 0x55e152c3b1c0 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa200100e0 con 0x55e152c864d0 2024-09-21T01:51:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.333+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa2000eb80 con 0x55e152ca71b0 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 -- 172.21.15.29:0/935573611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 msgr2=0x55e153076d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 -- 172.21.15.29:0/935573611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 msgr2=0x55e15304a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 0x55e15304a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 -- 172.21.15.29:0/935573611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e15306ac00 con 0x55e152c864d0 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa33fff640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 0x55e15304a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa32ffd640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.334+0000 7faa337fe640 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 0x55e15304c020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa24106500 tx=0x7faa240317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ade45f88262de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.335+0000 7faa197fa640 1 -- 172.21.15.29:0/935573611 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24049070 con 0x55e152c864d0 2024-09-21T01:51:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.336+0000 7faa197fa640 1 -- 172.21.15.29:0/935573611 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa240ce700 con 0x55e152c864d0 2024-09-21T01:51:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.336+0000 7faa197fa640 1 -- 172.21.15.29:0/935573611 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240d80d0 con 0x55e152c864d0 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 msgr2=0x55e15304c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 0x55e15304c020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa24106500 tx=0x7faa240317a0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 shutdown_connections 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152c3b1c0 0x55e15304a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152ca71b0 0x55e153076d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 --2- 172.21.15.29:0/935573611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c864d0 0x55e15304c020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 >> 172.21.15.29:0/935573611 conn(0x55e152c9b300 msgr2=0x55e153068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.337+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 shutdown_connections 2024-09-21T01:51:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.338+0000 7faa40e0b740 1 -- 172.21.15.29:0/935573611 wait complete. 2024-09-21T01:51:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.338+0000 7faa40e0b740 1 -- start start 2024-09-21T01:51:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 Processor -- start 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c3b1c0 0x55e15306e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e15306ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa2000f950 con 0x55e152ca71b0 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa2000f280 con 0x55e152c3b1c0 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa2000c490 con 0x55e152c864d0 2024-09-21T01:51:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.339+0000 7faa40e0b740 10 client.0 _ll_register_callbacks cb 0x7faa24005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38230/0 (socket says 172.21.15.29:38230) 2024-09-21T01:51:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 -- 172.21.15.29:0/522254610 learned_addr learned my addr 172.21.15.29:0/522254610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa32ffd640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e15306ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 -- 172.21.15.29:0/522254610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 msgr2=0x55e15306ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e15306ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.340+0000 7faa33fff640 1 -- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c3b1c0 msgr2=0x55e15306e8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa33fff640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c3b1c0 0x55e15306e8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa33fff640 1 -- 172.21.15.29:0/522254610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e153070230 con 0x55e152ca71b0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa32ffd640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e15306ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa33fff640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00fb60 tx=0x7faa2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfff305cb586b614 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c02e060 con 0x55e152ca71b0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faa2c0109b0 con 0x55e152ca71b0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.341+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa240ce700 con 0x55e152ca71b0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.342+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa2c00fe60 con 0x55e152ca71b0 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.342+0000 7faa40e0b740 10 client.15954 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.342+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e153071620 con 0x55e152ca71b0 2024-09-21T01:51:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.342+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa240ce700 con 0x55e152ca71b0 2024-09-21T01:51:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.342+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7faa2c0042b0 con 0x55e152ca71b0 2024-09-21T01:51:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 10 client.15954 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 20 client.15954 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1530e1c00 0x55e1530e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 4 client.15954 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.343+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e152fbf0f0 con 0x55e1530e1c00 2024-09-21T01:51:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.344+0000 7faa337fe640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1530e1c00 0x55e1530e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.346+0000 7faa337fe640 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1530e1c00 0x55e1530e4020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0c6397668ca6f3 server_cookie=2a392148708c6711 in_seq=0 out_seq=0 2024-09-21T01:51:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.346+0000 7faa12ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.357+0000 7faa12ffd640 1 -- 172.21.15.29:0/522254610 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e152fbf0f0 con 0x55e1530e1c00 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.357+0000 7faa12ffd640 10 client.15954 handle_command_reply: tid=0 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.357+0000 7faa40e0b740 1 client.15954 shutdown 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 msgr2=0x55e15306fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00fb60 tx=0x7faa2c004a40 comp rx=0 tx=0).stop 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 shutdown_connections 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1530e1c00 0x55e1530e4020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e152ca71b0 0x55e15306fad0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e152c864d0 0x55e15306ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa12ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 --2- 172.21.15.29:0/522254610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e152c3b1c0 0x55e15306e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 >> 172.21.15.29:0/522254610 conn(0x55e152c9b300 msgr2=0x55e153068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 shutdown_connections 2024-09-21T01:51:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.358+0000 7faa40e0b740 1 -- 172.21.15.29:0/522254610 wait complete. 2024-09-21T01:51:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.377+0000 7faa40e0b740 20 client.15954 trim_cache size 0 max 16384 2024-09-21T01:51:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:51:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:51:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:51:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.379+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 msgr2=0x7faa1c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.379+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 0x7faa1c0b8a10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0116b0 tx=0x7faa2c009500 comp rx=0 tx=0).stop 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 msgr2=0x7faa34197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa24031440 tx=0x7faa24003c10 comp rx=0 tx=0).stop 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 shutdown_connections 2024-09-21T01:51:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7faa1c0b6500 0x7faa1c0b8a10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa34103fd0 0x7faa34198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa34103630 0x7faa341983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 --2- 172.21.15.29:0/2744897253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa341025b0 0x7faa34197d70 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa24031440 tx=0x7faa24003c10 comp rx=0 tx=0).stop 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 >> 172.21.15.29:0/2744897253 conn(0x7faa340fdf30 msgr2=0x7faa3410d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 shutdown_connections 2024-09-21T01:51:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:02.380+0000 7faa39dc3640 1 -- 172.21.15.29:0/2744897253 wait complete. 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 -- 172.21.15.29:0/1628294257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 msgr2=0x7f0178102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 --2- 172.21.15.29:0/1628294257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f0178102e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0160009860 tx=0x7f016002a340 comp rx=0 tx=0).stop 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 -- 172.21.15.29:0/1628294257 shutdown_connections 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 --2- 172.21.15.29:0/1628294257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178108cf0 0x7f017810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 --2- 172.21.15.29:0/1628294257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01781083a0 0x7f01781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 --2- 172.21.15.29:0/1628294257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f0178102e70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 -- 172.21.15.29:0/1628294257 >> 172.21.15.29:0/1628294257 conn(0x7f0178079730 msgr2=0x7f0178079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 -- 172.21.15.29:0/1628294257 shutdown_connections 2024-09-21T01:51:02.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.638+0000 7f017d3e8640 1 -- 172.21.15.29:0/1628294257 wait complete. 2024-09-21T01:51:02.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 Processor -- start 2024-09-21T01:51:02.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 -- start start 2024-09-21T01:51:02.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f017819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01781083a0 0x7f01781980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0178111860 con 0x7f0178102a60 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01781116e0 con 0x7f01781083a0 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.640+0000 7f017d3e8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01781119e0 con 0x7f0178108cf0 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34218/0 (socket says 172.21.15.29:34218) 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 -- 172.21.15.29:0/2004035864 learned_addr learned my addr 172.21.15.29:0/2004035864 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01767fc640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01781083a0 0x7f01781980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f0176ffd640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f017819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 -- 172.21.15.29:0/2004035864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01781083a0 msgr2=0x7f01781980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01781083a0 0x7f01781980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 -- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 msgr2=0x7f017819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f017819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 -- 172.21.15.29:0/2004035864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0178198e80 con 0x7f0178108cf0 2024-09-21T01:51:02.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.641+0000 7f01777fe640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f017000efc0 tx=0x7f017000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.642+0000 7f016ffff640 1 -- 172.21.15.29:0/2004035864 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170003970 con 0x7f0178108cf0 2024-09-21T01:51:02.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.642+0000 7f016ffff640 1 -- 172.21.15.29:0/2004035864 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0170004020 con 0x7f0178108cf0 2024-09-21T01:51:02.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.642+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017819db00 con 0x7f0178108cf0 2024-09-21T01:51:02.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.642+0000 7f016ffff640 1 -- 172.21.15.29:0/2004035864 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017000b940 con 0x7f0178108cf0 2024-09-21T01:51:02.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.642+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017819e040 con 0x7f0178108cf0 2024-09-21T01:51:02.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.646+0000 7f018395c740 1 Processor -- start 2024-09-21T01:51:02.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.646+0000 7f016ffff640 1 -- 172.21.15.29:0/2004035864 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f017001d020 con 0x7f0178108cf0 2024-09-21T01:51:02.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.646+0000 7f018395c740 1 -- start start 2024-09-21T01:51:02.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b17c61dea0 con 0x55b17c2df390 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b17c60d2d0 con 0x55b17c2d71b0 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f018395c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b17c6110e0 con 0x55b17c300070 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f01767fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.647+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35392/0 (socket says 172.21.15.29:35392) 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 -- 172.21.15.29:0/1659624189 learned_addr learned my addr 172.21.15.29:0/1659624189 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 -- 172.21.15.29:0/1659624189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 msgr2=0x55b17c6a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 -- 172.21.15.29:0/1659624189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 msgr2=0x55b17c6a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f0176ffd640 1 -- 172.21.15.29:0/1659624189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17c36b650 con 0x55b17c2d71b0 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.648+0000 7f01767fc640 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0160009780 tx=0x7f0160005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9feb555aa6b6ace3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f016d7fa640 1 -- 172.21.15.29:0/1659624189 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160040020 con 0x55b17c2d71b0 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f016d7fa640 1 -- 172.21.15.29:0/1659624189 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0160038040 con 0x55b17c2d71b0 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f016d7fa640 1 -- 172.21.15.29:0/1659624189 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160037070 con 0x55b17c2d71b0 2024-09-21T01:51:02.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f016ffff640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 0x7f01640b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.649+0000 7f01767fc640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 0x7f01640b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.650+0000 7f016ffff640 1 -- 172.21.15.29:0/2004035864 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f01700e7250 con 0x7f0178108cf0 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 -- 172.21.15.29:0/1659624189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 msgr2=0x55b17c6a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0160009780 tx=0x7f0160005dc0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f01767fc640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 0x7f01640b8a80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0178100c50 tx=0x7f0168009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 -- 172.21.15.29:0/1659624189 shutdown_connections 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6a7800 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 --2- 172.21.15.29:0/1659624189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 -- 172.21.15.29:0/1659624189 >> 172.21.15.29:0/1659624189 conn(0x55b17c2f4460 msgr2=0x55b17c6a51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 -- 172.21.15.29:0/1659624189 shutdown_connections 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.652+0000 7f018395c740 1 -- 172.21.15.29:0/1659624189 wait complete. 2024-09-21T01:51:02.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 -- start start 2024-09-21T01:51:02.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 Processor -- start 2024-09-21T01:51:02.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b17c46e700 con 0x55b17c2df390 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b17c5a8160 con 0x55b17c2d71b0 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b17c6b5a40 con 0x55b17c300070 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f018395c740 10 client.0 _ll_register_callbacks cb 0x55b17c5c19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.653+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35400/0 (socket says 172.21.15.29:35400) 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.654+0000 7f0176ffd640 1 -- 172.21.15.29:0/3308853708 learned_addr learned my addr 172.21.15.29:0/3308853708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.654+0000 7f01767fc640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.654+0000 7f01767fc640 1 -- 172.21.15.29:0/3308853708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 msgr2=0x55b17c6bed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f01767fc640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f01767fc640 1 -- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 msgr2=0x55b17c6b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f01767fc640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f01767fc640 1 -- 172.21.15.29:0/3308853708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17c5d5c90 con 0x55b17c2df390 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f0176ffd640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f01767fc640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6b2ef0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01680055f0 tx=0x7f0168005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696df07cbbd8ede0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.655+0000 7f015e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168027440 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0160038040 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f016801d430 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f016801d630 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f018395c740 10 client.15960 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b17c6c17a0 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0160038040 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0168025070 con 0x55b17c2df390 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f018395c740 10 client.15960 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.656+0000 7f018395c740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:02.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.657+0000 7f018395c740 20 client.15960 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.657+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c2be9f0 0x55b17c6c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.657+0000 7f018395c740 4 client.15960 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.657+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b17c61fdf0 con 0x55b17c2be9f0 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.657+0000 7f0176ffd640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c2be9f0 0x55b17c6c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.659+0000 7f0176ffd640 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c2be9f0 0x55b17c6c4dd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba97cb1c1cd31854 server_cookie=a83c88f19b95e411 in_seq=0 out_seq=0 2024-09-21T01:51:02.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.659+0000 7f015e7fc640 10 client.15960 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.659+0000 7f015e7fc640 1 -- 172.21.15.29:0/3308853708 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b17c61fdf0 con 0x55b17c2be9f0 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.659+0000 7f015e7fc640 10 client.15960 handle_command_reply: tid=0 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 client.15960 shutdown 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 msgr2=0x55b17c6b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6b2ef0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01680055f0 tx=0x7f0168005f90 comp rx=0 tx=0).stop 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 shutdown_connections 2024-09-21T01:51:02.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c2be9f0 0x55b17c6c4dd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6bed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2df390 0x55b17c6b2ef0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 --2- 172.21.15.29:0/3308853708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2d71b0 0x55b17c6b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f015e7fc640 0 client.15960 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 >> 172.21.15.29:0/3308853708 conn(0x55b17c2f4460 msgr2=0x55b17c6aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 shutdown_connections 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 1 -- 172.21.15.29:0/3308853708 wait complete. 2024-09-21T01:51:02.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.660+0000 7f018395c740 20 client.15960 trim_cache size 0 max 16384 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.682+0000 7f018395c740 1 Processor -- start 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.682+0000 7f018395c740 1 -- start start 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01780749c0 con 0x55b17c2d71b0 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01781c8570 con 0x55b17c2df390 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f018395c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0178075090 con 0x55b17c300070 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35412/0 (socket says 172.21.15.29:35412) 2024-09-21T01:51:02.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 -- 172.21.15.29:0/3898904795 learned_addr learned my addr 172.21.15.29:0/3898904795 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01777fe640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f0176ffd640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 -- 172.21.15.29:0/3898904795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 msgr2=0x55b17c6aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 -- 172.21.15.29:0/3898904795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 msgr2=0x55b17c6b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f01767fc640 1 -- 172.21.15.29:0/3898904795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17c2bec20 con 0x55b17c2df390 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.683+0000 7f0176ffd640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.684+0000 7f01767fc640 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0168027bf0 tx=0x7f0168005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abfbb2a58354410 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.684+0000 7f016cff9640 1 -- 172.21.15.29:0/3898904795 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016801d430 con 0x55b17c2df390 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.684+0000 7f016cff9640 1 -- 172.21.15.29:0/3898904795 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016802d030 con 0x55b17c2df390 2024-09-21T01:51:02.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.684+0000 7f016cff9640 1 -- 172.21.15.29:0/3898904795 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016801ddd0 con 0x55b17c2df390 2024-09-21T01:51:02.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 -- 172.21.15.29:0/3898904795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 msgr2=0x55b17c6aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0168027bf0 tx=0x7f0168005d30 comp rx=0 tx=0).stop 2024-09-21T01:51:02.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 -- 172.21.15.29:0/3898904795 shutdown_connections 2024-09-21T01:51:02.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6aa3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 --2- 172.21.15.29:0/3898904795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.685+0000 7f018395c740 1 -- 172.21.15.29:0/3898904795 >> 172.21.15.29:0/3898904795 conn(0x55b17c2f4460 msgr2=0x55b17c6b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.686+0000 7f018395c740 1 -- 172.21.15.29:0/3898904795 shutdown_connections 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.686+0000 7f018395c740 1 -- 172.21.15.29:0/3898904795 wait complete. 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.686+0000 7f018395c740 1 -- start start 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.686+0000 7f018395c740 1 Processor -- start 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01781c7de0 con 0x55b17c2d71b0 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01781c7710 con 0x55b17c2df390 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f01777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01780729a0 con 0x55b17c300070 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f018395c740 10 client.0 _ll_register_callbacks cb 0x7f0168010eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:02.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f0176ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38276/0 (socket says 172.21.15.29:38276) 2024-09-21T01:51:02.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.687+0000 7f0176ffd640 1 -- 172.21.15.29:0/1217449304 learned_addr learned my addr 172.21.15.29:0/1217449304 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:02.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f01767fc640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 -- 172.21.15.29:0/1217449304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 msgr2=0x55b17c6b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 -- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 msgr2=0x55b17c6b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 -- 172.21.15.29:0/1217449304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17c72e7c0 con 0x55b17c2d71b0 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f0176ffd640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0160000c00 tx=0x7f0160005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967e0d57814607b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.688+0000 7f015e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160037040 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f01600304d0 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016802d030 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0160040040 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f018395c740 10 client.15963 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.689+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b17c72f9e0 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.690+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016802d030 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f01600046c0 con 0x55b17c2d71b0 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 20 client.15963 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c293da0 0x55b17c731ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 4 client.15963 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.691+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b17c61b8a0 con 0x55b17c293da0 2024-09-21T01:51:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.694+0000 7f01767fc640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c293da0 0x55b17c731ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.704+0000 7f015e7fc640 10 client.15963 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.704+0000 7f01767fc640 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c293da0 0x55b17c731ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1186d76d1138a87 server_cookie=3df25a8b6c6451aa in_seq=0 out_seq=0 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.704+0000 7f015e7fc640 1 -- 172.21.15.29:0/1217449304 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b17c61b8a0 con 0x55b17c293da0 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.704+0000 7f015e7fc640 10 client.15963 handle_command_reply: tid=0 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.704+0000 7f018395c740 1 client.15963 shutdown 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 msgr2=0x55b17c6b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0160000c00 tx=0x7f0160005900 comp rx=0 tx=0).stop 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 shutdown_connections 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b17c293da0 0x55b17c731ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b17c300070 0x55b17c6b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b17c2df390 0x55b17c6b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 --2- 172.21.15.29:0/1217449304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17c2d71b0 0x55b17c6b6c90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 >> 172.21.15.29:0/1217449304 conn(0x55b17c2f4460 msgr2=0x55b17c6b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.705+0000 7f015e7fc640 0 client.15963 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:02.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.708+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 shutdown_connections 2024-09-21T01:51:02.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.708+0000 7f018395c740 1 -- 172.21.15.29:0/1217449304 wait complete. 2024-09-21T01:51:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.710+0000 7f018395c740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T01:51:02.711 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:02.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.712+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 msgr2=0x7f01640b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.712+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 0x7f01640b8a80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0178100c50 tx=0x7f0168009940 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.712+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 msgr2=0x7f0178198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.712+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f017000efc0 tx=0x7f017000c800 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 shutdown_connections 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f01640b6570 0x7f01640b8a80 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0178100c50 tx=0x7f0168009940 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0178108cf0 0x7f0178198660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01781083a0 0x7f01781980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 --2- 172.21.15.29:0/2004035864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0178102a60 0x7f017819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 >> 172.21.15.29:0/2004035864 conn(0x7f0178079730 msgr2=0x7f01781014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 shutdown_connections 2024-09-21T01:51:02.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:02.713+0000 7f017d3e8640 1 -- 172.21.15.29:0/2004035864 wait complete. 2024-09-21T01:51:02.969 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 145.0 s, 3826.6 tps, lat 26.143 ms stddev 29.384 2024-09-21T01:51:03.638 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:03 smithi029 ceph-mon[31228]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 MiB/s wr, 726 op/s 2024-09-21T01:51:03.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:03 smithi190 ceph-mon[34368]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 MiB/s wr, 726 op/s 2024-09-21T01:51:03.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:03 smithi066 ceph-mon[35001]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 MiB/s wr, 726 op/s 2024-09-21T01:51:04.685 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:51:04.686 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:51:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:05 smithi190 ceph-mon[34368]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 744 op/s 2024-09-21T01:51:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:05 smithi066 ceph-mon[35001]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 744 op/s 2024-09-21T01:51:05.824 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:05 smithi029 ceph-mon[31228]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 744 op/s 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/3903271860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 msgr2=0x7f7bc41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/3903271860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc41087d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009900 tx=0x7f7bb402a410 comp rx=0 tx=0).stop 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/3903271860 shutdown_connections 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/3903271860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 0x7f7bc410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/3903271860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc41087d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/3903271860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/3903271860 >> 172.21.15.29:0/3903271860 conn(0x7f7bc4079740 msgr2=0x7f7bc4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/3903271860 shutdown_connections 2024-09-21T01:51:05.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/3903271860 wait complete. 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.931+0000 7f7bcc1cf640 1 Processor -- start 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 -- start start 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 0x7f7bc4100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4112980 con 0x7f7bc41083c0 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4112800 con 0x7f7bc4102a80 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bcc1cf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4112b00 con 0x7f7bc4108d10 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bca76c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 0x7f7bc4100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38294/0 (socket says 172.21.15.29:38294) 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35446/0 (socket says 172.21.15.29:35446) 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 -- 172.21.15.29:0/264764688 learned_addr learned my addr 172.21.15.29:0/264764688 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 -- 172.21.15.29:0/264764688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 msgr2=0x7f7bc4100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 0x7f7bc4100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 -- 172.21.15.29:0/264764688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 msgr2=0x7f7bc40ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc976a640 1 -- 172.21.15.29:0/264764688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc41a8740 con 0x7f7bc41083c0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.932+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bc976a640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4006010 tx=0x7f7bb4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bbaffd640 1 -- 172.21.15.29:0/264764688 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb403f030 con 0x7f7bc41083c0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc41a89d0 con 0x7f7bc41083c0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bbaffd640 1 -- 172.21.15.29:0/264764688 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb4003aa0 con 0x7f7bc41083c0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bbaffd640 1 -- 172.21.15.29:0/264764688 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4035070 con 0x7f7bc41083c0 2024-09-21T01:51:05.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.933+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41a8f00 con 0x7f7bc41083c0 2024-09-21T01:51:05.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 Processor -- start 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bbaffd640 1 -- 172.21.15.29:0/264764688 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7bb4042030 con 0x7f7bc41083c0 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 -- start start 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13a8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e13a05e60 con 0x563e136c7670 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563e139f5290 con 0x563e136e8350 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bd2743740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563e139f90a0 con 0x563e136bf490 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.936+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34300/0 (socket says 172.21.15.29:34300) 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/773889603 learned_addr learned my addr 172.21.15.29:0/773889603 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bca76c640 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13a8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc976a640 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/773889603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 msgr2=0x563e13a8f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13a8f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/773889603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 msgr2=0x563e13a8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/773889603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e13753930 con 0x563e136bf490 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bbaffd640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 0x7f7bb00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00114f0 tx=0x7f7bc00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176b958ed2a1c203 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bca76c640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 0x7f7bb00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7baffff640 1 -- 172.21.15.29:0/773889603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc001e070 con 0x563e136bf490 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.937+0000 7f7bbaffd640 1 -- 172.21.15.29:0/264764688 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7bb4107350 con 0x7f7bc41083c0 2024-09-21T01:51:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.938+0000 7f7baffff640 1 -- 172.21.15.29:0/773889603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc00035f0 con 0x563e136bf490 2024-09-21T01:51:05.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.938+0000 7f7baffff640 1 -- 172.21.15.29:0/773889603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000f800 con 0x563e136bf490 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.938+0000 7f7bd2743740 1 -- 172.21.15.29:0/773889603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 msgr2=0x563e13a8f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.938+0000 7f7bd2743740 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00114f0 tx=0x7f7bc00119b0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.938+0000 7f7bca76c640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 0x7f7bb00b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005fd0 tx=0x7f7bbc009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 -- 172.21.15.29:0/773889603 shutdown_connections 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136bf490 0x563e13a8f970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 --2- 172.21.15.29:0/773889603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13a8f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 -- 172.21.15.29:0/773889603 >> 172.21.15.29:0/773889603 conn(0x563e136dc740 msgr2=0x563e13a8d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 -- 172.21.15.29:0/773889603 shutdown_connections 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 -- 172.21.15.29:0/773889603 wait complete. 2024-09-21T01:51:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 -- start start 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.939+0000 7f7bd2743740 1 Processor -- start 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 0x563e13a9b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13aa6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e13856890 con 0x563e136bf490 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563e1398fd40 con 0x563e136e8350 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563e13a9dc10 con 0x563e136c7670 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bd2743740 10 client.0 _ll_register_callbacks cb 0x563e139f3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 0x563e13a9b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34310/0 (socket says 172.21.15.29:34310) 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bca76c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13aa6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc976a640 1 -- 172.21.15.29:0/38125264 learned_addr learned my addr 172.21.15.29:0/38125264 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/38125264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 msgr2=0x563e13a9b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 msgr2=0x563e13aa6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13aa6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/38125264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e13aa7540 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bc976a640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.940+0000 7f7bca76c640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13aa6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 0x563e13a9b040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0011eb0 tx=0x7f7bc0003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e6d4e36bab5f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc001e070 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7bc0004230 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc00035f0 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7bc0004430 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bd2743740 10 client.15975 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e13aa9e70 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc00035f0 con 0x563e136bf490 2024-09-21T01:51:05.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.941+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7bc0004b00 con 0x563e136bf490 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 20 client.15975 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e136a6cd0 0x563e13aad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 4 client.15975 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e13a07db0 con 0x563e136a6cd0 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bc976a640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e136a6cd0 0x563e13aad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bad7fa640 10 client.15975 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.942+0000 7f7bc976a640 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e136a6cd0 0x563e13aad280 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eeff701dcc42006 server_cookie=455ee2611b4660cb in_seq=0 out_seq=0 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bad7fa640 1 -- 172.21.15.29:0/38125264 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e13a07db0 con 0x563e136a6cd0 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bad7fa640 10 client.15975 handle_command_reply: tid=0 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 client.15975 shutdown 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 msgr2=0x563e13a9b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 0x563e13a9b040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0011eb0 tx=0x7f7bc0003d90 comp rx=0 tx=0).stop 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 shutdown_connections 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e136a6cd0 0x563e13aad280 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136e8350 0x563e13aa6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136c7670 0x563e13a9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 --2- 172.21.15.29:0/38125264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136bf490 0x563e13a9b040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bad7fa640 0 client.15975 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 >> 172.21.15.29:0/38125264 conn(0x563e136dc740 msgr2=0x563e13a925c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 shutdown_connections 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 1 -- 172.21.15.29:0/38125264 wait complete. 2024-09-21T01:51:05.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.943+0000 7f7bd2743740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 Processor -- start 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 -- start start 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13a965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a95550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13aa6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b1580 con 0x563e136c7670 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b31b0 con 0x563e136bf490 2024-09-21T01:51:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bd2743740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b1c50 con 0x563e136e8350 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a95550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bca76c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13aa6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bc9f6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13a965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a95550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38308/0 (socket says 172.21.15.29:38308) 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.961+0000 7f7bc976a640 1 -- 172.21.15.29:0/3886980177 learned_addr learned my addr 172.21.15.29:0/3886980177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/3886980177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 msgr2=0x563e13aa6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13aa6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/3886980177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 msgr2=0x563e13a95550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a95550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 -- 172.21.15.29:0/3886980177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e13aa7370 con 0x563e136bf490 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13a965f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0004ed0 tx=0x7f7bc00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba727f00fc1905ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7baf7fe640 1 -- 172.21.15.29:0/3886980177 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc001e040 con 0x563e136bf490 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7baf7fe640 1 -- 172.21.15.29:0/3886980177 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc0028070 con 0x563e136bf490 2024-09-21T01:51:05.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.962+0000 7f7baf7fe640 1 -- 172.21.15.29:0/3886980177 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000fe00 con 0x563e136bf490 2024-09-21T01:51:05.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- 172.21.15.29:0/3886980177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 msgr2=0x563e13a965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13a965f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0004ed0 tx=0x7f7bc00119b0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- 172.21.15.29:0/3886980177 shutdown_connections 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13aa6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a95550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3886980177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13a965f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- 172.21.15.29:0/3886980177 >> 172.21.15.29:0/3886980177 conn(0x563e136dc740 msgr2=0x563e13a90e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- 172.21.15.29:0/3886980177 shutdown_connections 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- 172.21.15.29:0/3886980177 wait complete. 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.963+0000 7f7bd2743740 1 -- start start 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 Processor -- start 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13aa3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b2a20 con 0x563e136c7670 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b2350 con 0x563e136bf490 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41af560 con 0x563e136e8350 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bd2743740 10 client.0 _ll_register_callbacks cb 0x7f7bc001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bca76c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bc976a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38312/0 (socket says 172.21.15.29:38312) 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bca76c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34326/0 (socket says 172.21.15.29:34326) 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bc976a640 1 -- 172.21.15.29:0/3690568824 learned_addr learned my addr 172.21.15.29:0/3690568824 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.964+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13aa3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 -- 172.21.15.29:0/3690568824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 msgr2=0x563e13a9f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 -- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 msgr2=0x563e13aa3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:05.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13aa3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 -- 172.21.15.29:0/3690568824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e13a9f800 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bca76c640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bc976a640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4031440 tx=0x7f7bb4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e2999f961ec2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4049040 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7bb40d6a80 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc0028070 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7bb40ccef0 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bd2743740 10 client.15981 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.965+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e13b17c10 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc0028070 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7bb403bb40 con 0x563e136c7670 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 10 client.15981 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 20 client.15981 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e1367c080 0x563e13b1a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 4 client.15981 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e13a03860 con 0x563e1367c080 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.966+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e1367c080 0x563e13b1a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.967+0000 7f7bad7fa640 10 client.15981 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:05.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.967+0000 7f7bc9f6b640 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e1367c080 0x563e13b1a440 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588e85bb07a2ba7d server_cookie=36b81f09091b4f07 in_seq=0 out_seq=0 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bad7fa640 1 -- 172.21.15.29:0/3690568824 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e13a03860 con 0x563e1367c080 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bad7fa640 10 client.15981 handle_command_reply: tid=0 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 client.15981 shutdown 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 msgr2=0x563e13a9eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4031440 tx=0x7f7bb4009260 comp rx=0 tx=0).stop 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 shutdown_connections 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563e1367c080 0x563e13b1a440 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563e136e8350 0x563e13a9f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e136c7670 0x563e13a9eb30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 --2- 172.21.15.29:0/3690568824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563e136bf490 0x563e13aa3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.968+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 >> 172.21.15.29:0/3690568824 conn(0x563e136dc740 msgr2=0x563e13a90bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bad7fa640 0 client.15981 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 shutdown_connections 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bd2743740 1 -- 172.21.15.29:0/3690568824 wait complete. 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bd2743740 20 client.15981 trim_cache size 0 max 16384 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:51:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "fd9b75aa-5267-413c-8378-05fbd6a25987", 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 msgr2=0x7f7bb00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 0x7f7bb00b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005fd0 tx=0x7f7bbc009600 comp rx=0 tx=0).stop 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 msgr2=0x7f7bc40ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4006010 tx=0x7f7bb4004060 comp rx=0 tx=0).stop 2024-09-21T01:51:05.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 shutdown_connections 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7bb00b6510 0x7f7bb00b8a20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bc4108d10 0x7f7bc4100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc41083c0 0x7f7bc40ffbd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 --2- 172.21.15.29:0/264764688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc4102a80 0x7f7bc40ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.969+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 >> 172.21.15.29:0/264764688 conn(0x7f7bc4079740 msgr2=0x7f7bc406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.970+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 shutdown_connections 2024-09-21T01:51:05.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:05.970+0000 7f7bcc1cf640 1 -- 172.21.15.29:0/264764688 wait complete. 2024-09-21T01:51:06.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:06.596 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:06 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:06.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:06 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:06.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:06 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:07.350 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:51:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:07 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:07.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:07 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:07.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:07 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:07.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:07 smithi029 ceph-mon[31228]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 45 MiB/s wr, 1.04k op/s 2024-09-21T01:51:07.610 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:07 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:07 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:07 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:07 smithi190 ceph-mon[34368]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 45 MiB/s wr, 1.04k op/s 2024-09-21T01:51:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:07 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:07 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:07 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:07 smithi066 ceph-mon[35001]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 45 MiB/s wr, 1.04k op/s 2024-09-21T01:51:07.969 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 150.0 s, 1194.5 tps, lat 37.514 ms stddev 79.185 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 -- 172.21.15.29:0/3627032900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b81062d0 msgr2=0x7f12b81066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/3627032900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b81062d0 0x7f12b81066e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009900 tx=0x7f12ac02a410 comp rx=0 tx=0).stop 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 -- 172.21.15.29:0/3627032900 shutdown_connections 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/3627032900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b8106c20 0x7f12b81106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/3627032900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b81062d0 0x7f12b81066e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/3627032900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 -- 172.21.15.29:0/3627032900 >> 172.21.15.29:0/3627032900 conn(0x7f12b8079f90 msgr2=0x7f12b807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 -- 172.21.15.29:0/3627032900 shutdown_connections 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.127+0000 7f12bf4e9640 1 -- 172.21.15.29:0/3627032900 wait complete. 2024-09-21T01:51:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 Processor -- start 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 -- start start 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8106c20 0x7f12b819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113af0 con 0x7f12b8106c20 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113970 con 0x7f12b8102080 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bf4e9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113c70 con 0x7f12b81062d0 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bca84640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bd285640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bca84640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34338/0 (socket says 172.21.15.29:34338) 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8106c20 0x7f12b819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.128+0000 7f12bca84640 1 -- 172.21.15.29:0/4185922092 learned_addr learned my addr 172.21.15.29:0/4185922092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 -- 172.21.15.29:0/4185922092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 msgr2=0x7f12b819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 -- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8106c20 msgr2=0x7f12b819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8106c20 0x7f12b819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 -- 172.21.15.29:0/4185922092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b81a04d0 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bca84640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bd285640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b819a560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12b400e9c0 tx=0x7f12b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12aa7fc640 1 -- 172.21.15.29:0/4185922092 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b400cd80 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12aa7fc640 1 -- 172.21.15.29:0/4185922092 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b40108e0 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b81a07c0 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12aa7fc640 1 -- 172.21.15.29:0/4185922092 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4018900 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.129+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a6f20 con 0x7f12b8102080 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.132+0000 7f12c5a5d740 1 Processor -- start 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.132+0000 7f12c5a5d740 1 -- start start 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d16eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56113d0e4e50 con 0x56113cdc7070 2024-09-21T01:51:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56113d0d4280 con 0x56113cda6390 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56113d0d8090 con 0x56113cd9e1b0 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12aa7fc640 1 -- 172.21.15.29:0/4185922092 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f12b401f020 con 0x7f12b8102080 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38324/0 (socket says 172.21.15.29:38324) 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 -- 172.21.15.29:0/1965140869 learned_addr learned my addr 172.21.15.29:0/1965140869 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bca84640 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bd285640 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d16eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 -- 172.21.15.29:0/1965140869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 msgr2=0x56113d16eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d16eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 -- 172.21.15.29:0/1965140869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 msgr2=0x56113d16f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.133+0000 7f12bda86640 1 -- 172.21.15.29:0/1965140869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113ce32650 con 0x56113cdc7070 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12bda86640 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009f50 tx=0x7f12b00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911641d254201eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12aa7fc640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 0x7f12a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12a37fe640 1 -- 172.21.15.29:0/1965140869 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004d20 con 0x56113cdc7070 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12a37fe640 1 -- 172.21.15.29:0/1965140869 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b0004050 con 0x56113cdc7070 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12a37fe640 1 -- 172.21.15.29:0/1965140869 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0005280 con 0x56113cdc7070 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12bd285640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 0x7f12a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.134+0000 7f12aa7fc640 1 -- 172.21.15.29:0/4185922092 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12b40e7270 con 0x7f12b8102080 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1965140869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 msgr2=0x56113d16e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12bd285640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 0x7f12a40b8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56113d16d2d0 tx=0x7f12b40ead50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009f50 tx=0x7f12b00118d0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1965140869 shutdown_connections 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d16eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1965140869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cdc7070 0x56113d16e650 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1965140869 >> 172.21.15.29:0/1965140869 conn(0x56113cdbb460 msgr2=0x56113d16c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1965140869 shutdown_connections 2024-09-21T01:51:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1965140869 wait complete. 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.135+0000 7f12c5a5d740 1 -- start start 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 Processor -- start 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d179d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cdc7070 0x56113d186120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56113cf35700 con 0x56113cd9e1b0 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56113d06f110 con 0x56113cdc7070 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56113d17ce30 con 0x56113cda6390 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12c5a5d740 10 client.0 _ll_register_callbacks cb 0x56113d088990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d179d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d179d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38338/0 (socket says 172.21.15.29:38338) 2024-09-21T01:51:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bda86640 1 -- 172.21.15.29:0/360163167 learned_addr learned my addr 172.21.15.29:0/360163167 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bca84640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bda86640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cdc7070 0x56113d186120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bca84640 1 -- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cdc7070 msgr2=0x56113d186120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.136+0000 7f12bca84640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cdc7070 0x56113d186120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12bca84640 1 -- 172.21.15.29:0/360163167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 msgr2=0x56113d179d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12bca84640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d179d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12bca84640 1 -- 172.21.15.29:0/360163167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113d09cc40 con 0x56113cda6390 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12bca84640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17a290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009230 tx=0x7f12ac0316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ecfa040a708043c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac035040 con 0x56113cda6390 2024-09-21T01:51:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12ac004450 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b0004050 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12ac005020 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12c5a5d740 10 client.25756 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56113d188ba0 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.137+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b0004050 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f12ac0035f0 con 0x56113cda6390 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 10 client.25756 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 20 client.25756 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cd859f0 0x56113d18c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 4 client.25756 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113d0e6da0 con 0x56113cd859f0 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12bda86640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cd859f0 0x56113d18c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12a0ff9640 10 client.25756 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.138+0000 7f12bda86640 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cd859f0 0x56113d18c100 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39a1f0b087303b8 server_cookie=706ba3df13fa3705 in_seq=0 out_seq=0 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12a0ff9640 1 -- 172.21.15.29:0/360163167 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56113d0e6da0 con 0x56113cd859f0 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12a0ff9640 10 client.25756 handle_command_reply: tid=0 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 client.25756 shutdown 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 msgr2=0x56113d17a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17a290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009230 tx=0x7f12ac0316d0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 shutdown_connections 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cd859f0 0x56113d18c100 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cdc7070 0x56113d186120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17a290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12a0ff9640 0 client.25756 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/360163167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d179d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 >> 172.21.15.29:0/360163167 conn(0x56113cdbb460 msgr2=0x56113d171800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 shutdown_connections 2024-09-21T01:51:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 1 -- 172.21.15.29:0/360163167 wait complete. 2024-09-21T01:51:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.139+0000 7f12c5a5d740 20 client.25756 trim_cache size 0 max 16384 2024-09-21T01:51:08.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 Processor -- start 2024-09-21T01:51:08.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 -- start start 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d186020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81af3d0 con 0x56113cd859f0 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b1000 con 0x56113cda6390 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81afaa0 con 0x56113cd9e1b0 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12bda86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38354/0 (socket says 172.21.15.29:38354) 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12bda86640 1 -- 172.21.15.29:0/617971361 learned_addr learned my addr 172.21.15.29:0/617971361 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12bca84640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d186020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.157+0000 7f12bda86640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 -- 172.21.15.29:0/617971361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 msgr2=0x56113d16be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 -- 172.21.15.29:0/617971361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 msgr2=0x56113d171e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 -- 172.21.15.29:0/617971361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113d174980 con 0x56113cd9e1b0 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bda86640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12bca84640 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d186020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006010 tx=0x7f12ac003df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31076c04f86d84b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12a2ffd640 1 -- 172.21.15.29:0/617971361 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac035030 con 0x56113cd9e1b0 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12a2ffd640 1 -- 172.21.15.29:0/617971361 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ac004250 con 0x56113cd9e1b0 2024-09-21T01:51:08.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.158+0000 7f12a2ffd640 1 -- 172.21.15.29:0/617971361 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac031c00 con 0x56113cd9e1b0 2024-09-21T01:51:08.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- 172.21.15.29:0/617971361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 msgr2=0x56113d186020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d186020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006010 tx=0x7f12ac003df0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- 172.21.15.29:0/617971361 shutdown_connections 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cda6390 0x56113d16be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cd9e1b0 0x56113d186020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/617971361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd859f0 0x56113d171e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- 172.21.15.29:0/617971361 >> 172.21.15.29:0/617971361 conn(0x56113cdbb460 msgr2=0x56113d171750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- 172.21.15.29:0/617971361 shutdown_connections 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- 172.21.15.29:0/617971361 wait complete. 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.159+0000 7f12c5a5d740 1 -- start start 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 Processor -- start 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cd859f0 0x56113d17e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b0870 con 0x56113cd9e1b0 2024-09-21T01:51:08.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b01a0 con 0x56113cd859f0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ad3b0 con 0x56113cda6390 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12c5a5d740 10 client.0 _ll_register_callbacks cb 0x7f12ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12bca84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12bca84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38368/0 (socket says 172.21.15.29:38368) 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12bca84640 1 -- 172.21.15.29:0/1295372683 learned_addr learned my addr 172.21.15.29:0/1295372683 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12bda86640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.160+0000 7f12bda86640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cd859f0 0x56113d17e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 -- 172.21.15.29:0/1295372683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 msgr2=0x56113d17ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 -- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cd859f0 msgr2=0x56113d17e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cd859f0 0x56113d17e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 -- 172.21.15.29:0/1295372683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113d1f5bc0 con 0x56113cd9e1b0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12bca84640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005c90 tx=0x7f12ac005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12b775a14a6e4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:08.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac05e070 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12ac0051c0 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ac004250 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12ac06a050 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12c5a5d740 10 client.15999 fetch_fsmap learned FSMap version 31 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.161+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56113d1f6e50 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ac004250 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f12ac03d450 con 0x56113cd9e1b0 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 20 client.15999 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cdc7070 0x56113d1f9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:08.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 4 client.15999 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:51:08.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56113d0e2850 con 0x56113cdc7070 2024-09-21T01:51:08.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12bda86640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cdc7070 0x56113d1f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:08.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12a0ff9640 10 client.15999 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:08.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.162+0000 7f12bda86640 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cdc7070 0x56113d1f9550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1ce95861c790cc server_cookie=a87df8bdaab5994c in_seq=0 out_seq=0 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12a0ff9640 1 -- 172.21.15.29:0/1295372683 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1428 (crc 0 0 0) 0x56113d0e2850 con 0x56113cdc7070 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12a0ff9640 10 client.15999 handle_command_reply: tid=0 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 client.15999 shutdown 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 msgr2=0x56113d17e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005c90 tx=0x7f12ac005cc0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 shutdown_connections 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56113cdc7070 0x56113d1f9550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56113cda6390 0x56113d17ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56113cd9e1b0 0x56113d17e690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 --2- 172.21.15.29:0/1295372683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113cd859f0 0x56113d17e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12a0ff9640 0 client.15999 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 >> 172.21.15.29:0/1295372683 conn(0x56113cdbb460 msgr2=0x56113d179b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 shutdown_connections 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 1 -- 172.21.15.29:0/1295372683 wait complete. 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.163+0000 7f12c5a5d740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "scrub active (6 inodes in the stack)", 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "fd9b75aa-5267-413c-8378-05fbd6a25987", 2024-09-21T01:51:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:51:08.171 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 msgr2=0x7f12a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 0x7f12a40b8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56113d16d2d0 tx=0x7f12b40ead50 comp rx=0 tx=0).stop 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 msgr2=0x7f12b819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b819a560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12b400e9c0 tx=0x7f12b400ee80 comp rx=0 tx=0).stop 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 shutdown_connections 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12a40b6510 0x7f12a40b8a20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8106c20 0x7f12b819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b81062d0 0x7f12b819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 --2- 172.21.15.29:0/4185922092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b8102080 0x7f12b819a560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:08.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 >> 172.21.15.29:0/4185922092 conn(0x7f12b8079f90 msgr2=0x7f12b8107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:08.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 shutdown_connections 2024-09-21T01:51:08.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:08.164+0000 7f12bf4e9640 1 -- 172.21.15.29:0/4185922092 wait complete. 2024-09-21T01:51:09.331 INFO:tasks.cephfs.filesystem:scrub status for tag:fd9b75aa-5267-413c-8378-05fbd6a25987 - {'path': '/', 'tag': 'fd9b75aa-5267-413c-8378-05fbd6a25987', 'options': 'recursive,force'} 2024-09-21T01:51:09.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:09 smithi029 ceph-mon[31228]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 807 op/s 2024-09-21T01:51:09.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:09 smithi190 ceph-mon[34368]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 807 op/s 2024-09-21T01:51:09.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:09 smithi066 ceph-mon[35001]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 807 op/s 2024-09-21T01:51:10.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:10 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:10.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:10 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:10.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:10 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:10.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:10 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:10.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:10 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:10.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:10 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 1.27k op/s 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 1.27k op/s 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 1.27k op/s 2024-09-21T01:51:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:11.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:12.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 155.0 s, 2033.7 tps, lat 76.087 ms stddev 511.798 2024-09-21T01:51:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:13 smithi190 ceph-mon[34368]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 MiB/s wr, 1.14k op/s 2024-09-21T01:51:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:13 smithi066 ceph-mon[35001]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 MiB/s wr, 1.14k op/s 2024-09-21T01:51:13.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:13 smithi029 ceph-mon[31228]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 MiB/s wr, 1.14k op/s 2024-09-21T01:51:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.532+0000 7fa3f159a640 1 -- 172.21.15.29:0/4022692550 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8030850 con 0x7fa3f4103620 2024-09-21T01:51:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.533+0000 7fa3f8f20640 1 -- 172.21.15.29:0/4022692550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 msgr2=0x7fa3f4103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.533+0000 7fa3f8f20640 1 --2- 172.21.15.29:0/4022692550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f4103a30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009860 tx=0x7fa3e802a340 comp rx=0 tx=0).stop 2024-09-21T01:51:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 -- 172.21.15.29:0/4022692550 shutdown_connections 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 --2- 172.21.15.29:0/4022692550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3f4103fc0 0x7fa3f410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 --2- 172.21.15.29:0/4022692550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f4103a30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 --2- 172.21.15.29:0/4022692550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 -- 172.21.15.29:0/4022692550 >> 172.21.15.29:0/4022692550 conn(0x7fa3f40fdee0 msgr2=0x7fa3f4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 -- 172.21.15.29:0/4022692550 shutdown_connections 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.534+0000 7fa3f8f20640 1 -- 172.21.15.29:0/4022692550 wait complete. 2024-09-21T01:51:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.535+0000 7fa3f8f20640 1 Processor -- start 2024-09-21T01:51:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 -- start start 2024-09-21T01:51:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3f4103fc0 0x7fa3f419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f406b630 con 0x7fa3f4103fc0 2024-09-21T01:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.536+0000 7fa3f8f20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f406b4b0 con 0x7fa3f41025a0 2024-09-21T01:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f8f20640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f406b7b0 con 0x7fa3f4103620 2024-09-21T01:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48972/0 (socket says 172.21.15.29:48972) 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f259c640 1 -- 172.21.15.29:0/2817259148 learned_addr learned my addr 172.21.15.29:0/2817259148 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/2817259148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 msgr2=0x7fa3f419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3f4103fc0 msgr2=0x7fa3f419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3f4103fc0 0x7fa3f419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.537+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/2817259148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f419dbb0 con 0x7fa3f4103620 2024-09-21T01:51:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.538+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f419c7e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3e802a860 tx=0x7fa3e8032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.540+0000 7fa3e37fe640 1 -- 172.21.15.29:0/2817259148 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8040020 con 0x7fa3f4103620 2024-09-21T01:51:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.540+0000 7fa3e37fe640 1 -- 172.21.15.29:0/2817259148 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e8030810 con 0x7fa3f4103620 2024-09-21T01:51:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.540+0000 7fa3f8f20640 1 -- 172.21.15.29:0/2817259148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f419de40 con 0x7fa3f4103620 2024-09-21T01:51:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.540+0000 7fa3f8f20640 1 -- 172.21.15.29:0/2817259148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a43b0 con 0x7fa3f4103620 2024-09-21T01:51:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.540+0000 7fa3e37fe640 1 -- 172.21.15.29:0/2817259148 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037040 con 0x7fa3f4103620 2024-09-21T01:51:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.549+0000 7fa3ffe90740 1 Processor -- start 2024-09-21T01:51:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.550+0000 7fa3e37fe640 1 -- 172.21.15.29:0/2817259148 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa3e8043020 con 0x7fa3f4103620 2024-09-21T01:51:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.551+0000 7fa3e37fe640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 0x7fa3dc0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.551+0000 7fa3e37fe640 1 -- 172.21.15.29:0/2817259148 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa3e8107300 con 0x7fa3f4103620 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 -- start start 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff9958cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff9959820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fff98c26f0 con 0x55fff95874d0 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fff98b1b20 con 0x55fff957f2f0 2024-09-21T01:51:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.552+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fff98b5930 con 0x55fff95a81b0 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44630/0 (socket says 172.21.15.29:44630) 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 -- 172.21.15.29:0/429955289 learned_addr learned my addr 172.21.15.29:0/429955289 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 -- 172.21.15.29:0/429955289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 msgr2=0x55fff9958cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff9958cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 -- 172.21.15.29:0/429955289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 msgr2=0x55fff9959820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff9959820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.553+0000 7fa3f259c640 1 -- 172.21.15.29:0/429955289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff9877530 con 0x55fff95a81b0 2024-09-21T01:51:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.555+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 0x7fa3dc0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.558+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 0x7fa3dc0b8bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009310 tx=0x7fa3ec0092a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.558+0000 7fa3f259c640 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400b890 tx=0x7fa3e400bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a3bba2d85c6f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.560+0000 7fa3f8f20640 1 -- 172.21.15.29:0/429955289 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003dd0 con 0x55fff95a81b0 2024-09-21T01:51:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.560+0000 7fa3f8f20640 1 -- 172.21.15.29:0/429955289 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4015070 con 0x55fff95a81b0 2024-09-21T01:51:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.561+0000 7fa3f8f20640 1 -- 172.21.15.29:0/429955289 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40117c0 con 0x55fff95a81b0 2024-09-21T01:51:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 -- 172.21.15.29:0/429955289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 msgr2=0x55fff9958660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400b890 tx=0x7fa3e400bd50 comp rx=0 tx=0).stop 2024-09-21T01:51:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 -- 172.21.15.29:0/429955289 shutdown_connections 2024-09-21T01:51:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff9959820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff9958cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/429955289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95a81b0 0x55fff9958660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 -- 172.21.15.29:0/429955289 >> 172.21.15.29:0/429955289 conn(0x55fff959c300 msgr2=0x55fff99542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 -- 172.21.15.29:0/429955289 shutdown_connections 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.562+0000 7fa3ffe90740 1 -- 172.21.15.29:0/429955289 wait complete. 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 -- start start 2024-09-21T01:51:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 Processor -- start 2024-09-21T01:51:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff957f2f0 0x55fff994cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9952d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fff97e9ab0 con 0x55fff95874d0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fff9857730 con 0x55fff95a81b0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.563+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fff995a390 con 0x55fff957f2f0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3ffe90740 10 client.0 _ll_register_callbacks cb 0x55fff98ab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3f1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3f1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43182/0 (socket says 172.21.15.29:43182) 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/718714261 learned_addr learned my addr 172.21.15.29:0/718714261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9952d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.564+0000 7fa3f259c640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff957f2f0 0x55fff994cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.565+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/718714261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff957f2f0 msgr2=0x55fff994cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.565+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff957f2f0 0x55fff994cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.565+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 msgr2=0x55fff9952d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.565+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9952d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.565+0000 7fa3f1d9b640 1 -- 172.21.15.29:0/718714261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff9613790 con 0x55fff95874d0 2024-09-21T01:51:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.566+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80c9a10 tx=0x7fa3e8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dba3755622625f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.567+0000 7fa3d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.567+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80ccbb0 con 0x55fff95874d0 2024-09-21T01:51:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.567+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa3e80cc6a0 con 0x55fff95874d0 2024-09-21T01:51:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.567+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e4015070 con 0x55fff95874d0 2024-09-21T01:51:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.569+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa3e80ccf50 con 0x55fff95874d0 2024-09-21T01:51:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.569+0000 7fa3ffe90740 10 client.16005 fetch_fsmap learned FSMap version 32 2024-09-21T01:51:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.569+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fff968c620 con 0x55fff95874d0 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4015070 con 0x55fff95874d0 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa3e8030b10 con 0x55fff95874d0 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 20 client.16005 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff9566d80 0x55fff9978150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 4 client.16005 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fff98c4640 con 0x55fff9566d80 2024-09-21T01:51:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.570+0000 7fa3f259c640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff9566d80 0x55fff9978150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.571+0000 7fa3d9ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.572+0000 7fa3f259c640 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff9566d80 0x55fff9978150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65bcca53045a72e server_cookie=178ca03744a3105a in_seq=0 out_seq=0 2024-09-21T01:51:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.573+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/718714261 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fff98c4640 con 0x55fff9566d80 2024-09-21T01:51:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.573+0000 7fa3d9ffb640 10 client.16005 handle_command_reply: tid=0 2024-09-21T01:51:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.573+0000 7fa3ffe90740 1 client.16005 shutdown 2024-09-21T01:51:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 msgr2=0x55fff994d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80c9a10 tx=0x7fa3e8004470 comp rx=0 tx=0).stop 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 shutdown_connections 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff9566d80 0x55fff9978150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9952d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95874d0 0x55fff994d190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/718714261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff957f2f0 0x55fff994cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 >> 172.21.15.29:0/718714261 conn(0x55fff959c300 msgr2=0x55fff994a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3d9ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 shutdown_connections 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 1 -- 172.21.15.29:0/718714261 wait complete. 2024-09-21T01:51:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.574+0000 7fa3ffe90740 20 client.16005 trim_cache size 0 max 16384 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 Processor -- start 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 -- start start 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff995d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff99595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41a8dc0 con 0x55fff95a81b0 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41aaa20 con 0x55fff957f2f0 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3ffe90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41a9490 con 0x55fff95874d0 2024-09-21T01:51:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43194/0 (socket says 172.21.15.29:43194) 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 -- 172.21.15.29:0/987483744 learned_addr learned my addr 172.21.15.29:0/987483744 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f259c640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff995d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff99595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 -- 172.21.15.29:0/987483744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 msgr2=0x55fff99595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff99595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 -- 172.21.15.29:0/987483744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 msgr2=0x55fff995d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff995d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f2d9d640 1 -- 172.21.15.29:0/987483744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff9953100 con 0x55fff95a81b0 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff99595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.603+0000 7fa3f259c640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff995d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.604+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00f600 tx=0x7fa3ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d37baf05700e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.604+0000 7fa3dbfff640 1 -- 172.21.15.29:0/987483744 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec029070 con 0x55fff95a81b0 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.604+0000 7fa3dbfff640 1 -- 172.21.15.29:0/987483744 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ec01b8d0 con 0x55fff95a81b0 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.604+0000 7fa3dbfff640 1 -- 172.21.15.29:0/987483744 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec015070 con 0x55fff95a81b0 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 -- 172.21.15.29:0/987483744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 msgr2=0x55fff9953c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00f600 tx=0x7fa3ec011000 comp rx=0 tx=0).stop 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 -- 172.21.15.29:0/987483744 shutdown_connections 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff95a81b0 0x55fff9953c20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff99595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/987483744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff957f2f0 0x55fff995d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.606+0000 7fa3ffe90740 1 -- 172.21.15.29:0/987483744 >> 172.21.15.29:0/987483744 conn(0x55fff959c300 msgr2=0x55fff99780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.607+0000 7fa3ffe90740 1 -- 172.21.15.29:0/987483744 shutdown_connections 2024-09-21T01:51:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.607+0000 7fa3ffe90740 1 -- 172.21.15.29:0/987483744 wait complete. 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.607+0000 7fa3ffe90740 1 -- start start 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.607+0000 7fa3ffe90740 1 Processor -- start 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff9987850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43208/0 (socket says 172.21.15.29:43208) 2024-09-21T01:51:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 -- 172.21.15.29:0/3330607718 learned_addr learned my addr 172.21.15.29:0/3330607718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9988350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41aa290 con 0x55fff957f2f0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41a9b90 con 0x55fff95a81b0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41a6de0 con 0x55fff95874d0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3ffe90740 10 client.0 _ll_register_callbacks cb 0x7fa3e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff9987850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9988350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 -- 172.21.15.29:0/3330607718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 msgr2=0x55fff9987850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff9987850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 -- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 msgr2=0x55fff9988350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9988350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f259c640 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff9988ab0 con 0x55fff957f2f0 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.608+0000 7fa3f2d9d640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9988350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.609+0000 7fa3f259c640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40039c0 tx=0x7fa3e4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f4c9bdc3ffedcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.609+0000 7fa3d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.609+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e401ae60 con 0x55fff957f2f0 2024-09-21T01:51:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.609+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa3e4005040 con 0x55fff957f2f0 2024-09-21T01:51:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3ec01b8d0 con 0x55fff957f2f0 2024-09-21T01:51:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa3e4005240 con 0x55fff957f2f0 2024-09-21T01:51:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3ffe90740 10 client.16011 fetch_fsmap learned FSMap version 32 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fff9989e30 con 0x55fff957f2f0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e402f020 con 0x55fff957f2f0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa3e40045e0 con 0x55fff957f2f0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.610+0000 7fa3ffe90740 10 client.16011 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3ffe90740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3ffe90740 20 client.16011 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff953c1c0 0x55fff99e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3ffe90740 4 client.16011 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa3ec01b8d0 con 0x55fff953c1c0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.611+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff953c1c0 0x55fff99e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.612+0000 7fa3f1d9b640 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff953c1c0 0x55fff99e4ab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dddf25b3551281 server_cookie=11b9dc87bb93ab26 in_seq=0 out_seq=0 2024-09-21T01:51:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.612+0000 7fa3d9ffb640 10 client.16011 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.629+0000 7fa3d9ffb640 1 -- 172.21.15.29:0/3330607718 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa3ec01b8d0 con 0x55fff953c1c0 2024-09-21T01:51:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.629+0000 7fa3d9ffb640 10 client.16011 handle_command_reply: tid=0 2024-09-21T01:51:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.630+0000 7fa3ffe90740 1 client.16011 shutdown 2024-09-21T01:51:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.630+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 msgr2=0x55fff9987190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.630+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40039c0 tx=0x7fa3e4004430 comp rx=0 tx=0).stop 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 shutdown_connections 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fff953c1c0 0x55fff99e4ab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3d9ffb640 0 client.16011 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fff95a81b0 0x55fff9988350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fff95874d0 0x55fff9987850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 --2- 172.21.15.29:0/3330607718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fff957f2f0 0x55fff9987190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.631+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 >> 172.21.15.29:0/3330607718 conn(0x55fff959c300 msgr2=0x55fff99595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.632+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 shutdown_connections 2024-09-21T01:51:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.632+0000 7fa3ffe90740 1 -- 172.21.15.29:0/3330607718 wait complete. 2024-09-21T01:51:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.632+0000 7fa3ffe90740 20 client.16011 trim_cache size 0 max 16384 2024-09-21T01:51:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:51:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:51:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:51:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 msgr2=0x7fa3dc0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 0x7fa3dc0b8bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009310 tx=0x7fa3ec0092a0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 msgr2=0x7fa3f419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f419c7e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3e802a860 tx=0x7fa3e8032d30 comp rx=0 tx=0).stop 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 shutdown_connections 2024-09-21T01:51:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3dc0b66e0 0x7fa3dc0b8bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3f4103fc0 0x7fa3f419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3f4103620 0x7fa3f419c7e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 --2- 172.21.15.29:0/2817259148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f41025a0 0x7fa3f419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 >> 172.21.15.29:0/2817259148 conn(0x7fa3f40fdee0 msgr2=0x7fa3f410d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 shutdown_connections 2024-09-21T01:51:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:15.633+0000 7fa3e17fa640 1 -- 172.21.15.29:0/2817259148 wait complete. 2024-09-21T01:51:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:15 smithi190 ceph-mon[34368]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 50 MiB/s wr, 1.20k op/s 2024-09-21T01:51:15.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:15 smithi066 ceph-mon[35001]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 50 MiB/s wr, 1.20k op/s 2024-09-21T01:51:15.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:15 smithi029 ceph-mon[31228]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 50 MiB/s wr, 1.20k op/s 2024-09-21T01:51:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:17 smithi190 ceph-mon[34368]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 51 MiB/s wr, 1.36k op/s 2024-09-21T01:51:17.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:17 smithi066 ceph-mon[35001]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 51 MiB/s wr, 1.36k op/s 2024-09-21T01:51:17.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:17 smithi029 ceph-mon[31228]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 51 MiB/s wr, 1.36k op/s 2024-09-21T01:51:17.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 160.0 s, 3884.0 tps, lat 25.793 ms stddev 29.452 2024-09-21T01:51:19.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:19 smithi190 ceph-mon[34368]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 36 MiB/s wr, 983 op/s 2024-09-21T01:51:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:19 smithi066 ceph-mon[35001]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 36 MiB/s wr, 983 op/s 2024-09-21T01:51:19.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:19 smithi029 ceph-mon[31228]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 36 MiB/s wr, 983 op/s 2024-09-21T01:51:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:20 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:51:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:20 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:51:20.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:20 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:51:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:20 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:51:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:20 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:51:20.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:20 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:51:20.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:20 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:51:20.878 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:20 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:51:20.879 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:20 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:51:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:21 smithi190 ceph-mon[34368]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 41 MiB/s wr, 1.27k op/s 2024-09-21T01:51:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:51:21.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:21 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:51:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:21 smithi066 ceph-mon[35001]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 41 MiB/s wr, 1.27k op/s 2024-09-21T01:51:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:51:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:21 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:51:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:21 smithi029 ceph-mon[31228]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 41 MiB/s wr, 1.27k op/s 2024-09-21T01:51:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:51:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:21 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:51:22.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 165.0 s, 3516.4 tps, lat 28.454 ms stddev 31.488 2024-09-21T01:51:23.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:23 smithi190 ceph-mon[34368]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 861 op/s 2024-09-21T01:51:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:23 smithi066 ceph-mon[35001]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 861 op/s 2024-09-21T01:51:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:23 smithi029 ceph-mon[31228]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 861 op/s 2024-09-21T01:51:25.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:25 smithi190 ceph-mon[34368]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-21T01:51:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:25 smithi066 ceph-mon[35001]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-21T01:51:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:25 smithi029 ceph-mon[31228]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-21T01:51:27.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:27 smithi190 ceph-mon[34368]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-09-21T01:51:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:27 smithi066 ceph-mon[35001]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-09-21T01:51:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:27 smithi029 ceph-mon[31228]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-09-21T01:51:27.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 170.0 s, 3784.4 tps, lat 26.436 ms stddev 28.623 2024-09-21T01:51:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/4019337366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4103500 msgr2=0x7fa3a4103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/4019337366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4103500 0x7fa3a4103970 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa398009900 tx=0x7fa39802a400 comp rx=0 tx=0).stop 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/4019337366 shutdown_connections 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/4019337366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4104f20 0x7fa3a410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/4019337366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 0x7fa3a4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/4019337366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4103500 0x7fa3a4103970 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/4019337366 >> 172.21.15.29:0/4019337366 conn(0x7fa3a40fee40 msgr2=0x7fa3a41012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/4019337366 shutdown_connections 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.795+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/4019337366 wait complete. 2024-09-21T01:51:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 Processor -- start 2024-09-21T01:51:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 -- start start 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4103500 0x7fa3a4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 0x7fa3a419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a410f180 con 0x7fa3a4104580 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a410f000 con 0x7fa3a4103500 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.796+0000 7fa3a9fd6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a410f300 con 0x7fa3a4104f20 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45344/0 (socket says 172.21.15.29:45344) 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 -- 172.21.15.29:0/3011037534 learned_addr learned my addr 172.21.15.29:0/3011037534 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4103500 0x7fa3a4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 0x7fa3a419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 -- 172.21.15.29:0/3011037534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4103500 msgr2=0x7fa3a4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4103500 0x7fa3a4199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 -- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 msgr2=0x7fa3a419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 0x7fa3a419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 -- 172.21.15.29:0/3011037534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a419b5d0 con 0x7fa3a4104f20 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.797+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009f60 tx=0x7fa39c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.798+0000 7fa3a1ffb640 1 -- 172.21.15.29:0/3011037534 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00cd80 con 0x7fa3a4104f20 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.798+0000 7fa3a1ffb640 1 -- 172.21.15.29:0/3011037534 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c013770 con 0x7fa3a4104f20 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.798+0000 7fa3a1ffb640 1 -- 172.21.15.29:0/3011037534 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c005380 con 0x7fa3a4104f20 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.798+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a419b8c0 con 0x7fa3a4104f20 2024-09-21T01:51:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.798+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a40757d0 con 0x7fa3a4104f20 2024-09-21T01:51:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.802+0000 7fa3b2284740 1 Processor -- start 2024-09-21T01:51:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.803+0000 7fa3a1ffb640 1 -- 172.21.15.29:0/3011037534 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa39c013920 con 0x7fa3a4104f20 2024-09-21T01:51:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.803+0000 7fa3b2284740 1 -- start start 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7e21b0 0x556eefb8a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556eefafc6f0 con 0x556eef7b92f0 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556eefaebb20 con 0x556eef7e21b0 2024-09-21T01:51:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3b2284740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556eefaef930 con 0x556eef7c14d0 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53222/0 (socket says 172.21.15.29:53222) 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/3912881950 learned_addr learned my addr 172.21.15.29:0/3912881950 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7e21b0 0x556eefb8a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 -- 172.21.15.29:0/3912881950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7e21b0 msgr2=0x556eefb8a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7e21b0 0x556eefb8a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 -- 172.21.15.29:0/3912881950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 msgr2=0x556eefb8a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a3fff640 1 -- 172.21.15.29:0/3912881950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eefab1530 con 0x556eef7c14d0 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.804+0000 7fa3a1ffb640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 0x7fa3940c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa3a1ffb640 1 -- 172.21.15.29:0/3011037534 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa39c0e76e0 con 0x7fa3a4104f20 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 0x7fa3940c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb8b200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa390014930 tx=0x7fa39000c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfc7e515ba054c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa38effd640 1 -- 172.21.15.29:0/3912881950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390004040 con 0x556eef7c14d0 2024-09-21T01:51:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.805+0000 7fa38effd640 1 -- 172.21.15.29:0/3912881950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39000fc10 con 0x556eef7c14d0 2024-09-21T01:51:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 -- 172.21.15.29:0/3912881950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 msgr2=0x556eefb8b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb8b200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa390014930 tx=0x7fa39000c430 comp rx=0 tx=0).stop 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 0x7fa3940c1330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa398005f70 tx=0x7fa39800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 -- 172.21.15.29:0/3912881950 shutdown_connections 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb8b200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7b92f0 0x556eefb8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3912881950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7e21b0 0x556eefb8a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 -- 172.21.15.29:0/3912881950 >> 172.21.15.29:0/3912881950 conn(0x556eef7d6300 msgr2=0x556eefb85ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 -- 172.21.15.29:0/3912881950 shutdown_connections 2024-09-21T01:51:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.807+0000 7fa3b2284740 1 -- 172.21.15.29:0/3912881950 wait complete. 2024-09-21T01:51:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 -- start start 2024-09-21T01:51:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 Processor -- start 2024-09-21T01:51:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb93ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556eefa23ab0 con 0x556eef7c14d0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556eefa91730 con 0x556eef7b92f0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556eefb8bdc0 con 0x556eef7e21b0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3b2284740 10 client.0 _ll_register_callbacks cb 0x556eefae56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.808+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48364/0 (socket says 172.21.15.29:48364) 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/3929751403 learned_addr learned my addr 172.21.15.29:0/3929751403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/3929751403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 msgr2=0x556eefb949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/3929751403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 msgr2=0x556eefb93ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb93ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/3929751403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eef84d790 con 0x556eef7b92f0 2024-09-21T01:51:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.809+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3980096c0 tx=0x7fa39802a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d922e4359ee49ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.810+0000 7fa383fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.810+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398035040 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.810+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa39803d9a0 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39000fc10 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa39803dba0 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556eef8c6620 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39000fc10 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa39804a420 con 0x556eef7b92f0 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 20 client.25739 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7a0d80 0x556eefba1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 4 client.25739 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.811+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eefafe640 con 0x556eef7a0d80 2024-09-21T01:51:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.812+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7a0d80 0x556eefba1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.812+0000 7fa383fff640 10 client.25739 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.812+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7a0d80 0x556eefba1080 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ddc99e464a2370 server_cookie=c9affabd5a30d2a6 in_seq=0 out_seq=0 2024-09-21T01:51:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa383fff640 1 -- 172.21.15.29:0/3929751403 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556eefafe640 con 0x556eef7a0d80 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa383fff640 10 client.25739 handle_command_reply: tid=0 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 client.25739 shutdown 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 msgr2=0x556eefb93820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3980096c0 tx=0x7fa39802a400 comp rx=0 tx=0).stop 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 shutdown_connections 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7a0d80 0x556eefba1080 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb93ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 --2- 172.21.15.29:0/3929751403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb93820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa383fff640 0 client.25739 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.813+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 >> 172.21.15.29:0/3929751403 conn(0x556eef7d6300 msgr2=0x556eefb866d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.814+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 shutdown_connections 2024-09-21T01:51:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.814+0000 7fa3b2284740 1 -- 172.21.15.29:0/3929751403 wait complete. 2024-09-21T01:51:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.814+0000 7fa3b2284740 20 client.25739 trim_cache size 0 max 16384 2024-09-21T01:51:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.839+0000 7fa3b2284740 1 Processor -- start 2024-09-21T01:51:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.839+0000 7fa3b2284740 1 -- start start 2024-09-21T01:51:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb92a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb85410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41a8be0 con 0x556eef7c14d0 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41aa840 con 0x556eef7b92f0 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3b2284740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41a92b0 con 0x556eef7e21b0 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53234/0 (socket says 172.21.15.29:53234) 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2791082855 learned_addr learned my addr 172.21.15.29:0/2791082855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb85410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb92a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2791082855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 msgr2=0x556eefb85410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb85410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2791082855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 msgr2=0x556eefb92a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb92a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2791082855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eefb8afa0 con 0x556eef7c14d0 2024-09-21T01:51:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.840+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb92a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.841+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa390014440 tx=0x7fa390014500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827fd81ca48d5f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.841+0000 7fa38e7fc640 1 -- 172.21.15.29:0/2791082855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000f730 con 0x556eef7c14d0 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.841+0000 7fa38e7fc640 1 -- 172.21.15.29:0/2791082855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa390024030 con 0x556eef7c14d0 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.841+0000 7fa38e7fc640 1 -- 172.21.15.29:0/2791082855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390012980 con 0x556eef7c14d0 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 -- 172.21.15.29:0/2791082855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 msgr2=0x556eefb89fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa390014440 tx=0x7fa390014500 comp rx=0 tx=0).stop 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 -- 172.21.15.29:0/2791082855 shutdown_connections 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7e21b0 0x556eefb85410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7c14d0 0x556eefb89fe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 --2- 172.21.15.29:0/2791082855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb92a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 -- 172.21.15.29:0/2791082855 >> 172.21.15.29:0/2791082855 conn(0x556eef7d6300 msgr2=0x556eefb80870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 -- 172.21.15.29:0/2791082855 shutdown_connections 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.843+0000 7fa3b2284740 1 -- 172.21.15.29:0/2791082855 wait complete. 2024-09-21T01:51:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.844+0000 7fa3b2284740 1 -- start start 2024-09-21T01:51:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.844+0000 7fa3b2284740 1 Processor -- start 2024-09-21T01:51:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.845+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7e21b0 0x556eefb99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a8fd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48380/0 (socket says 172.21.15.29:48380) 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb98a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45384/0 (socket says 172.21.15.29:45384) 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41aa0b0 con 0x556eef7e21b0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41a99b0 con 0x556eef7b92f0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41a6bc0 con 0x556eef7c14d0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3b2284740 10 client.0 _ll_register_callbacks cb 0x7fa3980326e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/889267752 learned_addr learned my addr 172.21.15.29:0/889267752 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.846+0000 7fa3a97d5640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7e21b0 0x556eefb99590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/889267752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 msgr2=0x556eefb98a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb98a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/889267752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7e21b0 msgr2=0x556eefb99590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7e21b0 0x556eefb99590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 -- 172.21.15.29:0/889267752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eefb99cf0 con 0x556eef7b92f0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa3a8fd4640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa398009230 tx=0x7fa39803df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237306759b8ea620 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.847+0000 7fa383fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39806a030 con 0x556eef7b92f0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa398061040 con 0x556eef7b92f0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa390024030 con 0x556eef7b92f0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa39803d7a0 con 0x556eef7b92f0 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa3b2284740 10 client.25745 fetch_fsmap learned FSMap version 32 2024-09-21T01:51:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.848+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556eefb9b080 con 0x556eef7b92f0 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa390024030 con 0x556eef7b92f0 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa398065050 con 0x556eef7b92f0 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 10 client.25745 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 20 client.25745 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7761c0 0x556eefc0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 4 client.25745 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:51:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eefafa0f0 con 0x556eef7761c0 2024-09-21T01:51:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.849+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7761c0 0x556eefc0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.851+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7761c0 0x556eefc0da00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3f0d510c0b523c server_cookie=bc1522deaef6ff94 in_seq=0 out_seq=0 2024-09-21T01:51:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.851+0000 7fa383fff640 10 client.25745 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.866+0000 7fa383fff640 1 -- 172.21.15.29:0/889267752 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556eefafa0f0 con 0x556eef7761c0 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.866+0000 7fa383fff640 10 client.25745 handle_command_reply: tid=0 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.866+0000 7fa3b2284740 1 client.25745 shutdown 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.866+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 msgr2=0x556eefb98360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.866+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa398009230 tx=0x7fa39803df70 comp rx=0 tx=0).stop 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 shutdown_connections 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556eef7761c0 0x556eefc0da00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556eef7e21b0 0x556eefb99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556eef7c14d0 0x556eefb98a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 --2- 172.21.15.29:0/889267752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556eef7b92f0 0x556eefb98360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 >> 172.21.15.29:0/889267752 conn(0x556eef7d6300 msgr2=0x556eefb92a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa383fff640 0 client.25745 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 shutdown_connections 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 1 -- 172.21.15.29:0/889267752 wait complete. 2024-09-21T01:51:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.867+0000 7fa3b2284740 20 client.25745 trim_cache size 0 max 16384 2024-09-21T01:51:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:51:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:51:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:51:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 msgr2=0x7fa3940c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 0x7fa3940c1330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa398005f70 tx=0x7fa39800b040 comp rx=0 tx=0).stop 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 msgr2=0x7fa3a419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009f60 tx=0x7fa39c009f90 comp rx=0 tx=0).stop 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 shutdown_connections 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa3940bee20 0x7fa3940c1330 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3a4104f20 0x7fa3a419adb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3a4104580 0x7fa3a419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 --2- 172.21.15.29:0/3011037534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a4103500 0x7fa3a4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 >> 172.21.15.29:0/3011037534 conn(0x7fa3a40fee40 msgr2=0x7fa3a41078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 shutdown_connections 2024-09-21T01:51:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:28.868+0000 7fa3a9fd6640 1 -- 172.21.15.29:0/3011037534 wait complete. 2024-09-21T01:51:29.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:29 smithi190 ceph-mon[34368]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-09-21T01:51:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:29 smithi066 ceph-mon[35001]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-09-21T01:51:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:29 smithi029 ceph-mon[31228]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-09-21T01:51:31.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:31 smithi066 ceph-mon[35001]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T01:51:31.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:51:31.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:31 smithi029 ceph-mon[31228]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T01:51:31.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:51:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:31 smithi190 ceph-mon[34368]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T01:51:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:51:32.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:51:32.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:51:32.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:51:32.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:51:32.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:51:32.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:51:32.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 175.0 s, 3812.0 tps, lat 26.229 ms stddev 30.203 2024-09-21T01:51:32.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:51:32.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:51:32.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:51:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:33 smithi190 ceph-mon[34368]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 696 op/s 2024-09-21T01:51:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:33 smithi066 ceph-mon[35001]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 696 op/s 2024-09-21T01:51:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:33 smithi029 ceph-mon[31228]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 696 op/s 2024-09-21T01:51:35.896 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:35 smithi029 ceph-mon[31228]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T01:51:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:35 smithi190 ceph-mon[34368]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T01:51:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:35 smithi066 ceph-mon[35001]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T01:51:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 180.0 s, 4339.9 tps, lat 22.939 ms stddev 26.427 2024-09-21T01:51:37.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:37 smithi190 ceph-mon[34368]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 855 op/s 2024-09-21T01:51:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:37 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:37 smithi066 ceph-mon[35001]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 855 op/s 2024-09-21T01:51:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:37 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:37 smithi029 ceph-mon[31228]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 855 op/s 2024-09-21T01:51:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:37 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:51:39.332 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:51:39.730 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:39 smithi190 ceph-mon[34368]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 646 op/s 2024-09-21T01:51:40.013 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:39 smithi029 ceph-mon[31228]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 646 op/s 2024-09-21T01:51:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:39 smithi066 ceph-mon[35001]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 646 op/s 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 -- 172.21.15.29:0/1400114898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0102a60 msgr2=0x7f48f0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 --2- 172.21.15.29:0/1400114898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0102a60 0x7f48f0102e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009900 tx=0x7f48e002a410 comp rx=0 tx=0).stop 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 -- 172.21.15.29:0/1400114898 shutdown_connections 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 --2- 172.21.15.29:0/1400114898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f0108cf0 0x7f48f010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 --2- 172.21.15.29:0/1400114898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f01083a0 0x7f48f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 --2- 172.21.15.29:0/1400114898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0102a60 0x7f48f0102e70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 -- 172.21.15.29:0/1400114898 >> 172.21.15.29:0/1400114898 conn(0x7f48f0079730 msgr2=0x7f48f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 -- 172.21.15.29:0/1400114898 shutdown_connections 2024-09-21T01:51:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.309+0000 7f48f544b640 1 -- 172.21.15.29:0/1400114898 wait complete. 2024-09-21T01:51:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.310+0000 7f48f544b640 1 Processor -- start 2024-09-21T01:51:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.310+0000 7f48f544b640 1 -- start start 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f0102a60 0x7f48f019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0108cf0 0x7f48f01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0111860 con 0x7f48f0108cf0 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01116e0 con 0x7f48f0102a60 2024-09-21T01:51:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48f544b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01119e0 con 0x7f48f01083a0 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48e67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48e67fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51606/0 (socket says 172.21.15.29:51606) 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48e67fc640 1 -- 172.21.15.29:0/1295458574 learned_addr learned my addr 172.21.15.29:0/1295458574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.311+0000 7f48ef7fe640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0108cf0 0x7f48f01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48eeffd640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f0102a60 0x7f48f019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48e67fc640 1 -- 172.21.15.29:0/1295458574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f0102a60 msgr2=0x7f48f019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f0102a60 0x7f48f019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48e67fc640 1 -- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0108cf0 msgr2=0x7f48f01a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0108cf0 0x7f48f01a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.312+0000 7f48e67fc640 1 -- 172.21.15.29:0/1295458574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f01a25d0 con 0x7f48f01083a0 2024-09-21T01:51:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48d800e9c0 tx=0x7f48d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48ecff9640 1 -- 172.21.15.29:0/1295458574 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d800cd80 con 0x7f48f01083a0 2024-09-21T01:51:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48ecff9640 1 -- 172.21.15.29:0/1295458574 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d80054d0 con 0x7f48f01083a0 2024-09-21T01:51:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f01a28c0 con 0x7f48f01083a0 2024-09-21T01:51:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a9020 con 0x7f48f01083a0 2024-09-21T01:51:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.313+0000 7f48ecff9640 1 -- 172.21.15.29:0/1295458574 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8010430 con 0x7f48f01083a0 2024-09-21T01:51:40.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 Processor -- start 2024-09-21T01:51:40.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 -- start start 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 0x55dd07f17270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b4f390 0x55dd07f17cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dd07e8de50 con 0x55dd07b70070 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dd07e7d280 con 0x55dd07b4f390 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd07e81090 con 0x55dd07b471b0 2024-09-21T01:51:40.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 0x55dd07f17270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 0x55dd07f17270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33708/0 (socket says 172.21.15.29:33708) 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.317+0000 7f48ef7fe640 1 -- 172.21.15.29:0/3905198479 learned_addr learned my addr 172.21.15.29:0/3905198479 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.318+0000 7f48eeffd640 1 -- 172.21.15.29:0/3905198479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b4f390 msgr2=0x55dd07f17cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.318+0000 7f48eeffd640 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b4f390 0x55dd07f17cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.318+0000 7f48eeffd640 1 -- 172.21.15.29:0/3905198479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 msgr2=0x55dd07f17270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.318+0000 7f48eeffd640 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 0x55dd07f17270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.318+0000 7f48eeffd640 1 -- 172.21.15.29:0/3905198479 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd07bdb650 con 0x55dd07b471b0 2024-09-21T01:51:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.319+0000 7f48ecff9640 1 -- 172.21.15.29:0/1295458574 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f48d80046c0 con 0x7f48f01083a0 2024-09-21T01:51:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.319+0000 7f48eeffd640 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f177b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005ab0 tx=0x7f48e0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51fe426d4f7e96db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.319+0000 7f48e57fa640 1 -- 172.21.15.29:0/3905198479 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0035040 con 0x55dd07b471b0 2024-09-21T01:51:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.319+0000 7f48e57fa640 1 -- 172.21.15.29:0/3905198479 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e00054f0 con 0x55dd07b471b0 2024-09-21T01:51:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.319+0000 7f48e57fa640 1 -- 172.21.15.29:0/3905198479 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0004cc0 con 0x55dd07b471b0 2024-09-21T01:51:40.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48ecff9640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 0x7f48dc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48ecff9640 1 -- 172.21.15.29:0/1295458574 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f48d80e7500 con 0x7f48f01083a0 2024-09-21T01:51:40.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48ef7fe640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 0x7f48dc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48fb9bf740 1 -- 172.21.15.29:0/3905198479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 msgr2=0x55dd07f177b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48ef7fe640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 0x7f48dc0b8b20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48e800bfd0 tx=0x7f48e800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.320+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f177b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005ab0 tx=0x7f48e0031400 comp rx=0 tx=0).stop 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 -- 172.21.15.29:0/3905198479 shutdown_connections 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b4f390 0x55dd07f17cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f177b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/3905198479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b70070 0x55dd07f17270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 -- 172.21.15.29:0/3905198479 >> 172.21.15.29:0/3905198479 conn(0x55dd07b64460 msgr2=0x55dd07f15160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 -- 172.21.15.29:0/3905198479 shutdown_connections 2024-09-21T01:51:40.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 -- 172.21.15.29:0/3905198479 wait complete. 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 -- start start 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.321+0000 7f48fb9bf740 1 Processor -- start 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.322+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.322+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f2b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.322+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.322+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f2b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51626/0 (socket says 172.21.15.29:51626) 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.322+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 0x55dd07f374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48eeffd640 1 -- 172.21.15.29:0/313788090 learned_addr learned my addr 172.21.15.29:0/313788090 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dd07cde700 con 0x55dd07b4f390 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48ef7fe640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 0x55dd07f374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dd07e18110 con 0x55dd07b70070 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd07f2e120 con 0x55dd07b471b0 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48fb9bf740 10 client.0 _ll_register_callbacks cb 0x55dd07e31990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.323+0000 7f48e67fc640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f2b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 -- 172.21.15.29:0/313788090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 msgr2=0x55dd07f2b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f2b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 -- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 msgr2=0x55dd07f374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 0x55dd07f374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd07e45c40 con 0x55dd07b4f390 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48ef7fe640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 0x55dd07f374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48e67fc640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f2b630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c6a0 tx=0x7f48d80ad050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e4bb08cdd5b9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80b6240 con 0x55dd07b4f390 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f48d8004fb0 con 0x55dd07b4f390 2024-09-21T01:51:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.324+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e00054f0 con 0x55dd07b4f390 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48d80e72f0 con 0x55dd07b4f390 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 10 client.16038 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dd07f39ef0 con 0x55dd07b4f390 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e00054f0 con 0x55dd07b4f390 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f48d80051b0 con 0x55dd07b4f390 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 10 client.16038 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 20 client.16038 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b2e9f0 0x55dd07f3d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 4 client.16038 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.325+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd07e8fda0 con 0x55dd07b2e9f0 2024-09-21T01:51:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.326+0000 7f48eeffd640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b2e9f0 0x55dd07f3d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.326+0000 7f48d67fc640 10 client.16038 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.326+0000 7f48eeffd640 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b2e9f0 0x55dd07f3d510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d2148ffd414d2f server_cookie=3fbdd4c790c29e7f in_seq=0 out_seq=0 2024-09-21T01:51:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.327+0000 7f48d67fc640 1 -- 172.21.15.29:0/313788090 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd07e8fda0 con 0x55dd07b2e9f0 2024-09-21T01:51:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.327+0000 7f48d67fc640 10 client.16038 handle_command_reply: tid=0 2024-09-21T01:51:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.327+0000 7f48fb9bf740 1 client.16038 shutdown 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.327+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 msgr2=0x55dd07f2b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.327+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f2b630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c6a0 tx=0x7f48d80ad050 comp rx=0 tx=0).stop 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 shutdown_connections 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b2e9f0 0x55dd07f3d510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48d67fc640 0 client.16038 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b70070 0x55dd07f374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f2b630 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/313788090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b471b0 0x55dd07f2b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 >> 172.21.15.29:0/313788090 conn(0x55dd07b64460 msgr2=0x55dd07f13700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 shutdown_connections 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 1 -- 172.21.15.29:0/313788090 wait complete. 2024-09-21T01:51:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.328+0000 7f48fb9bf740 20 client.16038 trim_cache size 0 max 16384 2024-09-21T01:51:40.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.356+0000 7f48fb9bf740 1 Processor -- start 2024-09-21T01:51:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.356+0000 7f48fb9bf740 1 -- start start 2024-09-21T01:51:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b471b0 0x55dd07f373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b4f390 0x55dd07f378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000e3c0 con 0x55dd07b471b0 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000fff0 con 0x55dd07b2e9f0 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48fb9bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000ea90 con 0x55dd07b4f390 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40824/0 (socket says 172.21.15.29:40824) 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48eeffd640 1 -- 172.21.15.29:0/2619247194 learned_addr learned my addr 172.21.15.29:0/2619247194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.357+0000 7f48e67fc640 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b471b0 0x55dd07f373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 -- 172.21.15.29:0/2619247194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b4f390 msgr2=0x55dd07f378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b4f390 0x55dd07f378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 -- 172.21.15.29:0/2619247194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b471b0 msgr2=0x55dd07f373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b471b0 0x55dd07f373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 -- 172.21.15.29:0/2619247194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd07f156a0 con 0x55dd07b2e9f0 2024-09-21T01:51:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.358+0000 7f48eeffd640 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009900 tx=0x7f48e0004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a046f745250e7a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.359+0000 7f48e4ff9640 1 -- 172.21.15.29:0/2619247194 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0035040 con 0x55dd07b2e9f0 2024-09-21T01:51:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.359+0000 7f48e4ff9640 1 -- 172.21.15.29:0/2619247194 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e0031c80 con 0x55dd07b2e9f0 2024-09-21T01:51:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.359+0000 7f48e4ff9640 1 -- 172.21.15.29:0/2619247194 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0003a80 con 0x55dd07b2e9f0 2024-09-21T01:51:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 -- 172.21.15.29:0/2619247194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 msgr2=0x55dd07f2b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009900 tx=0x7f48e0004960 comp rx=0 tx=0).stop 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 -- 172.21.15.29:0/2619247194 shutdown_connections 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b4f390 0x55dd07f378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b471b0 0x55dd07f373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/2619247194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2b5b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.360+0000 7f48fb9bf740 1 -- 172.21.15.29:0/2619247194 >> 172.21.15.29:0/2619247194 conn(0x55dd07b64460 msgr2=0x55dd07f2aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.361+0000 7f48fb9bf740 1 -- 172.21.15.29:0/2619247194 shutdown_connections 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.361+0000 7f48fb9bf740 1 -- 172.21.15.29:0/2619247194 wait complete. 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.361+0000 7f48fb9bf740 1 -- start start 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.361+0000 7f48fb9bf740 1 Processor -- start 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.362+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.362+0000 7f48fb9bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 0x55dd07f2fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51654/0 (socket says 172.21.15.29:51654) 2024-09-21T01:51:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48eeffd640 1 -- 172.21.15.29:0/1329610548 learned_addr learned my addr 172.21.15.29:0/1329610548 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 0x55dd07f2fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000f860 con 0x55dd07b4f390 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000f160 con 0x55dd07b471b0 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.363+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000c3a0 con 0x55dd07b2e9f0 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48fb9bf740 10 client.0 _ll_register_callbacks cb 0x7f48d800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 -- 172.21.15.29:0/1329610548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 msgr2=0x55dd07f2f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 -- 172.21.15.29:0/1329610548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 msgr2=0x55dd07f30060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:51:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f30060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd07fa70d0 con 0x55dd07b471b0 2024-09-21T01:51:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48eeffd640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48e67fc640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 0x55dd07f2fb20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48d8000c00 tx=0x7f48d8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b4464e8419386e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80e72f0 con 0x55dd07b471b0 2024-09-21T01:51:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f48d80051c0 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.364+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e0031c80 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48d80ad150 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 10 client.25757 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dd07fa8340 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d80add40 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f48d80cf2b0 con 0x55dd07b471b0 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 10 client.25757 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 20 client.25757 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b70070 0x55dd07faabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 4 client.25757 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f48e0031c80 con 0x55dd07b70070 2024-09-21T01:51:40.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.365+0000 7f48eeffd640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b70070 0x55dd07faabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:40.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.366+0000 7f48d67fc640 10 client.25757 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:40.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.366+0000 7f48eeffd640 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b70070 0x55dd07faabc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6896e039f05c324b server_cookie=a9c59b65b1a6972c in_seq=0 out_seq=0 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.366+0000 7f48d67fc640 1 -- 172.21.15.29:0/1329610548 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x7f48e0031c80 con 0x55dd07b70070 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.366+0000 7f48d67fc640 10 client.25757 handle_command_reply: tid=0 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 client.25757 shutdown 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 msgr2=0x55dd07f2fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 0x55dd07f2fb20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48d8000c00 tx=0x7f48d8005880 comp rx=0 tx=0).stop 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 shutdown_connections 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55dd07b70070 0x55dd07faabc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dd07b4f390 0x55dd07f30060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48d67fc640 0 client.25757 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd07b471b0 0x55dd07f2fb20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 --2- 172.21.15.29:0/1329610548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd07b2e9f0 0x55dd07f2f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.367+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 >> 172.21.15.29:0/1329610548 conn(0x55dd07b64460 msgr2=0x55dd07f18210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 shutdown_connections 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48fb9bf740 1 -- 172.21.15.29:0/1329610548 wait complete. 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48fb9bf740 20 client.25757 trim_cache size 0 max 16384 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:51:40.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 msgr2=0x7f48dc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 0x7f48dc0b8b20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48e800bfd0 tx=0x7f48e800d040 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 msgr2=0x7f48f019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.368+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48d800e9c0 tx=0x7f48d800ee80 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 shutdown_connections 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f48dc0b6610 0x7f48dc0b8b20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48f0108cf0 0x7f48f01a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f01083a0 0x7f48f019cc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 --2- 172.21.15.29:0/1295458574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48f0102a60 0x7f48f019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 >> 172.21.15.29:0/1295458574 conn(0x7f48f0079730 msgr2=0x7f48f0106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 shutdown_connections 2024-09-21T01:51:40.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:40.369+0000 7f48f544b640 1 -- 172.21.15.29:0/1295458574 wait complete. 2024-09-21T01:51:41.253 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:51:41.253 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:51:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:41 smithi190 ceph-mon[34368]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-21T01:51:41.980 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:41 smithi029 ceph-mon[31228]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 -- 172.21.15.29:0/2773649162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08041056e0 msgr2=0x7f0804107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 --2- 172.21.15.29:0/2773649162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08041056e0 0x7f0804107bc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009900 tx=0x7f07f802a410 comp rx=0 tx=0).stop 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 -- 172.21.15.29:0/2773649162 shutdown_connections 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 --2- 172.21.15.29:0/2773649162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080410a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 --2- 172.21.15.29:0/2773649162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08041056e0 0x7f0804107bc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 --2- 172.21.15.29:0/2773649162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0804102c80 0x7f08041050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 -- 172.21.15.29:0/2773649162 >> 172.21.15.29:0/2773649162 conn(0x7f08040fd690 msgr2=0x7f08040ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 -- 172.21.15.29:0/2773649162 shutdown_connections 2024-09-21T01:51:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.027+0000 7f080b962640 1 -- 172.21.15.29:0/2773649162 wait complete. 2024-09-21T01:51:42.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 Processor -- start 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 -- start start 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 0x7f080419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f080410f3a0 con 0x7f0804102c80 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f080410f220 con 0x7f0804108300 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f080b962640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f080410f520 con 0x7f08041056e0 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38256/0 (socket says 172.21.15.29:38256) 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.028+0000 7f08096fe640 1 -- 172.21.15.29:0/3621887129 learned_addr learned my addr 172.21.15.29:0/3621887129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f0808efd640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 0x7f080419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 -- 172.21.15.29:0/3621887129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 msgr2=0x7f080419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 0x7f080419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 -- 172.21.15.29:0/3621887129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 msgr2=0x7f080419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f0809eff640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 -- 172.21.15.29:0/3621887129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080419ba40 con 0x7f0804102c80 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f0808efd640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 0x7f080419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f0809eff640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f08096fe640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f080000eb90 tx=0x7f080000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f07f67fc640 1 -- 172.21.15.29:0/3621887129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004d20 con 0x7f0804102c80 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f07f67fc640 1 -- 172.21.15.29:0/3621887129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080000ee90 con 0x7f0804102c80 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080419bd30 con 0x7f0804102c80 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f07f67fc640 1 -- 172.21.15.29:0/3621887129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800010670 con 0x7f0804102c80 2024-09-21T01:51:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.029+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08041a2420 con 0x7f0804102c80 2024-09-21T01:51:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.032+0000 7f07f67fc640 1 -- 172.21.15.29:0/3621887129 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0800005020 con 0x7f0804102c80 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 Processor -- start 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 -- start start 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563b221d6f0 con 0x5563b1f031b0 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563b220cb20 con 0x5563b1eda2f0 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08128d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563b2210930 con 0x5563b1ee24d0 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f07f67fc640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 0x7f07f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f0809eff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f08096fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f0809eff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38272/0 (socket says 172.21.15.29:38272) 2024-09-21T01:51:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f0809eff640 1 -- 172.21.15.29:0/3060277296 learned_addr learned my addr 172.21.15.29:0/3060277296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.033+0000 7f07f67fc640 1 -- 172.21.15.29:0/3621887129 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f08000e7a80 con 0x7f0804102c80 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 0x7f07f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0808efd640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 -- 172.21.15.29:0/3060277296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 msgr2=0x5563b22ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 -- 172.21.15.29:0/3060277296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 msgr2=0x5563b22ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 -- 172.21.15.29:0/3060277296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b21d2530 con 0x5563b1f031b0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0808efd640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f08096fe640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0133b0 tx=0x7f07fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abddea5fbd175b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f07ef7fe640 1 -- 172.21.15.29:0/3060277296 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc01a030 con 0x5563b1f031b0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f07ef7fe640 1 -- 172.21.15.29:0/3060277296 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc00ca10 con 0x5563b1f031b0 2024-09-21T01:51:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f07ef7fe640 1 -- 172.21.15.29:0/3060277296 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00f040 con 0x5563b1f031b0 2024-09-21T01:51:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.034+0000 7f0809eff640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 0x7f07f00b8a20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f07fc01d000 tx=0x7f07fc01b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 -- 172.21.15.29:0/3060277296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 msgr2=0x5563b22aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0133b0 tx=0x7f07fc00cf90 comp rx=0 tx=0).stop 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 -- 172.21.15.29:0/3060277296 shutdown_connections 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 --2- 172.21.15.29:0/3060277296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1f031b0 0x5563b22aae50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 -- 172.21.15.29:0/3060277296 >> 172.21.15.29:0/3060277296 conn(0x5563b1ef7300 msgr2=0x5563b22a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 -- 172.21.15.29:0/3060277296 shutdown_connections 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.035+0000 7f08128d1740 1 -- 172.21.15.29:0/3060277296 wait complete. 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 -- start start 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 Processor -- start 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1ee24d0 0x5563b22bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1f031b0 0x5563b22be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563b2144ab0 con 0x5563b1eda2f0 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563b21b2730 con 0x5563b1ee24d0 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563b22acb30 con 0x5563b1f031b0 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08128d1740 10 client.0 _ll_register_callbacks cb 0x5563b22066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38288/0 (socket says 172.21.15.29:38288) 2024-09-21T01:51:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f08096fe640 1 -- 172.21.15.29:0/3239478735 learned_addr learned my addr 172.21.15.29:0/3239478735 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.036+0000 7f0808efd640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1ee24d0 0x5563b22bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f0809eff640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1f031b0 0x5563b22be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 -- 172.21.15.29:0/3239478735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1f031b0 msgr2=0x5563b22be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1f031b0 0x5563b22be0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 -- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1ee24d0 msgr2=0x5563b22bd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1ee24d0 0x5563b22bd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 -- 172.21.15.29:0/3239478735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b1f6e790 con 0x5563b1eda2f0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08096fe640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08000e5590 tx=0x7f0800076790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e15d0221bd19b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000a7070 con 0x5563b1eda2f0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f08000b73b0 con 0x5563b1eda2f0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07fc00ca10 con 0x5563b1eda2f0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0800016540 con 0x5563b1eda2f0 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08128d1740 10 client.16053 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563b1fe7620 con 0x5563b1eda2f0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc00ca10 con 0x5563b1eda2f0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f08000d1040 con 0x5563b1eda2f0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.037+0000 7f08128d1740 10 client.16053 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f08128d1740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f08128d1740 20 client.16053 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1ec1d80 0x5563b22ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f08128d1740 4 client.16053 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563b221f640 con 0x5563b1ec1d80 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f0808efd640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1ec1d80 0x5563b22ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f07ecff9640 10 client.16053 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.038+0000 7f0808efd640 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1ec1d80 0x5563b22ca6f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8444a9a0dea87d1c server_cookie=d59df0bc91db7798 in_seq=0 out_seq=0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f07ecff9640 1 -- 172.21.15.29:0/3239478735 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563b221f640 con 0x5563b1ec1d80 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f07ecff9640 10 client.16053 handle_command_reply: tid=0 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 client.16053 shutdown 2024-09-21T01:51:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 msgr2=0x5563b22bce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08000e5590 tx=0x7f0800076790 comp rx=0 tx=0).stop 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 shutdown_connections 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1ec1d80 0x5563b22ca6f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1f031b0 0x5563b22be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1ee24d0 0x5563b22bd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 --2- 172.21.15.29:0/3239478735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1eda2f0 0x5563b22bce70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f07ecff9640 0 client.16053 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 >> 172.21.15.29:0/3239478735 conn(0x5563b1ef7300 msgr2=0x5563b22a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 shutdown_connections 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 1 -- 172.21.15.29:0/3239478735 wait complete. 2024-09-21T01:51:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.039+0000 7f08128d1740 20 client.16053 trim_cache size 0 max 16384 2024-09-21T01:51:42.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:41 smithi066 ceph-mon[35001]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-21T01:51:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 Processor -- start 2024-09-21T01:51:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 -- start start 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 0x5563b22a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ee24d0 0x5563b22a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08041a8dc0 con 0x5563b1ee24d0 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08041aa9f0 con 0x5563b1eda2f0 2024-09-21T01:51:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.071+0000 7f08128d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f08041a9490 con 0x5563b1ec1d80 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f08096fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 0x5563b22a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f08096fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 0x5563b22a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:47974/0 (socket says 172.21.15.29:47974) 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f08096fe640 1 -- 172.21.15.29:0/2178399710 learned_addr learned my addr 172.21.15.29:0/2178399710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 -- 172.21.15.29:0/2178399710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 msgr2=0x5563b22a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 0x5563b22a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 -- 172.21.15.29:0/2178399710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ee24d0 msgr2=0x5563b22a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ee24d0 0x5563b22a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.072+0000 7f0808efd640 1 -- 172.21.15.29:0/2178399710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b22be410 con 0x5563b1eda2f0 2024-09-21T01:51:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.073+0000 7f0808efd640 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f8005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9795e3d904ee4ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.073+0000 7f07eeffd640 1 -- 172.21.15.29:0/2178399710 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8035040 con 0x5563b1eda2f0 2024-09-21T01:51:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.073+0000 7f07eeffd640 1 -- 172.21.15.29:0/2178399710 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8047070 con 0x5563b1eda2f0 2024-09-21T01:51:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.074+0000 7f07eeffd640 1 -- 172.21.15.29:0/2178399710 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8004920 con 0x5563b1eda2f0 2024-09-21T01:51:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.075+0000 7f08128d1740 1 -- 172.21.15.29:0/2178399710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 msgr2=0x5563b22c4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.075+0000 7f08128d1740 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f8005d10 comp rx=0 tx=0).stop 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 -- 172.21.15.29:0/2178399710 shutdown_connections 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ee24d0 0x5563b22a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c4f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 --2- 172.21.15.29:0/2178399710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ec1d80 0x5563b22a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 -- 172.21.15.29:0/2178399710 >> 172.21.15.29:0/2178399710 conn(0x5563b1ef7300 msgr2=0x5563b22a6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 -- 172.21.15.29:0/2178399710 shutdown_connections 2024-09-21T01:51:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.076+0000 7f08128d1740 1 -- 172.21.15.29:0/2178399710 wait complete. 2024-09-21T01:51:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.077+0000 7f08128d1740 1 -- start start 2024-09-21T01:51:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.077+0000 7f08128d1740 1 Processor -- start 2024-09-21T01:51:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08041aa260 con 0x5563b1ec1d80 2024-09-21T01:51:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08041a9b60 con 0x5563b1eda2f0 2024-09-21T01:51:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f08041a6da0 con 0x5563b1ee24d0 2024-09-21T01:51:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08128d1740 10 client.0 _ll_register_callbacks cb 0x7f08000a9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08096fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38306/0 (socket says 172.21.15.29:38306) 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f08096fe640 1 -- 172.21.15.29:0/880548009 learned_addr learned my addr 172.21.15.29:0/880548009 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f0809eff640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.078+0000 7f0808efd640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 -- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 msgr2=0x5563b22c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 -- 172.21.15.29:0/880548009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 msgr2=0x5563b22c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 -- 172.21.15.29:0/880548009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b22c2740 con 0x5563b1ee24d0 2024-09-21T01:51:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f08096fe640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f0809eff640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22c4310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cf60 tx=0x7f07fc013e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ee982904d64428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.079+0000 7f07ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc02d060 con 0x5563b1ee24d0 2024-09-21T01:51:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f07fc016400 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f8047070 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f07fc024030 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f08128d1740 10 client.25822 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.080+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563b22b32b0 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.081+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8047070 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.081+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f07fc015040 con 0x5563b1ee24d0 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.081+0000 7f08128d1740 10 client.25822 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.081+0000 7f08128d1740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.081+0000 7f08128d1740 20 client.25822 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.082+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1f031b0 0x5563b22b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.082+0000 7f08128d1740 4 client.25822 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:51:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.082+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563b221b0f0 con 0x5563b1f031b0 2024-09-21T01:51:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.082+0000 7f08096fe640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1f031b0 0x5563b22b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.084+0000 7f08096fe640 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1f031b0 0x5563b22b5bd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc3a4f56df8b537 server_cookie=7c5e92d8d87d458b in_seq=0 out_seq=0 2024-09-21T01:51:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.084+0000 7f07ecff9640 10 client.25822 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.097+0000 7f07ecff9640 1 -- 172.21.15.29:0/880548009 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563b221b0f0 con 0x5563b1f031b0 2024-09-21T01:51:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.097+0000 7f07ecff9640 10 client.25822 handle_command_reply: tid=0 2024-09-21T01:51:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.097+0000 7f08128d1740 1 client.25822 shutdown 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 msgr2=0x5563b22c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22c4310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cf60 tx=0x7f07fc013e40 comp rx=0 tx=0).stop 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 shutdown_connections 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563b1f031b0 0x5563b22b5bd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563b1ee24d0 0x5563b22c4310 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563b1eda2f0 0x5563b22c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 --2- 172.21.15.29:0/880548009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563b1ec1d80 0x5563b22c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.098+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 >> 172.21.15.29:0/880548009 conn(0x5563b1ef7300 msgr2=0x5563b22a55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.099+0000 7f07ecff9640 0 client.25822 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.099+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 shutdown_connections 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.099+0000 7f08128d1740 1 -- 172.21.15.29:0/880548009 wait complete. 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f08128d1740 20 client.25822 trim_cache size 0 max 16384 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 msgr2=0x7f07f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 0x7f07f00b8a20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f07fc01d000 tx=0x7f07fc01b000 comp rx=0 tx=0).stop 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 msgr2=0x7f080419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f080000eb90 tx=0x7f080000c6a0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 shutdown_connections 2024-09-21T01:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f07f00b6510 0x7f07f00b8a20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0804108300 0x7f080419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08041056e0 0x7f080419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 --2- 172.21.15.29:0/3621887129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0804102c80 0x7f080419a000 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.100+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 >> 172.21.15.29:0/3621887129 conn(0x7f08040fd690 msgr2=0x7f0804103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.101+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 shutdown_connections 2024-09-21T01:51:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:42.101+0000 7f080b962640 1 -- 172.21.15.29:0/3621887129 wait complete. 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3930608318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10a3c0 msgr2=0x7f7eec10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3930608318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10a3c0 0x7f7eec10a7d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009900 tx=0x7f7ee802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3930608318 shutdown_connections 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3930608318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10ad10 0x7f7eec116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3930608318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10a3c0 0x7f7eec10a7d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3930608318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10f150 0x7f7eec10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3930608318 >> 172.21.15.29:0/3930608318 conn(0x7f7eec0712f0 msgr2=0x7f7eec073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3930608318 shutdown_connections 2024-09-21T01:51:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.283+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3930608318 wait complete. 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 Processor -- start 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 -- start start 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 0x7f7eec07cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10f150 0x7f7eec085630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec077080 con 0x7f7eec10a3c0 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec076f00 con 0x7f7eec10ad10 2024-09-21T01:51:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef41f1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec077200 con 0x7f7eec10f150 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef1f8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 0x7f7eec07cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef178c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef178c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44318/0 (socket says 172.21.15.29:44318) 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef1f8d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 0x7f7eec07cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38318/0 (socket says 172.21.15.29:38318) 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.284+0000 7f7ef178c640 1 -- 172.21.15.29:0/3124980561 learned_addr learned my addr 172.21.15.29:0/3124980561 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef278e640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10f150 0x7f7eec085630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 -- 172.21.15.29:0/3124980561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10f150 msgr2=0x7f7eec085630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10f150 0x7f7eec085630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 -- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 msgr2=0x7f7eec07cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 0x7f7eec07cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 -- 172.21.15.29:0/3124980561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec085c70 con 0x7f7eec10ad10 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.285+0000 7f7ef178c640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ee802a910 tx=0x7f7ee8030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.286+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee803f070 con 0x7f7eec10ad10 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.286+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee8004460 con 0x7f7eec10ad10 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.286+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8035030 con 0x7f7eec10ad10 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.286+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec085f00 con 0x7f7eec10ad10 2024-09-21T01:51:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.286+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec086390 con 0x7f7eec10ad10 2024-09-21T01:51:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.287+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec07dda0 con 0x7f7eec10ad10 2024-09-21T01:51:42.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.290+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7ee8042070 con 0x7f7eec10ad10 2024-09-21T01:51:42.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.291+0000 7f7ee2ffd640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 0x7f7ed80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:42.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.291+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7ee8107fa0 con 0x7f7eec10ad10 2024-09-21T01:51:42.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.292+0000 7f7ef1f8d640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 0x7f7ed80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:42.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.293+0000 7f7ef1f8d640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 0x7f7ed80b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7eec083540 tx=0x7f7edc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:42.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.294+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ee80ce840 con 0x7f7eec10ad10 2024-09-21T01:51:42.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.742+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eec0824b0 con 0x7f7eec10ad10 2024-09-21T01:51:42.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.743+0000 7f7ee2ffd640 1 -- 172.21.15.29:0/3124980561 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11978 (secure 0 0 0) 0x7f7ee80d2c20 con 0x7f7eec10ad10 2024-09-21T01:51:42.745 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:51:42.745 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"btime":"2024-09-21T01:51:36:612582+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:51:36.249660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:51:42.745 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 33 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 msgr2=0x7f7ed80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 0x7f7ed80b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7eec083540 tx=0x7f7edc00a2d0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 msgr2=0x7f7eec07d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ee802a910 tx=0x7f7ee8030650 comp rx=0 tx=0).stop 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 shutdown_connections 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ed80b6590 0x7f7ed80b8aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eec10f150 0x7f7eec085630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec10ad10 0x7f7eec07d1f0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ee802a910 tx=0x7f7ee8030650 comp rx=0 tx=0).stop 2024-09-21T01:51:42.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 --2- 172.21.15.29:0/3124980561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec10a3c0 0x7f7eec07cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:42.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.747+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 >> 172.21.15.29:0/3124980561 conn(0x7f7eec0712f0 msgr2=0x7f7eec10cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:42.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.748+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 shutdown_connections 2024-09-21T01:51:42.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:42.748+0000 7f7ef41f1640 1 -- 172.21.15.29:0/3124980561 wait complete. 2024-09-21T01:51:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 185.0 s, 3494.6 tps, lat 28.542 ms stddev 34.968 2024-09-21T01:51:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:43 smithi190 ceph-mon[34368]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-09-21T01:51:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:43 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3124980561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:51:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:43 smithi066 ceph-mon[35001]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-09-21T01:51:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:43 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3124980561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:51:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:43 smithi029 ceph-mon[31228]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-09-21T01:51:44.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:43 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3124980561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:51:44.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:51:44.883 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:45.597 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:45.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 -- 172.21.15.29:0/3258302518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f10105db0 msgr2=0x7f8f101061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 --2- 172.21.15.29:0/3258302518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f10105db0 0x7f8f101061c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009860 tx=0x7f8f0002a340 comp rx=0 tx=0).stop 2024-09-21T01:51:45.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 -- 172.21.15.29:0/3258302518 shutdown_connections 2024-09-21T01:51:45.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 --2- 172.21.15.29:0/3258302518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f10112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 --2- 172.21.15.29:0/3258302518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1010b6f0 0x7f8f1010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 --2- 172.21.15.29:0/3258302518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f10105db0 0x7f8f101061c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 -- 172.21.15.29:0/3258302518 >> 172.21.15.29:0/3258302518 conn(0x7f8f101017c0 msgr2=0x7f8f10103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 -- 172.21.15.29:0/3258302518 shutdown_connections 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.480+0000 7f8f15e24640 1 -- 172.21.15.29:0/3258302518 wait complete. 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.481+0000 7f8f15e24640 1 Processor -- start 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.481+0000 7f8f15e24640 1 -- start start 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.482+0000 7f8f15e24640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 0x7f8f1019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f15e24640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1010b6f0 0x7f8f1019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f15e24640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 0x7f8f1019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f15e24640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10115d00 con 0x7f8f10105db0 2024-09-21T01:51:45.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f15e24640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10115b80 con 0x7f8f1010c040 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f15e24640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10115e80 con 0x7f8f1010b6f0 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 0x7f8f1019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38326/0 (socket says 172.21.15.29:38326) 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f0f7fe640 1 -- 172.21.15.29:0/1787979631 learned_addr learned my addr 172.21.15.29:0/1787979631 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44324/0 (socket says 172.21.15.29:44324) 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.483+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1010b6f0 0x7f8f1019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 -- 172.21.15.29:0/1787979631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1010b6f0 msgr2=0x7f8f1019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1010b6f0 0x7f8f1019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 -- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 msgr2=0x7f8f1019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 0x7f8f1019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 -- 172.21.15.29:0/1787979631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a2660 con 0x7f8f1010c040 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.484+0000 7f8f0ffff640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800ef00 tx=0x7f8f0800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.485+0000 7f8f0d7fa640 1 -- 172.21.15.29:0/1787979631 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08004b20 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.485+0000 7f8f0d7fa640 1 -- 172.21.15.29:0/1787979631 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f08003e20 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.485+0000 7f8f0d7fa640 1 -- 172.21.15.29:0/1787979631 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800b570 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.485+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a2950 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.485+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a90b0 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 Processor -- start 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 -- start start 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f0d7fa640 1 -- 172.21.15.29:0/1787979631 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8f0801d020 con 0x7f8f1010c040 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a525928010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525557390 0x55a525928550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a525895ea0 con 0x55a525557390 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a5258852d0 con 0x55a525578070 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.491+0000 7f8f1c398740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a5258890e0 con 0x55a52554f1b0 2024-09-21T01:51:45.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44328/0 (socket says 172.21.15.29:44328) 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 -- 172.21.15.29:0/4132743224 learned_addr learned my addr 172.21.15.29:0/4132743224 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525557390 0x55a525928550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 -- 172.21.15.29:0/4132743224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 msgr2=0x55a525928010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a525928010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.492+0000 7f8f06ffd640 1 -- 172.21.15.29:0/4132743224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525557390 msgr2=0x55a525928550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.493+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525557390 0x55a525928550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.493+0000 7f8f06ffd640 1 -- 172.21.15.29:0/4132743224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5255e3650 con 0x55a525578070 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.493+0000 7f8f0d7fa640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 0x7f8efc0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.493+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005d00 tx=0x7f8ef8005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c09c5d91b11dbc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.493+0000 7f8f05ffb640 1 -- 172.21.15.29:0/4132743224 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8012ca0 con 0x55a525578070 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.494+0000 7f8f0d7fa640 1 -- 172.21.15.29:0/1787979631 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8f080e7080 con 0x7f8f1010c040 2024-09-21T01:51:45.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.494+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 0x7f8efc0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.495+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 0x7f8efc0b8bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a525926750 tx=0x7f8f00030340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.495+0000 7f8f05ffb640 1 -- 172.21.15.29:0/4132743224 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef8012e40 con 0x55a525578070 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.495+0000 7f8f05ffb640 1 -- 172.21.15.29:0/4132743224 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8004a00 con 0x55a525578070 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 -- 172.21.15.29:0/4132743224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 msgr2=0x55a525927ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005d00 tx=0x7f8ef8005c90 comp rx=0 tx=0).stop 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 -- 172.21.15.29:0/4132743224 shutdown_connections 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525557390 0x55a525928550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a525928010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 --2- 172.21.15.29:0/4132743224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525578070 0x55a525927ad0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 -- 172.21.15.29:0/4132743224 >> 172.21.15.29:0/4132743224 conn(0x55a52556c460 msgr2=0x55a5259259c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 -- 172.21.15.29:0/4132743224 shutdown_connections 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.497+0000 7f8f1c398740 1 -- 172.21.15.29:0/4132743224 wait complete. 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.498+0000 7f8f1c398740 1 -- start start 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.498+0000 7f8f1c398740 1 Processor -- start 2024-09-21T01:51:45.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a52554f1b0 0x55a525933200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 0x55a52593f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a5256e6700 con 0x55a52554f1b0 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a525820160 con 0x55a525557390 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a5259362b0 con 0x55a525578070 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f1c398740 10 client.0 _ll_register_callbacks cb 0x55a5258399e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44332/0 (socket says 172.21.15.29:44332) 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 0x55a52593f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 -- 172.21.15.29:0/638745582 learned_addr learned my addr 172.21.15.29:0/638745582 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a52554f1b0 0x55a525933200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 -- 172.21.15.29:0/638745582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 msgr2=0x55a52593f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 0x55a52593f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.499+0000 7f8f06ffd640 1 -- 172.21.15.29:0/638745582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a52554f1b0 msgr2=0x55a525933200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a52554f1b0 0x55a525933200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8f06ffd640 1 -- 172.21.15.29:0/638745582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52584dc90 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8f0ffff640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 0x55a52593f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005510 tx=0x7f8ef8005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5dd9f5afaba9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8012ca0 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.500+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8ef8010b80 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.501+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a525897df0 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.501+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ef8010d80 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.502+0000 7f8f1c398740 10 client.25787 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.502+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a52593fb10 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a525897df0 con 0x55a525557390 2024-09-21T01:51:45.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f8ef801f5a0 con 0x55a525557390 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 10 client.25787 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 10 client.25787 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 20 client.25787 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a5255369f0 0x55a525942f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 4 client.25787 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5258938a0 con 0x55a5255369f0 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.503+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a5255369f0 0x55a525942f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.504+0000 7f8eeeffd640 10 client.25787 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.505+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a5255369f0 0x55a525942f80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=881aeecd9ffe870e server_cookie=48cb4a4491c5d691 in_seq=0 out_seq=0 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.506+0000 7f8eeeffd640 1 -- 172.21.15.29:0/638745582 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5258938a0 con 0x55a5255369f0 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.506+0000 7f8eeeffd640 10 client.25787 handle_command_reply: tid=0 2024-09-21T01:51:45.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.506+0000 7f8f1c398740 1 client.25787 shutdown 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 msgr2=0x55a525933740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005510 tx=0x7f8ef8005250 comp rx=0 tx=0).stop 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 shutdown_connections 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a5255369f0 0x55a525942f80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525578070 0x55a52593f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a525933740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 --2- 172.21.15.29:0/638745582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a52554f1b0 0x55a525933200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 >> 172.21.15.29:0/638745582 conn(0x55a52556c460 msgr2=0x55a52592ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8eeeffd640 0 client.25787 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.507+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 shutdown_connections 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.508+0000 7f8f1c398740 1 -- 172.21.15.29:0/638745582 wait complete. 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.508+0000 7f8f1c398740 20 client.25787 trim_cache size 0 max 16384 2024-09-21T01:51:45.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.534+0000 7f8f1c398740 1 Processor -- start 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 -- start start 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a5259336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a525950410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10070e00 con 0x55a525578070 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10072a30 con 0x55a525557390 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.535+0000 7f8f1c398740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8f100714d0 con 0x55a52554f1b0 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48028/0 (socket says 172.21.15.29:48028) 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 -- 172.21.15.29:0/1619486331 learned_addr learned my addr 172.21.15.29:0/1619486331 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a5259336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 -- 172.21.15.29:0/1619486331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 msgr2=0x55a5259336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a5259336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 -- 172.21.15.29:0/1619486331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 msgr2=0x55a525950410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a525950410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.536+0000 7f8f0f7fe640 1 -- 172.21.15.29:0/1619486331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a525929ff0 con 0x55a52554f1b0 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.537+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f000095f0 tx=0x7f8f0003c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971bbb66deec4751 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.538+0000 7f8f057fa640 1 -- 172.21.15.29:0/1619486331 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00078070 con 0x55a52554f1b0 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.538+0000 7f8f057fa640 1 -- 172.21.15.29:0/1619486331 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f000307d0 con 0x55a52554f1b0 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.538+0000 7f8f057fa640 1 -- 172.21.15.29:0/1619486331 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00037040 con 0x55a52554f1b0 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 -- 172.21.15.29:0/1619486331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 msgr2=0x55a52593f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f000095f0 tx=0x7f8f0003c000 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 -- 172.21.15.29:0/1619486331 shutdown_connections 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a525950410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a525557390 0x55a5259336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1619486331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a52554f1b0 0x55a52593f540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.539+0000 7f8f1c398740 1 -- 172.21.15.29:0/1619486331 >> 172.21.15.29:0/1619486331 conn(0x55a52556c460 msgr2=0x55a525932fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.540+0000 7f8f1c398740 1 -- 172.21.15.29:0/1619486331 shutdown_connections 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.540+0000 7f8f1c398740 1 -- 172.21.15.29:0/1619486331 wait complete. 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.540+0000 7f8f1c398740 1 -- start start 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.541+0000 7f8f1c398740 1 Processor -- start 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.541+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a52554f1b0 0x55a5259428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.541+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f1c398740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a5259a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f1c398740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f100722a0 con 0x55a525578070 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f1c398740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10071ba0 con 0x55a52554f1b0 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f1c398740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1006ede0 con 0x55a525557390 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f1c398740 10 client.0 _ll_register_callbacks cb 0x7f8ef800be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a52554f1b0 0x55a5259428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a5259a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48032/0 (socket says 172.21.15.29:48032) 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 -- 172.21.15.29:0/1270510164 learned_addr learned my addr 172.21.15.29:0/1270510164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 -- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a52554f1b0 msgr2=0x55a5259428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a52554f1b0 0x55a5259428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 -- 172.21.15.29:0/1270510164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 msgr2=0x55a5259a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a5259a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.542+0000 7f8f06ffd640 1 -- 172.21.15.29:0/1270510164 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5259a8920 con 0x55a525557390 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.543+0000 7f8f06ffd640 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8010740 tx=0x7f8ef8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44fac296424cd3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.543+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.543+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800b390 con 0x55a525557390 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.543+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f000307d0 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.544+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8ef800ba20 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.545+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ef8005340 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.545+0000 7f8f1c398740 10 client.25843 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.545+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5259afc40 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f000307d0 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f8ef80038e0 con 0x55a525557390 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 10 client.25843 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 10 client.25843 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 20 client.25843 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a52550bda0 0x55a5259b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 4 client.25843 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.546+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a525890cd0 con 0x55a52550bda0 2024-09-21T01:51:45.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.547+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a52550bda0 0x55a5259b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.549+0000 7f8f0f7fe640 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a52550bda0 0x55a5259b23a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3bc8d8430eeddd5 server_cookie=899fc06407a3219e in_seq=0 out_seq=0 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.549+0000 7f8eeeffd640 10 client.25843 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.549+0000 7f8eeeffd640 1 -- 172.21.15.29:0/1270510164 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a525890cd0 con 0x55a52550bda0 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.549+0000 7f8eeeffd640 10 client.25843 handle_command_reply: tid=0 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.549+0000 7f8f1c398740 1 client.25843 shutdown 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 msgr2=0x55a525942e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8010740 tx=0x7f8ef8005750 comp rx=0 tx=0).stop 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 shutdown_connections 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55a52550bda0 0x55a5259b23a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a525578070 0x55a5259a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a525557390 0x55a525942e90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 --2- 172.21.15.29:0/1270510164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a52554f1b0 0x55a5259428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 >> 172.21.15.29:0/1270510164 conn(0x55a52556c460 msgr2=0x55a525928a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8eeeffd640 0 client.25843 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:51:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 shutdown_connections 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 1 -- 172.21.15.29:0/1270510164 wait complete. 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.550+0000 7f8f1c398740 20 client.25843 trim_cache size 0 max 16384 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.551+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 msgr2=0x7f8efc0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.551+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 0x7f8efc0b8bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a525926750 tx=0x7f8f00030340 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.551+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 msgr2=0x7f8f101a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.551+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800ef00 tx=0x7f8f0800c740 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 shutdown_connections 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8efc0b66e0 0x7f8efc0b8bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f1010c040 0x7f8f101a1f30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1010b6f0 0x7f8f1019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 --2- 172.21.15.29:0/1787979631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f10105db0 0x7f8f1019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 >> 172.21.15.29:0/1787979631 conn(0x7f8f101017c0 msgr2=0x7f8f10104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 shutdown_connections 2024-09-21T01:51:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:45.552+0000 7f8f15e24640 1 -- 172.21.15.29:0/1787979631 wait complete. 2024-09-21T01:51:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:45 smithi190 ceph-mon[34368]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 781 op/s 2024-09-21T01:51:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:45 smithi066 ceph-mon[35001]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 781 op/s 2024-09-21T01:51:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:45 smithi029 ceph-mon[31228]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 781 op/s 2024-09-21T01:51:47.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:51:47.720 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:47.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 190.0 s, 4307.1 tps, lat 23.162 ms stddev 40.009 2024-09-21T01:51:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:47 smithi190 ceph-mon[34368]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-21T01:51:48.007 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:47 smithi029 ceph-mon[31228]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-21T01:51:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:47 smithi066 ceph-mon[35001]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 -- 172.21.15.29:0/2649069670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b41094b0 msgr2=0x7fc2b41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 --2- 172.21.15.29:0/2649069670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b41094b0 0x7fc2b41098c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009860 tx=0x7fc2a402a340 comp rx=0 tx=0).stop 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 -- 172.21.15.29:0/2649069670 shutdown_connections 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 --2- 172.21.15.29:0/2649069670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4109e00 0x7fc2b4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 --2- 172.21.15.29:0/2649069670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b41094b0 0x7fc2b41098c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 --2- 172.21.15.29:0/2649069670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b4103b70 0x7fc2b4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.277+0000 7fc2b9436640 1 -- 172.21.15.29:0/2649069670 >> 172.21.15.29:0/2649069670 conn(0x7fc2b40ff5e0 msgr2=0x7fc2b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.278+0000 7fc2b9436640 1 -- 172.21.15.29:0/2649069670 shutdown_connections 2024-09-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.278+0000 7fc2b9436640 1 -- 172.21.15.29:0/2649069670 wait complete. 2024-09-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.279+0000 7fc2b9436640 1 Processor -- start 2024-09-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.279+0000 7fc2b9436640 1 -- start start 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b9436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 0x7fc2b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b9436640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 0x7fc2b419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 0x7fc2b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 0x7fc2b419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38376/0 (socket says 172.21.15.29:38376) 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b9436640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 0x7fc2b419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 0x7fc2b419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44358/0 (socket says 172.21.15.29:44358) 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b9436640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4113ac0 con 0x7fc2b4103b70 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.280+0000 7fc2b9436640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4113940 con 0x7fc2b41094b0 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.281+0000 7fc2b9436640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4113c40 con 0x7fc2b4109e00 2024-09-21T01:51:48.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.281+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.281+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48048/0 (socket says 172.21.15.29:48048) 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.281+0000 7fc2b37fe640 1 -- 172.21.15.29:0/1432796417 learned_addr learned my addr 172.21.15.29:0/1432796417 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 -- 172.21.15.29:0/1432796417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 msgr2=0x7fc2b419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 0x7fc2b419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 -- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 msgr2=0x7fc2b419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 0x7fc2b419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 -- 172.21.15.29:0/1432796417 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41a0420 con 0x7fc2b4109e00 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009fd0 tx=0x7fc2ac00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc29bfff640 1 -- 172.21.15.29:0/1432796417 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004b20 con 0x7fc2b4109e00 2024-09-21T01:51:48.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.282+0000 7fc29bfff640 1 -- 172.21.15.29:0/1432796417 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac003e20 con 0x7fc2b4109e00 2024-09-21T01:51:48.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.283+0000 7fc29bfff640 1 -- 172.21.15.29:0/1432796417 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004820 con 0x7fc2b4109e00 2024-09-21T01:51:48.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.283+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41a0710 con 0x7fc2b4109e00 2024-09-21T01:51:48.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.283+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b40755f0 con 0x7fc2b4109e00 2024-09-21T01:51:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.286+0000 7fc2bf9aa740 1 Processor -- start 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.287+0000 7fc29bfff640 1 -- 172.21.15.29:0/1432796417 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc2ac01d020 con 0x7fc2b4109e00 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 -- start start 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a7882390 0x5650a7c4ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5650a7bc0ea0 con 0x5650a78a3070 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5650a7bb02d0 con 0x5650a7882390 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650a7bb40e0 con 0x5650a787a1b0 2024-09-21T01:51:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38378/0 (socket says 172.21.15.29:38378) 2024-09-21T01:51:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48056/0 (socket says 172.21.15.29:48056) 2024-09-21T01:51:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b27fc640 1 -- 172.21.15.29:0/493616839 learned_addr learned my addr 172.21.15.29:0/493616839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.288+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a7882390 0x5650a7c4ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/493616839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a7882390 msgr2=0x5650a7c4ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a7882390 0x5650a7c4ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/493616839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 msgr2=0x5650a7c4a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/493616839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a790e650 con 0x5650a787a1b0 2024-09-21T01:51:48.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc29bfff640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 0x7fc29c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8004520 tx=0x7fc2a8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243534d9b16d3bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 0x7fc29c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2997fa640 1 -- 172.21.15.29:0/493616839 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8009c40 con 0x5650a787a1b0 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2997fa640 1 -- 172.21.15.29:0/493616839 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a8009de0 con 0x5650a787a1b0 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.289+0000 7fc2997fa640 1 -- 172.21.15.29:0/493616839 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004de0 con 0x5650a787a1b0 2024-09-21T01:51:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.290+0000 7fc29bfff640 1 -- 172.21.15.29:0/1432796417 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2ac0e7260 con 0x7fc2b4109e00 2024-09-21T01:51:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/493616839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 msgr2=0x5650a7c4a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8004520 tx=0x7fc2a8005f90 comp rx=0 tx=0).stop 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 0x7fc29c0b8aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009830 tx=0x7fc2a40056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/493616839 shutdown_connections 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a7882390 0x5650a7c4ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c4a800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/493616839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/493616839 >> 172.21.15.29:0/493616839 conn(0x5650a7897460 msgr2=0x5650a7c481b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/493616839 shutdown_connections 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.291+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/493616839 wait complete. 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.292+0000 7fc2bf9aa740 1 -- start start 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.292+0000 7fc2bf9aa740 1 Processor -- start 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a78a3070 0x5650a7c640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5650a7a11700 con 0x5650a7882390 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5650a7b4b160 con 0x5650a787a1b0 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650a7c58aa0 con 0x5650a78a3070 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2bf9aa740 10 client.0 _ll_register_callbacks cb 0x5650a7b649e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c5f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38384/0 (socket says 172.21.15.29:38384) 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3652264477 learned_addr learned my addr 172.21.15.29:0/3652264477 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44370/0 (socket says 172.21.15.29:44370) 2024-09-21T01:51:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a78a3070 0x5650a7c640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a78a3070 msgr2=0x5650a7c640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a78a3070 0x5650a7c640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/3652264477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 msgr2=0x5650a7c5f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c5f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.293+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/3652264477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a7b78c90 con 0x5650a787a1b0 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005890 tx=0x7fc2a8012f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8262c3863ad0fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8029060 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a8009de0 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2a8012260 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc2a8012460 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc2bf9aa740 10 client.25802 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5650a7c6cff0 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a8009de0 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.294+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fc2a800f490 con 0x5650a787a1b0 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 10 client.25802 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 10 client.25802 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 20 client.25802 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a78619f0 0x5650a7c703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 4 client.25802 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a7bc2df0 con 0x5650a78619f0 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a78619f0 0x5650a7c703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a78619f0 0x5650a7c703b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c87c4deb8fe6af9 server_cookie=5e1cac99e09e19a6 in_seq=0 out_seq=0 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.295+0000 7fc296ff5640 10 client.25802 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.296+0000 7fc296ff5640 1 -- 172.21.15.29:0/3652264477 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650a7bc2df0 con 0x5650a78619f0 2024-09-21T01:51:48.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.296+0000 7fc296ff5640 10 client.25802 handle_command_reply: tid=0 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 client.25802 shutdown 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 msgr2=0x5650a7c5eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005890 tx=0x7fc2a8012f20 comp rx=0 tx=0).stop 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 shutdown_connections 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a78619f0 0x5650a7c703b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a78a3070 0x5650a7c640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc296ff5640 0 client.25802 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:51:48.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.297+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3652264477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c5eaa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.298+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 >> 172.21.15.29:0/3652264477 conn(0x5650a7897460 msgr2=0x5650a7c4d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.298+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 shutdown_connections 2024-09-21T01:51:48.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.298+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3652264477 wait complete. 2024-09-21T01:51:48.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.298+0000 7fc2bf9aa740 20 client.25802 trim_cache size 0 max 16384 2024-09-21T01:51:48.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 Processor -- start 2024-09-21T01:51:48.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 -- start start 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c57dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ad210 con 0x5650a78a3070 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41aee40 con 0x5650a787a1b0 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ad8e0 con 0x5650a7882390 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2b27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48080/0 (socket says 172.21.15.29:48080) 2024-09-21T01:51:48.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3855537100 learned_addr learned my addr 172.21.15.29:0/3855537100 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.324+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c57dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3855537100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 msgr2=0x5650a7c57dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c57dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3855537100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 msgr2=0x5650a7c5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3855537100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a7c5fb60 con 0x5650a7882390 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009860 tx=0x7fc2a4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5421156b3fc55dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.325+0000 7fc298ff9640 1 -- 172.21.15.29:0/3855537100 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4058070 con 0x5650a7882390 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.326+0000 7fc298ff9640 1 -- 172.21.15.29:0/3855537100 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a4003aa0 con 0x5650a7882390 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.326+0000 7fc298ff9640 1 -- 172.21.15.29:0/3855537100 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4046040 con 0x5650a7882390 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3855537100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 msgr2=0x5650a7c4ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009860 tx=0x7fc2a4005f90 comp rx=0 tx=0).stop 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3855537100 shutdown_connections 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a78a3070 0x5650a7c5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a7882390 0x5650a7c4ba00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3855537100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a787a1b0 0x5650a7c57dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3855537100 >> 172.21.15.29:0/3855537100 conn(0x5650a7897460 msgr2=0x5650a7c4b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3855537100 shutdown_connections 2024-09-21T01:51:48.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.327+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3855537100 wait complete. 2024-09-21T01:51:48.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.328+0000 7fc2bf9aa740 1 -- start start 2024-09-21T01:51:48.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.328+0000 7fc2bf9aa740 1 Processor -- start 2024-09-21T01:51:48.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.328+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c6a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.328+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.328+0000 7fc2bf9aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 0x5650a7cd30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ae6b0 con 0x5650a7882390 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41adfb0 con 0x5650a78a3070 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2bf9aa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ab1f0 con 0x5650a787a1b0 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c6a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2bf9aa740 10 client.0 _ll_register_callbacks cb 0x7fc2a4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c6a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48096/0 (socket says 172.21.15.29:48096) 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b2ffd640 1 -- 172.21.15.29:0/3512341508 learned_addr learned my addr 172.21.15.29:0/3512341508 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:48.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c6aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 0x5650a7cd30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 msgr2=0x5650a7c6a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c6a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3512341508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 msgr2=0x5650a7cd30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 0x5650a7cd30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b27fc640 1 -- 172.21.15.29:0/3512341508 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a7cd3710 con 0x5650a7882390 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.329+0000 7fc2b37fe640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 0x5650a7cd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc2b27fc640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c6aa70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40095f0 tx=0x7fc2a4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da44eb565c5cbae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4046070 con 0x5650a7882390 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2a40035f0 con 0x5650a7882390 2024-09-21T01:51:48.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a4003aa0 con 0x5650a7882390 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4058070 con 0x5650a7882390 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc2bf9aa740 10 client.16092 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.330+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5650a7cdab00 con 0x5650a7882390 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.331+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a4003e50 con 0x5650a7882390 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.331+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fc2a404c440 con 0x5650a7882390 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.331+0000 7fc2bf9aa740 10 client.16092 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.332+0000 7fc2bf9aa740 10 client.16092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.332+0000 7fc2bf9aa740 20 client.16092 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.332+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a7836da0 0x5650a7cdd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.332+0000 7fc2bf9aa740 4 client.16092 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.332+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a7836da0 0x5650a7cdd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.333+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc2a4003aa0 con 0x5650a7836da0 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.334+0000 7fc2b2ffd640 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a7836da0 0x5650a7cdd190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1515824e1a12cc5 server_cookie=f4e7adbed8e4c38f in_seq=0 out_seq=0 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.334+0000 7fc296ff5640 10 client.16092 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc296ff5640 1 -- 172.21.15.29:0/3512341508 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc2a4003aa0 con 0x5650a7836da0 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc296ff5640 10 client.16092 handle_command_reply: tid=0 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 client.16092 shutdown 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 msgr2=0x5650a7c6aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c6aa70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40095f0 tx=0x7fc2a4005da0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 shutdown_connections 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650a78a3070 0x5650a7cd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5650a7882390 0x5650a7c6aa70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5650a7836da0 0x5650a7cdd190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 --2- 172.21.15.29:0/3512341508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650a787a1b0 0x5650a7c6a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 >> 172.21.15.29:0/3512341508 conn(0x5650a7897460 msgr2=0x5650a7c4bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc296ff5640 0 client.16092 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 shutdown_connections 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.335+0000 7fc2bf9aa740 1 -- 172.21.15.29:0/3512341508 wait complete. 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.336+0000 7fc2bf9aa740 20 client.16092 trim_cache size 0 max 16384 2024-09-21T01:51:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.336+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 msgr2=0x7fc29c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.336+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 0x7fc29c0b8aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009830 tx=0x7fc2a40056d0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 msgr2=0x7fc2b419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009fd0 tx=0x7fc2ac00c800 comp rx=0 tx=0).stop 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 shutdown_connections 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc29c0b6590 0x7fc29c0b8aa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b4109e00 0x7fc2b419fcf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2b41094b0 0x7fc2b419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 --2- 172.21.15.29:0/1432796417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2b4103b70 0x7fc2b419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 >> 172.21.15.29:0/1432796417 conn(0x7fc2b40ff5e0 msgr2=0x7fc2b4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 shutdown_connections 2024-09-21T01:51:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:48.337+0000 7fc2b9436640 1 -- 172.21.15.29:0/1432796417 wait complete. 2024-09-21T01:51:49.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:51:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:49 smithi190 ceph-mon[34368]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-21T01:51:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:49 smithi066 ceph-mon[35001]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-21T01:51:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:49 smithi029 ceph-mon[31228]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-21T01:51:50.100 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 -- 172.21.15.29:0/2532913447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 msgr2=0x7fc530102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 --2- 172.21.15.29:0/2532913447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc530102e70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc520009900 tx=0x7fc52002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 -- 172.21.15.29:0/2532913447 shutdown_connections 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 --2- 172.21.15.29:0/2532913447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc530108cf0 0x7fc53010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 --2- 172.21.15.29:0/2532913447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5301083a0 0x7fc5301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 --2- 172.21.15.29:0/2532913447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc530102e70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 -- 172.21.15.29:0/2532913447 >> 172.21.15.29:0/2532913447 conn(0x7fc530079730 msgr2=0x7fc530079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.692+0000 7fc534abd640 1 -- 172.21.15.29:0/2532913447 shutdown_connections 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.693+0000 7fc534abd640 1 -- 172.21.15.29:0/2532913447 wait complete. 2024-09-21T01:51:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.694+0000 7fc534abd640 1 Processor -- start 2024-09-21T01:51:50.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.695+0000 7fc534abd640 1 -- start start 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.695+0000 7fc534abd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc53019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.695+0000 7fc534abd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 0x7fc53019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc53019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc53019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44414/0 (socket says 172.21.15.29:44414) 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc534abd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 0x7fc53019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52e59c640 1 -- 172.21.15.29:0/651940775 learned_addr learned my addr 172.21.15.29:0/651940775 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 0x7fc53019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc530113a80 con 0x7fc5301083a0 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc530113900 con 0x7fc530102a60 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc530113c00 con 0x7fc530108cf0 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 0x7fc53019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 -- 172.21.15.29:0/651940775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 msgr2=0x7fc53019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc53019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 -- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 msgr2=0x7fc53019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 0x7fc53019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52ed9d640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5301a0400 con 0x7fc530108cf0 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.696+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 0x7fc53019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.697+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 0x7fc53019fcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc528009fd0 tx=0x7fc52800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.697+0000 7fc51f7fe640 1 -- 172.21.15.29:0/651940775 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52800ce20 con 0x7fc530108cf0 2024-09-21T01:51:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.697+0000 7fc51f7fe640 1 -- 172.21.15.29:0/651940775 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5280137d0 con 0x7fc530108cf0 2024-09-21T01:51:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.697+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5301a06f0 con 0x7fc530108cf0 2024-09-21T01:51:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.698+0000 7fc51f7fe640 1 -- 172.21.15.29:0/651940775 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528005510 con 0x7fc530108cf0 2024-09-21T01:51:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.699+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53010fe90 con 0x7fc530108cf0 2024-09-21T01:51:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.702+0000 7fc53b031740 1 Processor -- start 2024-09-21T01:51:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.703+0000 7fc51f7fe640 1 -- 172.21.15.29:0/651940775 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc5280057d0 con 0x7fc530108cf0 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 -- start start 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87761c390 0x55a8779ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a87795aea0 con 0x55a8776141b0 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a87794a2d0 con 0x55a87763d070 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc53b031740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a87794e0e0 con 0x55a87761c390 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44424/0 (socket says 172.21.15.29:44424) 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.704+0000 7fc52dd9b640 1 -- 172.21.15.29:0/1731625552 learned_addr learned my addr 172.21.15.29:0/1731625552 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52e59c640 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87761c390 0x55a8779ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 -- 172.21.15.29:0/1731625552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87761c390 msgr2=0x55a8779ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87761c390 0x55a8779ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 -- 172.21.15.29:0/1731625552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 msgr2=0x55a8779ed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779ed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 -- 172.21.15.29:0/1731625552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8776a8650 con 0x55a87763d070 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.705+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b620 tx=0x7fc52400bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a95470fb28427f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc51f7fe640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 0x7fc5180b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc51f7fe640 1 -- 172.21.15.29:0/651940775 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5280e75c0 con 0x7fc530108cf0 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc52e59c640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 0x7fc5180b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc51cff9640 1 -- 172.21.15.29:0/1731625552 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524004020 con 0x55a87763d070 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc51cff9640 1 -- 172.21.15.29:0/1731625552 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5240054d0 con 0x55a87763d070 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.706+0000 7fc51cff9640 1 -- 172.21.15.29:0/1731625552 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524010cf0 con 0x55a87763d070 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- 172.21.15.29:0/1731625552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 msgr2=0x55a8779ecad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b620 tx=0x7fc52400bb70 comp rx=0 tx=0).stop 2024-09-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc52e59c640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 0x7fc5180b8970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc520031440 tx=0x7fc5200058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- 172.21.15.29:0/1731625552 shutdown_connections 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87761c390 0x55a8779ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 --2- 172.21.15.29:0/1731625552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87763d070 0x55a8779ecad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- 172.21.15.29:0/1731625552 >> 172.21.15.29:0/1731625552 conn(0x55a877631460 msgr2=0x55a8779ea9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- 172.21.15.29:0/1731625552 shutdown_connections 2024-09-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- 172.21.15.29:0/1731625552 wait complete. 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.707+0000 7fc53b031740 1 -- start start 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.708+0000 7fc53b031740 1 Processor -- start 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779f8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a8777ab700 con 0x55a8776141b0 2024-09-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a8778e5160 con 0x55a87761c390 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8779fb2b0 con 0x55a87763d070 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc52ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc52ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48116/0 (socket says 172.21.15.29:48116) 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc52ed9d640 1 -- 172.21.15.29:0/1779090830 learned_addr learned my addr 172.21.15.29:0/1779090830 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc53b031740 10 client.0 _ll_register_callbacks cb 0x55a8778fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc52e59c640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.709+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 -- 172.21.15.29:0/1779090830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 msgr2=0x55a8779f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 -- 172.21.15.29:0/1779090830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 msgr2=0x55a8779f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 -- 172.21.15.29:0/1779090830 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a877912c90 con 0x55a87763d070 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c760 tx=0x7fc5280ad180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a958ad4154d1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280ad210 con 0x55a87763d070 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5280e72f0 con 0x55a87763d070 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5240054d0 con 0x55a87763d070 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.710+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc5280ade00 con 0x55a87763d070 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 10 client.25873 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:50.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a8779e4030 con 0x55a87763d070 2024-09-21T01:51:50.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5240054d0 con 0x55a87763d070 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fc528013980 con 0x55a87763d070 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 10 client.25873 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 10 client.25873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 20 client.25873 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775fb9f0 0x55a8779e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 4 client.25873 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87795cdf0 con 0x55a8775fb9f0 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.711+0000 7fc52e59c640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775fb9f0 0x55a8779e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.712+0000 7fc52e59c640 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775fb9f0 0x55a8779e7510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a8d19de41086dd5 server_cookie=8a7145047b119a5a in_seq=0 out_seq=0 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.712+0000 7fc515ffb640 10 client.25873 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.713+0000 7fc515ffb640 1 -- 172.21.15.29:0/1779090830 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a87795cdf0 con 0x55a8775fb9f0 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.713+0000 7fc515ffb640 10 client.25873 handle_command_reply: tid=0 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.713+0000 7fc53b031740 1 client.25873 shutdown 2024-09-21T01:51:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 msgr2=0x55a877a045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c760 tx=0x7fc5280ad180 comp rx=0 tx=0).stop 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 shutdown_connections 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775fb9f0 0x55a8779e7510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a045a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 --2- 172.21.15.29:0/1779090830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 >> 172.21.15.29:0/1779090830 conn(0x55a877631460 msgr2=0x55a8779efcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc515ffb640 0 client.25873 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 shutdown_connections 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 1 -- 172.21.15.29:0/1779090830 wait complete. 2024-09-21T01:51:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.714+0000 7fc53b031740 20 client.25873 trim_cache size 0 max 16384 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 Processor -- start 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 -- start start 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8776141b0 0x55a877a0a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a8779f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301ace60 con 0x55a87761c390 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301aea90 con 0x55a8776141b0 2024-09-21T01:51:50.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.738+0000 7fc53b031740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301ad530 con 0x55a87763d070 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.739+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.739+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38462/0 (socket says 172.21.15.29:38462) 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.739+0000 7fc52dd9b640 1 -- 172.21.15.29:0/3183213421 learned_addr learned my addr 172.21.15.29:0/3183213421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.739+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a8779f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 -- 172.21.15.29:0/3183213421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 msgr2=0x55a8779f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a8779f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 -- 172.21.15.29:0/3183213421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8776141b0 msgr2=0x55a877a0a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8776141b0 0x55a877a0a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 -- 172.21.15.29:0/3183213421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8779faa10 con 0x55a87761c390 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b620 tx=0x7fc52401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92031b7c4bf908ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.740+0000 7fc517fff640 1 -- 172.21.15.29:0/3183213421 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52402c070 con 0x55a87761c390 2024-09-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.741+0000 7fc517fff640 1 -- 172.21.15.29:0/3183213421 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc524016040 con 0x55a87761c390 2024-09-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.741+0000 7fc517fff640 1 -- 172.21.15.29:0/3183213421 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5240048c0 con 0x55a87761c390 2024-09-21T01:51:50.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 -- 172.21.15.29:0/3183213421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 msgr2=0x55a8779e1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b620 tx=0x7fc52401b040 comp rx=0 tx=0).stop 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 -- 172.21.15.29:0/3183213421 shutdown_connections 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a8779f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a87761c390 0x55a8779e1760 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 --2- 172.21.15.29:0/3183213421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8776141b0 0x55a877a0a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.742+0000 7fc53b031740 1 -- 172.21.15.29:0/3183213421 >> 172.21.15.29:0/3183213421 conn(0x55a877631460 msgr2=0x55a8779efbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.743+0000 7fc53b031740 1 -- 172.21.15.29:0/3183213421 shutdown_connections 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.743+0000 7fc53b031740 1 -- 172.21.15.29:0/3183213421 wait complete. 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.743+0000 7fc53b031740 1 -- start start 2024-09-21T01:51:50.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.743+0000 7fc53b031740 1 Processor -- start 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779fdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301ae300 con 0x55a8776141b0 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301adc00 con 0x55a87761c390 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301aae40 con 0x55a87763d070 2024-09-21T01:51:50.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc53b031740 10 client.0 _ll_register_callbacks cb 0x7fc528005ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48124/0 (socket says 172.21.15.29:48124) 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44446/0 (socket says 172.21.15.29:44446) 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52ed9d640 1 -- 172.21.15.29:0/2697624112 learned_addr learned my addr 172.21.15.29:0/2697624112 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.744+0000 7fc52e59c640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779fdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 -- 172.21.15.29:0/2697624112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 msgr2=0x55a8779fe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 -- 172.21.15.29:0/2697624112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 msgr2=0x55a8779fdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779fdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 -- 172.21.15.29:0/2697624112 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a877a65340 con 0x55a87763d070 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52dd9b640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:50.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc52ed9d640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5280aa970 tx=0x7fc528005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c374411cfcf3163 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280b6f10 con 0x55a87763d070 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5280b63c0 con 0x55a87763d070 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.745+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc524016040 con 0x55a87763d070 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.746+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc5280cf050 con 0x55a87763d070 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.746+0000 7fc53b031740 10 client.25879 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.746+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a877a6c640 con 0x55a87763d070 2024-09-21T01:51:50.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.746+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc524016040 con 0x55a87763d070 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fc5280b3370 con 0x55a87763d070 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 10 client.25879 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 10 client.25879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 20 client.25879 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775d0da0 0x55a877a6ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:50.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 4 client.25879 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:50.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.747+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8779588a0 con 0x55a8775d0da0 2024-09-21T01:51:50.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.748+0000 7fc52e59c640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775d0da0 0x55a877a6ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:50.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.748+0000 7fc515ffb640 10 client.25879 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:51:50.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.748+0000 7fc52e59c640 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775d0da0 0x55a877a6ed90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c550c9a76097558 server_cookie=35b88070f10fae11 in_seq=0 out_seq=0 2024-09-21T01:51:50.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.748+0000 7fc515ffb640 1 -- 172.21.15.29:0/2697624112 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8779588a0 con 0x55a8775d0da0 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc515ffb640 10 client.25879 handle_command_reply: tid=0 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 client.25879 shutdown 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 msgr2=0x55a877a64cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5280aa970 tx=0x7fc528005740 comp rx=0 tx=0).stop 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 shutdown_connections 2024-09-21T01:51:50.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55a8775d0da0 0x55a877a6ed90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a87763d070 0x55a877a64cd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc515ffb640 0 client.25879 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a87761c390 0x55a8779fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 --2- 172.21.15.29:0/2697624112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8776141b0 0x55a8779fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 >> 172.21.15.29:0/2697624112 conn(0x55a877631460 msgr2=0x55a8779f0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 shutdown_connections 2024-09-21T01:51:50.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 1 -- 172.21.15.29:0/2697624112 wait complete. 2024-09-21T01:51:50.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.749+0000 7fc53b031740 20 client.25879 trim_cache size 0 max 16384 2024-09-21T01:51:50.757 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:50.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 msgr2=0x7fc5180b8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 0x7fc5180b8970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc520031440 tx=0x7fc5200058d0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 msgr2=0x7fc53019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:50.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 0x7fc53019fcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc528009fd0 tx=0x7fc52800ef10 comp rx=0 tx=0).stop 2024-09-21T01:51:50.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 shutdown_connections 2024-09-21T01:51:50.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5180b6460 0x7fc5180b8970 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc520031440 tx=0x7fc5200058d0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc530108cf0 0x7fc53019fcd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5301083a0 0x7fc53019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 --2- 172.21.15.29:0/651940775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc530102a60 0x7fc53019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.750+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 >> 172.21.15.29:0/651940775 conn(0x7fc530079730 msgr2=0x7fc5301014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:50.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.752+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 shutdown_connections 2024-09-21T01:51:50.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:50.752+0000 7fc534abd640 1 -- 172.21.15.29:0/651940775 wait complete. 2024-09-21T01:51:51.696 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:51:51.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:51 smithi190 ceph-mon[34368]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-09-21T01:51:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:51 smithi066 ceph-mon[35001]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-09-21T01:51:52.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:51 smithi029 ceph-mon[31228]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-09-21T01:51:52.195 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.799+0000 7fdb28c21640 1 -- 172.21.15.29:0/2304790265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410a190 msgr2=0x7fdb2410a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.799+0000 7fdb28c21640 1 --2- 172.21.15.29:0/2304790265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410a190 0x7fdb2410a5a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009be0 tx=0x7fdb1402a6d0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 -- 172.21.15.29:0/2304790265 shutdown_connections 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 --2- 172.21.15.29:0/2304790265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb24110420 0x7fdb24116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 --2- 172.21.15.29:0/2304790265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2410fad0 0x7fdb2410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 --2- 172.21.15.29:0/2304790265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410a190 0x7fdb2410a5a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 -- 172.21.15.29:0/2304790265 >> 172.21.15.29:0/2304790265 conn(0x7fdb240780f0 msgr2=0x7fdb241081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 -- 172.21.15.29:0/2304790265 shutdown_connections 2024-09-21T01:51:52.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.800+0000 7fdb28c21640 1 -- 172.21.15.29:0/2304790265 wait complete. 2024-09-21T01:51:52.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.802+0000 7fdb28c21640 1 Processor -- start 2024-09-21T01:51:52.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.802+0000 7fdb28c21640 1 -- start start 2024-09-21T01:51:52.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 0x7fdb2406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 0x7fdb2406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2411a0d0 con 0x7fdb2410fad0 2024-09-21T01:51:52.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24119f50 con 0x7fdb24110420 2024-09-21T01:51:52.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.803+0000 7fdb28c21640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2411a250 con 0x7fdb2410a190 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:53862/0 (socket says 172.21.15.29:53862) 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 0x7fdb2406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 0x7fdb2406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb2259c640 1 -- 172.21.15.29:0/1572942520 learned_addr learned my addr 172.21.15.29:0/1572942520 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.804+0000 7fdb21d9b640 1 -- 172.21.15.29:0/1572942520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 msgr2=0x7fdb24074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb21d9b640 1 -- 172.21.15.29:0/1572942520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 msgr2=0x7fdb2406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 0x7fdb2406e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb21d9b640 1 -- 172.21.15.29:0/1572942520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2406ee90 con 0x7fdb2410fad0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb2259c640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 0x7fdb2406e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 0x7fdb2406e110 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406bb30 tx=0x7fdb1800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb137fe640 1 -- 172.21.15.29:0/1572942520 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18009e60 con 0x7fdb2410fad0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb24073b10 con 0x7fdb2410fad0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.805+0000 7fdb137fe640 1 -- 172.21.15.29:0/1572942520 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1800ccc0 con 0x7fdb2410fad0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.806+0000 7fdb137fe640 1 -- 172.21.15.29:0/1572942520 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18018740 con 0x7fdb2410fad0 2024-09-21T01:51:52.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.806+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb24073f90 con 0x7fdb2410fad0 2024-09-21T01:51:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.809+0000 7fdb2f195740 1 Processor -- start 2024-09-21T01:51:52.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2f195740 1 -- start start 2024-09-21T01:51:52.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff12b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff12b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff12bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff12b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff12b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51234/0 (socket says 172.21.15.29:51234) 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2259c640 1 -- 172.21.15.29:0/3361741641 learned_addr learned my addr 172.21.15.29:0/3361741641 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.810+0000 7fdb2f195740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ddff0a1ea0 con 0x55ddfed5b1b0 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.811+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ddff0912d0 con 0x55ddfed84070 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.811+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ddff0950e0 con 0x55ddfed63390 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.811+0000 7fdb137fe640 1 -- 172.21.15.29:0/1572942520 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdb180188e0 con 0x7fdb2410fad0 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.811+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff12b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.811+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff12bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 -- 172.21.15.29:0/3361741641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 msgr2=0x55ddff12bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff12bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 -- 172.21.15.29:0/3361741641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 msgr2=0x55ddff12b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff12b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 -- 172.21.15.29:0/3361741641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfedef650 con 0x55ddfed84070 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff12bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:52.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb137fe640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 0x7fdb0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb137fe640 1 -- 172.21.15.29:0/1572942520 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb180e75b0 con 0x7fdb2410fad0 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.813+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff12b2c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009f60 tx=0x7fdb1c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db0dbe9d5aa18d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.814+0000 7fdb2259c640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 0x7fdb0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.814+0000 7fdb10ff9640 1 -- 172.21.15.29:0/3361741641 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c01e070 con 0x55ddfed84070 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.814+0000 7fdb10ff9640 1 -- 172.21.15.29:0/3361741641 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1c004310 con 0x55ddfed84070 2024-09-21T01:51:52.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.814+0000 7fdb10ff9640 1 -- 172.21.15.29:0/3361741641 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00f880 con 0x55ddfed84070 2024-09-21T01:51:52.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 msgr2=0x55ddff12b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff12b2c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009f60 tx=0x7fdb1c009f90 comp rx=0 tx=0).stop 2024-09-21T01:51:52.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2259c640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 0x7fdb0c0b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ddff129f40 tx=0x7fdb140303a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 shutdown_connections 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff12bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff12b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3361741641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff12b2c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.815+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 >> 172.21.15.29:0/3361741641 conn(0x55ddfed78460 msgr2=0x55ddff1291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.816+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 shutdown_connections 2024-09-21T01:51:52.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.816+0000 7fdb2f195740 1 -- 172.21.15.29:0/3361741641 wait complete. 2024-09-21T01:51:52.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 -- start start 2024-09-21T01:51:52.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 Processor -- start 2024-09-21T01:51:52.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff142de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55444/0 (socket says 172.21.15.29:55444) 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2259c640 1 -- 172.21.15.29:0/3639081915 learned_addr learned my addr 172.21.15.29:0/3639081915 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ddfeef2700 con 0x55ddfed63390 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.817+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ddff02c160 con 0x55ddfed5b1b0 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ddff139af0 con 0x55ddfed84070 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb2f195740 10 client.0 _ll_register_callbacks cb 0x55ddff0459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 -- 172.21.15.29:0/3639081915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 msgr2=0x55ddff142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 -- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 msgr2=0x55ddff136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb21d9b640 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddff059c90 con 0x55ddfed63390 2024-09-21T01:51:52.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:52.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.818+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.819+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff136fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb180e5110 tx=0x7fdb18010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5622f5cecbc5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.819+0000 7fdb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180acec0 con 0x55ddfed63390 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb18018dd0 con 0x55ddfed63390 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1c004310 con 0x55ddfed63390 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb180ad260 con 0x55ddfed63390 2024-09-21T01:51:52.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb2f195740 10 client.16122 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.820+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ddff1457f0 con 0x55ddfed63390 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1c004310 con 0x55ddfed63390 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdb180a90a0 con 0x55ddfed63390 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 10 client.16122 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 20 client.16122 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed429f0 0x55ddff149160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 4 client.16122 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddff0a3df0 con 0x55ddfed429f0 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.821+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed429f0 0x55ddff149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.822+0000 7fdb09ffb640 10 client.16122 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:52.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.822+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed429f0 0x55ddff149160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bf87d983e19b5ed server_cookie=c42bae1d8ebcc6bd in_seq=0 out_seq=0 2024-09-21T01:51:52.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb09ffb640 1 -- 172.21.15.29:0/3639081915 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ddff0a3df0 con 0x55ddfed429f0 2024-09-21T01:51:52.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb09ffb640 10 client.16122 handle_command_reply: tid=0 2024-09-21T01:51:52.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 client.16122 shutdown 2024-09-21T01:51:52.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 msgr2=0x55ddff136fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff136fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb180e5110 tx=0x7fdb18010c30 comp rx=0 tx=0).stop 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 shutdown_connections 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed429f0 0x55ddff149160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff136fc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3639081915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.823+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 >> 172.21.15.29:0/3639081915 conn(0x55ddfed78460 msgr2=0x55ddff12e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.824+0000 7fdb09ffb640 0 client.16122 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.824+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 shutdown_connections 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.824+0000 7fdb2f195740 1 -- 172.21.15.29:0/3639081915 wait complete. 2024-09-21T01:51:52.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.824+0000 7fdb2f195740 20 client.16122 trim_cache size 0 max 16384 2024-09-21T01:51:52.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.839+0000 7fdb2f195740 1 Processor -- start 2024-09-21T01:51:52.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.839+0000 7fdb2f195740 1 -- start start 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff128af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff142ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51264/0 (socket says 172.21.15.29:51264) 2024-09-21T01:51:52.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb21d9b640 1 -- 172.21.15.29:0/3984166784 learned_addr learned my addr 172.21.15.29:0/3984166784 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:52.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241be2a0 con 0x55ddfed63390 2024-09-21T01:51:52.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241bfed0 con 0x55ddfed5b1b0 2024-09-21T01:51:52.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241be970 con 0x55ddfed84070 2024-09-21T01:51:52.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.840+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff128af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 -- 172.21.15.29:0/3984166784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 msgr2=0x55ddff128af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff128af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 -- 172.21.15.29:0/3984166784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 msgr2=0x55ddff142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 -- 172.21.15.29:0/3984166784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddff12d7e0 con 0x55ddfed5b1b0 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb2259c640 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136f40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55ddff1498d0 tx=0x7fdb14042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cc4473521455fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb0bfff640 1 -- 172.21.15.29:0/3984166784 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14036040 con 0x55ddfed5b1b0 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb0bfff640 1 -- 172.21.15.29:0/3984166784 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb14059070 con 0x55ddfed5b1b0 2024-09-21T01:51:52.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.841+0000 7fdb0bfff640 1 -- 172.21.15.29:0/3984166784 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14004780 con 0x55ddfed5b1b0 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.842+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 msgr2=0x55ddff136f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.842+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136f40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55ddff1498d0 tx=0x7fdb14042000 comp rx=0 tx=0).stop 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 shutdown_connections 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed84070 0x55ddff128af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed63390 0x55ddff142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 --2- 172.21.15.29:0/3984166784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed5b1b0 0x55ddff136f40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 >> 172.21.15.29:0/3984166784 conn(0x55ddfed78460 msgr2=0x55ddff136830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 shutdown_connections 2024-09-21T01:51:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.843+0000 7fdb2f195740 1 -- 172.21.15.29:0/3984166784 wait complete. 2024-09-21T01:51:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 -- start start 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 Processor -- start 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff13a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff1b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241bf740 con 0x55ddfed5b1b0 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241bf040 con 0x55ddfed84070 2024-09-21T01:51:52.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241bc280 con 0x55ddfed63390 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb2f195740 10 client.0 _ll_register_callbacks cb 0x7fdb18010cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.844+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:53904/0 (socket says 172.21.15.29:53904) 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 -- 172.21.15.29:0/368585654 learned_addr learned my addr 172.21.15.29:0/368585654 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb2259c640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff13a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb22d9d640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff1b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 -- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 msgr2=0x55ddff1b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff1b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 -- 172.21.15.29:0/368585654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 msgr2=0x55ddff13a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff13a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 -- 172.21.15.29:0/368585654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddff1b2090 con 0x55ddfed63390 2024-09-21T01:51:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.845+0000 7fdb21d9b640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ad400 tx=0x7fdb180e7300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d46921d936318d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.846+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180a9da0 con 0x55ddfed63390 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.846+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb18018bd0 con 0x55ddfed63390 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.846+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb14059070 con 0x55ddfed63390 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb180a95f0 con 0x55ddfed63390 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 10 client.25894 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ddff1b31f0 con 0x55ddfed63390 2024-09-21T01:51:52.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb14059070 con 0x55ddfed63390 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdb18010430 con 0x55ddfed63390 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 10 client.25894 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 10 client.25894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 20 client.25894 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed17da0 0x55ddff1b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 4 client.25894 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.847+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddff09f8a0 con 0x55ddfed17da0 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.848+0000 7fdb2259c640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed17da0 0x55ddff1b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.848+0000 7fdb09ffb640 10 client.25894 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.848+0000 7fdb2259c640 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed17da0 0x55ddff1b5850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d94d677b6c5c6af server_cookie=a0e98b2e85bdb930 in_seq=0 out_seq=0 2024-09-21T01:51:52.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.849+0000 7fdb09ffb640 1 -- 172.21.15.29:0/368585654 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddff09f8a0 con 0x55ddfed17da0 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.849+0000 7fdb09ffb640 10 client.25894 handle_command_reply: tid=0 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.849+0000 7fdb2f195740 1 client.25894 shutdown 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.849+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 msgr2=0x55ddff13a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.849+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ad400 tx=0x7fdb180e7300 comp rx=0 tx=0).stop 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 shutdown_connections 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55ddfed17da0 0x55ddff1b5850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddfed84070 0x55ddff1b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddfed63390 0x55ddff13a650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 --2- 172.21.15.29:0/368585654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ddfed5b1b0 0x55ddff13a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.850+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 >> 172.21.15.29:0/368585654 conn(0x55ddfed78460 msgr2=0x55ddff128ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.851+0000 7fdb09ffb640 0 client.25894 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:51:52.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.851+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 shutdown_connections 2024-09-21T01:51:52.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.851+0000 7fdb2f195740 1 -- 172.21.15.29:0/368585654 wait complete. 2024-09-21T01:51:52.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.851+0000 7fdb2f195740 20 client.25894 trim_cache size 0 max 16384 2024-09-21T01:51:52.854 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.852+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 msgr2=0x7fdb0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.852+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 0x7fdb0c0b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ddff129f40 tx=0x7fdb140303a0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.852+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 msgr2=0x7fdb2406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.852+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 0x7fdb2406e110 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406bb30 tx=0x7fdb1800ed90 comp rx=0 tx=0).stop 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 shutdown_connections 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb0c0b6510 0x7fdb0c0b8a20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb24110420 0x7fdb2406e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb2410fad0 0x7fdb2406e110 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 --2- 172.21.15.29:0/1572942520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb2410a190 0x7fdb24074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:52.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 >> 172.21.15.29:0/1572942520 conn(0x7fdb240780f0 msgr2=0x7fdb24108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:52.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.853+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 shutdown_connections 2024-09-21T01:51:52.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:52.854+0000 7fdb28c21640 1 -- 172.21.15.29:0/1572942520 wait complete. 2024-09-21T01:51:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 195.0 s, 3419.6 tps, lat 29.474 ms stddev 39.283 2024-09-21T01:51:53.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:51:53.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:53 smithi190 ceph-mon[34368]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-21T01:51:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:53 smithi066 ceph-mon[35001]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-21T01:51:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:53 smithi029 ceph-mon[31228]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-21T01:51:54.117 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:55.095 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 -- 172.21.15.29:0/2718971868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde801083d0 msgr2=0x7fde801087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 --2- 172.21.15.29:0/2718971868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde801083d0 0x7fde801087e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde74009900 tx=0x7fde7402a410 comp rx=0 tx=0).stop 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 -- 172.21.15.29:0/2718971868 shutdown_connections 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 --2- 172.21.15.29:0/2718971868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde8010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 --2- 172.21.15.29:0/2718971868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde801083d0 0x7fde801087e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 --2- 172.21.15.29:0/2718971868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde80102ae0 0x7fde80102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 -- 172.21.15.29:0/2718971868 >> 172.21.15.29:0/2718971868 conn(0x7fde80079760 msgr2=0x7fde80079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 -- 172.21.15.29:0/2718971868 shutdown_connections 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.678+0000 7fde86852640 1 -- 172.21.15.29:0/2718971868 wait complete. 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 Processor -- start 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 -- start start 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde801083d0 0x7fde800ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde801000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde80112990 con 0x7fde80102ae0 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde80112810 con 0x7fde801083d0 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde86852640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde80112b10 con 0x7fde80108d20 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51294/0 (socket says 172.21.15.29:51294) 2024-09-21T01:51:55.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde7ffff640 1 -- 172.21.15.29:0/2249946137 learned_addr learned my addr 172.21.15.29:0/2249946137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.679+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde801083d0 0x7fde800ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde84def640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde801000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 -- 172.21.15.29:0/2249946137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 msgr2=0x7fde801000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde801000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 -- 172.21.15.29:0/2249946137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde801083d0 msgr2=0x7fde800ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde801083d0 0x7fde800ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 -- 172.21.15.29:0/2249946137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde801a87d0 con 0x7fde80102ae0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde84def640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde801000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7ffff640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde7000e9f0 tx=0x7fde7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7d7fa640 1 -- 172.21.15.29:0/2249946137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7000cd80 con 0x7fde80102ae0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7d7fa640 1 -- 172.21.15.29:0/2249946137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde700108e0 con 0x7fde80102ae0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde7d7fa640 1 -- 172.21.15.29:0/2249946137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70018980 con 0x7fde80102ae0 2024-09-21T01:51:55.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde801a8ac0 con 0x7fde80102ae0 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.680+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde801a9020 con 0x7fde80102ae0 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde7d7fa640 1 -- 172.21.15.29:0/2249946137 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fde70010430 con 0x7fde80102ae0 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 Processor -- start 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 -- start start 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630cbebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cbf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561630c34ea0 con 0x561630917070 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561630c242d0 con 0x5616308ee1b0 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561630c280e0 con 0x5616308f6390 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51306/0 (socket says 172.21.15.29:51306) 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.683+0000 7fde84def640 1 -- 172.21.15.29:0/1636873299 learned_addr learned my addr 172.21.15.29:0/1636873299 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde7ffff640 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630cbebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cbf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 -- 172.21.15.29:0/1636873299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 msgr2=0x561630cbf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cbf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 -- 172.21.15.29:0/1636873299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 msgr2=0x561630cbebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630cbebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 -- 172.21.15.29:0/1636873299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561630982650 con 0x561630917070 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde84def640 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fde78009fd0 tx=0x7fde7800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b79a1742ec9b433 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde667fc640 1 -- 172.21.15.29:0/1636873299 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78015070 con 0x561630917070 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde667fc640 1 -- 172.21.15.29:0/1636873299 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde78011bb0 con 0x561630917070 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde667fc640 1 -- 172.21.15.29:0/1636873299 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78010bc0 con 0x561630917070 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde7d7fa640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 0x7fde680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.684+0000 7fde7ffff640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 0x7fde680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde7d7fa640 1 -- 172.21.15.29:0/2249946137 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde700e7610 con 0x7fde80102ae0 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 -- 172.21.15.29:0/1636873299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 msgr2=0x561630cbe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fde78009fd0 tx=0x7fde7800f7f0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde7ffff640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 0x7fde680b8aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x561630cbd320 tx=0x7fde700ed550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 -- 172.21.15.29:0/1636873299 shutdown_connections 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cbf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630cbebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/1636873299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561630917070 0x561630cbe6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 -- 172.21.15.29:0/1636873299 >> 172.21.15.29:0/1636873299 conn(0x56163090b460 msgr2=0x561630cbc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 -- 172.21.15.29:0/1636873299 shutdown_connections 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.685+0000 7fde8cdc6740 1 -- 172.21.15.29:0/1636873299 wait complete. 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 -- start start 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 Processor -- start 2024-09-21T01:51:55.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561630917070 0x561630cd61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561630a85700 con 0x5616308ee1b0 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561630bbf160 con 0x561630917070 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561630ccce80 con 0x5616308f6390 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde8cdc6740 10 client.0 _ll_register_callbacks cb 0x561630bd89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51322/0 (socket says 172.21.15.29:51322) 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.686+0000 7fde84def640 1 -- 172.21.15.29:0/2087636109 learned_addr learned my addr 172.21.15.29:0/2087636109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561630917070 0x561630cd61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 -- 172.21.15.29:0/2087636109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 msgr2=0x561630cca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 -- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561630917070 msgr2=0x561630cd61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561630917070 0x561630cd61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 -- 172.21.15.29:0/2087636109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561630becc90 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde84def640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde78009fa0 tx=0x7fde780108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33f5945bb01654a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78015050 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde780109a0 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde78011bb0 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde78004e50 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde8cdc6740 10 client.16140 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.687+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561630cd8b60 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde78011bb0 con 0x5616308ee1b0 2024-09-21T01:51:55.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fde78027020 con 0x5616308ee1b0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 10 client.16140 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 20 client.16140 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5616308d59f0 0x561630cdbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 4 client.16140 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561630c36df0 con 0x5616308d59f0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5616308d59f0 0x561630cdbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde5f7fe640 10 client.16140 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.688+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5616308d59f0 0x561630cdbf70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f8ef39545f84bbd server_cookie=5fb7db59c065decc in_seq=0 out_seq=0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde5f7fe640 1 -- 172.21.15.29:0/2087636109 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561630c36df0 con 0x5616308d59f0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde5f7fe640 10 client.16140 handle_command_reply: tid=0 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 client.16140 shutdown 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 msgr2=0x561630cc9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde78009fa0 tx=0x7fde780108c0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 shutdown_connections 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5616308d59f0 0x561630cdbf70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde5f7fe640 0 client.16140 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561630917070 0x561630cd61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/2087636109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cc9d60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.689+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 >> 172.21.15.29:0/2087636109 conn(0x56163090b460 msgr2=0x561630cc1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.690+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 shutdown_connections 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.690+0000 7fde8cdc6740 1 -- 172.21.15.29:0/2087636109 wait complete. 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.690+0000 7fde8cdc6740 20 client.16140 trim_cache size 0 max 16384 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 Processor -- start 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 -- start start 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630ccbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde801b1690 con 0x5616308d59f0 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde801b32c0 con 0x5616308ee1b0 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde801b1d60 con 0x5616308f6390 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde84def640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51326/0 (socket says 172.21.15.29:51326) 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.708+0000 7fde84def640 1 -- 172.21.15.29:0/501004871 learned_addr learned my addr 172.21.15.29:0/501004871 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630ccbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 -- 172.21.15.29:0/501004871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 msgr2=0x561630cd60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cd60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 -- 172.21.15.29:0/501004871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 msgr2=0x561630ccbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630ccbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 -- 172.21.15.29:0/501004871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561630ccac90 con 0x5616308d59f0 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde84def640 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fde78002230 tx=0x7fde7800ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87df7eba247626ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde65ffb640 1 -- 172.21.15.29:0/501004871 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78009ae0 con 0x5616308d59f0 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde65ffb640 1 -- 172.21.15.29:0/501004871 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7803a070 con 0x5616308d59f0 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.709+0000 7fde65ffb640 1 -- 172.21.15.29:0/501004871 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78010930 con 0x5616308d59f0 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 -- 172.21.15.29:0/501004871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 msgr2=0x561630cc01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fde78002230 tx=0x7fde7800ff60 comp rx=0 tx=0).stop 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 -- 172.21.15.29:0/501004871 shutdown_connections 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630cd60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308ee1b0 0x561630ccbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/501004871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308d59f0 0x561630cc01a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 -- 172.21.15.29:0/501004871 >> 172.21.15.29:0/501004871 conn(0x56163090b460 msgr2=0x561630cc37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 -- 172.21.15.29:0/501004871 shutdown_connections 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.710+0000 7fde8cdc6740 1 -- 172.21.15.29:0/501004871 wait complete. 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 -- start start 2024-09-21T01:51:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 Processor -- start 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 0x561630cce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630ccec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde801b2b30 con 0x5616308ee1b0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde801b2430 con 0x5616308d59f0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde801af670 con 0x5616308f6390 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.711+0000 7fde8cdc6740 10 client.0 _ll_register_callbacks cb 0x7fde700ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51332/0 (socket says 172.21.15.29:51332) 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 -- 172.21.15.29:0/3139675679 learned_addr learned my addr 172.21.15.29:0/3139675679 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde84def640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630ccec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde84def640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 0x561630cce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 -- 172.21.15.29:0/3139675679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 msgr2=0x561630ccec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630ccec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 -- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 msgr2=0x561630cce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 0x561630cce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 -- 172.21.15.29:0/3139675679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561630d45890 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde84def640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 0x561630cce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde7f7fe640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde74005940 tx=0x7fde74004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7ffc7c2f34cd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74035040 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde74005240 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7803a070 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde74005440 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.712+0000 7fde8cdc6740 10 client.16146 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561630d46b70 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7803a070 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fde74003920 con 0x5616308ee1b0 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 10 client.16146 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 20 client.16146 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561630917070 0x561630d493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 4 client.16146 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561630c328a0 con 0x561630917070 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.713+0000 7fde84def640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561630917070 0x561630d493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde84def640 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561630917070 0x561630d493b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b98d77f909a47a server_cookie=c151ff98578200b9 in_seq=0 out_seq=0 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde5f7fe640 10 client.16146 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde5f7fe640 1 -- 172.21.15.29:0/3139675679 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561630c328a0 con 0x561630917070 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde5f7fe640 10 client.16146 handle_command_reply: tid=0 2024-09-21T01:51:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 client.16146 shutdown 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 msgr2=0x561630cce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde74005940 tx=0x7fde74004230 comp rx=0 tx=0).stop 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 shutdown_connections 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561630917070 0x561630d493b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616308f6390 0x561630ccec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5616308ee1b0 0x561630cce750 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde5f7fe640 0 client.16146 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 --2- 172.21.15.29:0/3139675679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616308d59f0 0x561630cce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 >> 172.21.15.29:0/3139675679 conn(0x56163090b460 msgr2=0x561630ccb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 shutdown_connections 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 1 -- 172.21.15.29:0/3139675679 wait complete. 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.714+0000 7fde8cdc6740 20 client.16146 trim_cache size 0 max 16384 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 msgr2=0x7fde680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 0x7fde680b8aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x561630cbd320 tx=0x7fde700ed550 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 msgr2=0x7fde800ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde7000e9f0 tx=0x7fde7000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 shutdown_connections 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde680b6590 0x7fde680b8aa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde80108d20 0x7fde801000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde801083d0 0x7fde800ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 --2- 172.21.15.29:0/2249946137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde80102ae0 0x7fde800ff670 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 >> 172.21.15.29:0/2249946137 conn(0x7fde80079760 msgr2=0x7fde8006bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 shutdown_connections 2024-09-21T01:51:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:54.715+0000 7fde86852640 1 -- 172.21.15.29:0/2249946137 wait complete. 2024-09-21T01:51:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.256+0000 7fbb86840640 1 -- 172.21.15.29:0/892005675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb801025a0 msgr2=0x7fbb80102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.256+0000 7fbb86840640 1 --2- 172.21.15.29:0/892005675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb801025a0 0x7fbb80102a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009900 tx=0x7fbb7002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 -- 172.21.15.29:0/892005675 shutdown_connections 2024-09-21T01:51:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 --2- 172.21.15.29:0/892005675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb80103fc0 0x7fbb8006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 --2- 172.21.15.29:0/892005675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103620 0x7fbb80103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 --2- 172.21.15.29:0/892005675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb801025a0 0x7fbb80102a10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 -- 172.21.15.29:0/892005675 >> 172.21.15.29:0/892005675 conn(0x7fbb800fdee0 msgr2=0x7fbb80100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 -- 172.21.15.29:0/892005675 shutdown_connections 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.257+0000 7fbb86840640 1 -- 172.21.15.29:0/892005675 wait complete. 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 Processor -- start 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 -- start start 2024-09-21T01:51:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb801025a0 0x7fbb8019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 0x7fbb8019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb801115a0 con 0x7fbb80103620 2024-09-21T01:51:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb80111420 con 0x7fbb80103fc0 2024-09-21T01:51:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.258+0000 7fbb86840640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb80111720 con 0x7fbb801025a0 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55538/0 (socket says 172.21.15.29:55538) 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 -- 172.21.15.29:0/4036621979 learned_addr learned my addr 172.21.15.29:0/4036621979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 0x7fbb8019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 -- 172.21.15.29:0/4036621979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb801025a0 msgr2=0x7fbb8019c150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb801025a0 0x7fbb8019c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 -- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 msgr2=0x7fbb8019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 0x7fbb8019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb84ddd640 1 -- 172.21.15.29:0/4036621979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8019db90 con 0x7fbb80103fc0 2024-09-21T01:51:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.259+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 0x7fbb8019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:51:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800eb40 tx=0x7fbb7800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb7d7fa640 1 -- 172.21.15.29:0/4036621979 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78010430 con 0x7fbb80103fc0 2024-09-21T01:51:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8019de20 con 0x7fbb80103fc0 2024-09-21T01:51:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8006cfc0 con 0x7fbb80103fc0 2024-09-21T01:51:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb7d7fa640 1 -- 172.21.15.29:0/4036621979 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb78010a80 con 0x7fbb80103fc0 2024-09-21T01:51:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.260+0000 7fbb7d7fa640 1 -- 172.21.15.29:0/4036621979 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78018b60 con 0x7fbb80103fc0 2024-09-21T01:51:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.264+0000 7fbb8d889740 1 Processor -- start 2024-09-21T01:51:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.264+0000 7fbb7d7fa640 1 -- 172.21.15.29:0/4036621979 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbb78018d00 con 0x7fbb80103fc0 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.264+0000 7fbb8d889740 1 -- start start 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaee9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fcfae526f0 con 0x55fcfab0f2f0 2024-09-21T01:51:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fcfae41b20 con 0x55fcfab174d0 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb8d889740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fcfae45930 con 0x55fcfab381b0 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:53996/0 (socket says 172.21.15.29:53996) 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/4091859941 learned_addr learned my addr 172.21.15.29:0/4091859941 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaee9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/4091859941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 msgr2=0x55fcfaee9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaee9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/4091859941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 msgr2=0x55fcfaee8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.265+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/4091859941 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcfae07530 con 0x55fcfab381b0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb7d7fa640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 0x7fbb680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb74014940 tx=0x7fbb74009690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4497ace6f1537cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 0x7fbb680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb7d7fa640 1 -- 172.21.15.29:0/4036621979 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb780e7950 con 0x7fbb80103fc0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb667fc640 1 -- 172.21.15.29:0/4091859941 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74004030 con 0x55fcfab381b0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb667fc640 1 -- 172.21.15.29:0/4091859941 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb7400fc10 con 0x55fcfab381b0 2024-09-21T01:51:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.266+0000 7fbb667fc640 1 -- 172.21.15.29:0/4091859941 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb740035f0 con 0x55fcfab381b0 2024-09-21T01:51:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.267+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 0x7fbb680b8aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb70031500 tx=0x7fbb70000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 -- 172.21.15.29:0/4091859941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 msgr2=0x55fcfaee8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb74014940 tx=0x7fbb74009690 comp rx=0 tx=0).stop 2024-09-21T01:51:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 -- 172.21.15.29:0/4091859941 shutdown_connections 2024-09-21T01:51:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaee9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 --2- 172.21.15.29:0/4091859941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab381b0 0x55fcfaee8660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 -- 172.21.15.29:0/4091859941 >> 172.21.15.29:0/4091859941 conn(0x55fcfab2c300 msgr2=0x55fcfaee40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 -- 172.21.15.29:0/4091859941 shutdown_connections 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.268+0000 7fbb8d889740 1 -- 172.21.15.29:0/4091859941 wait complete. 2024-09-21T01:51:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.269+0000 7fbb8d889740 1 -- start start 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.269+0000 7fbb8d889740 1 Processor -- start 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab174d0 0x55fcfaef24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fcfad79ab0 con 0x55fcfab0f2f0 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fcfade7730 con 0x55fcfab381b0 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fcfaeea2e0 con 0x55fcfab174d0 2024-09-21T01:51:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb8d889740 10 client.0 _ll_register_callbacks cb 0x55fcfae3b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb84ddd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb84ddd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55548/0 (socket says 172.21.15.29:55548) 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.270+0000 7fbb84ddd640 1 -- 172.21.15.29:0/110406597 learned_addr learned my addr 172.21.15.29:0/110406597 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 -- 172.21.15.29:0/110406597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab174d0 msgr2=0x55fcfaef24e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab174d0 0x55fcfaef24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 -- 172.21.15.29:0/110406597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 msgr2=0x55fcfaef1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 -- 172.21.15.29:0/110406597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcfaba3790 con 0x55fcfab381b0 2024-09-21T01:51:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ad790 tx=0x7fbb780b73e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b157da3f3816164f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb780acef0 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb780b20e0 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb7400fc10 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbb780ad5a0 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb8d889740 10 client.25871 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.271+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fcfac1c620 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb780ae190 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fbb780ac760 con 0x55fcfab381b0 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 10 client.25871 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 20 client.25871 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaaf6d80 0x55fcfaeff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 4 client.25871 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb7400fc10 con 0x55fcfaaf6d80 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.272+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaaf6d80 0x55fcfaeff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.273+0000 7fbb5f7fe640 10 client.25871 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.273+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaaf6d80 0x55fcfaeff780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752ecd45d293e8ae server_cookie=e18d100e6201878a in_seq=0 out_seq=0 2024-09-21T01:51:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.274+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/110406597 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbb7400fc10 con 0x55fcfaaf6d80 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.274+0000 7fbb5f7fe640 10 client.25871 handle_command_reply: tid=0 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.274+0000 7fbb8d889740 1 client.25871 shutdown 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.274+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 msgr2=0x55fcfaef3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.274+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ad790 tx=0x7fbb780b73e0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 shutdown_connections 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaaf6d80 0x55fcfaeff780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab381b0 0x55fcfaef3050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab174d0 0x55fcfaef24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb5f7fe640 0 client.25871 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 --2- 172.21.15.29:0/110406597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 >> 172.21.15.29:0/110406597 conn(0x55fcfab2c300 msgr2=0x55fcfaed9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 shutdown_connections 2024-09-21T01:51:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 1 -- 172.21.15.29:0/110406597 wait complete. 2024-09-21T01:51:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.275+0000 7fbb8d889740 20 client.25871 trim_cache size 0 max 16384 2024-09-21T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.303+0000 7fbb8d889740 1 Processor -- start 2024-09-21T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.303+0000 7fbb8d889740 1 -- start start 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.303+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.303+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.303+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaed65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb8d889740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00e420 con 0x55fcfab381b0 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb8d889740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c010050 con 0x55fcfab174d0 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb8d889740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00eaf0 con 0x55fcfab0f2f0 2024-09-21T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaed65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55558/0 (socket says 172.21.15.29:55558) 2024-09-21T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/3217347747 learned_addr learned my addr 172.21.15.29:0/3217347747 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 -- 172.21.15.29:0/3217347747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 msgr2=0x55fcfaee95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 -- 172.21.15.29:0/3217347747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 msgr2=0x55fcfaef9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.304+0000 7fbb84ddd640 1 -- 172.21.15.29:0/3217347747 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcfaef2810 con 0x55fcfab381b0 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.305+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.305+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaed65c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ad740 tx=0x7fbb780ad8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e64950a21722ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.305+0000 7fbb65ffb640 1 -- 172.21.15.29:0/3217347747 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb780b2570 con 0x55fcfab381b0 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.306+0000 7fbb65ffb640 1 -- 172.21.15.29:0/3217347747 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb780b74b0 con 0x55fcfab381b0 2024-09-21T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.306+0000 7fbb65ffb640 1 -- 172.21.15.29:0/3217347747 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb780a9760 con 0x55fcfab381b0 2024-09-21T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 -- 172.21.15.29:0/3217347747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 msgr2=0x55fcfaed65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaed65c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ad740 tx=0x7fbb780ad8f0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 -- 172.21.15.29:0/3217347747 shutdown_connections 2024-09-21T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaed65c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3217347747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaee95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 -- 172.21.15.29:0/3217347747 >> 172.21.15.29:0/3217347747 conn(0x55fcfab2c300 msgr2=0x55fcfaef1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 -- 172.21.15.29:0/3217347747 shutdown_connections 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.307+0000 7fbb8d889740 1 -- 172.21.15.29:0/3217347747 wait complete. 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.308+0000 7fbb8d889740 1 -- start start 2024-09-21T01:51:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.308+0000 7fbb8d889740 1 Processor -- start 2024-09-21T01:51:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaef8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00f8c0 con 0x55fcfab381b0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00f1c0 con 0x55fcfab174d0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00c400 con 0x55fcfab0f2f0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb8d889740 10 client.0 _ll_register_callbacks cb 0x7fbb780ad3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55562/0 (socket says 172.21.15.29:55562) 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/3594330798 learned_addr learned my addr 172.21.15.29:0/3594330798 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb84ddd640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaef8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/3594330798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 msgr2=0x55fcfaef9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/3594330798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 msgr2=0x55fcfaef8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaef8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.309+0000 7fbb7f7fe640 1 -- 172.21.15.29:0/3594330798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcfaef8df0 con 0x55fcfab174d0 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.310+0000 7fbb7f7fe640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005870 tx=0x7fbb740054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98385a2fd673098f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.310+0000 7fbb5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.310+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7402c060 con 0x55fcfab174d0 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.310+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcfaee9ca0 con 0x55fcfab174d0 2024-09-21T01:51:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.310+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb7400f930 con 0x55fcfab174d0 2024-09-21T01:51:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbb74005b90 con 0x55fcfab174d0 2024-09-21T01:51:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb8d889740 10 client.25877 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fcfaef7c30 con 0x55fcfab174d0 2024-09-21T01:51:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fcfaee9ca0 con 0x55fcfab174d0 2024-09-21T01:51:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fbb74003820 con 0x55fcfab174d0 2024-09-21T01:51:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb8d889740 10 client.25877 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb8d889740 10 client.25877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.312+0000 7fbb8d889740 20 client.25877 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.313+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaacc1c0 0x55fcfaf6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.313+0000 7fbb8d889740 4 client.25877 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.313+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcfae54640 con 0x55fcfaacc1c0 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.313+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaacc1c0 0x55fcfaf6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.314+0000 7fbb5f7fe640 10 client.25877 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.314+0000 7fbb7ffff640 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaacc1c0 0x55fcfaf6c4d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f58895bbde3e587 server_cookie=6fbdaed4dd4d436f in_seq=0 out_seq=0 2024-09-21T01:51:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.370+0000 7fbb5f7fe640 1 -- 172.21.15.29:0/3594330798 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcfae54640 con 0x55fcfaacc1c0 2024-09-21T01:51:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.370+0000 7fbb5f7fe640 10 client.25877 handle_command_reply: tid=0 2024-09-21T01:51:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.370+0000 7fbb8d889740 1 client.25877 shutdown 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.371+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 msgr2=0x55fcfaef9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.371+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005870 tx=0x7fbb740054d0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.371+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 shutdown_connections 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.371+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fcfaacc1c0 0x55fcfaf6c4d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcfab381b0 0x55fcfaef8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb5f7fe640 0 client.25877 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcfab174d0 0x55fcfaef9f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 --2- 172.21.15.29:0/3594330798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcfab0f2f0 0x55fcfaef9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 >> 172.21.15.29:0/3594330798 conn(0x55fcfab2c300 msgr2=0x55fcfaef1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 shutdown_connections 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 1 -- 172.21.15.29:0/3594330798 wait complete. 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.372+0000 7fbb8d889740 20 client.25877 trim_cache size 0 max 16384 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:51:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:51:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:51:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.373+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 msgr2=0x7fbb680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.373+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 0x7fbb680b8aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb70031500 tx=0x7fbb70000bc0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 msgr2=0x7fbb8019d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800eb40 tx=0x7fbb7800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 shutdown_connections 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb680b6590 0x7fbb680b8aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb80103fc0 0x7fbb8019d370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb80103620 0x7fbb8019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.374+0000 7fbb86840640 1 --2- 172.21.15.29:0/4036621979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb801025a0 0x7fbb8019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.375+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 >> 172.21.15.29:0/4036621979 conn(0x7fbb800fdee0 msgr2=0x7fbb80104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.376+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 shutdown_connections 2024-09-21T01:51:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:51:55.376+0000 7fbb86840640 1 -- 172.21.15.29:0/4036621979 wait complete. 2024-09-21T01:51:55.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:55 smithi029 ceph-mon[31228]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 802 op/s 2024-09-21T01:51:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:55 smithi190 ceph-mon[34368]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 802 op/s 2024-09-21T01:51:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:55 smithi066 ceph-mon[35001]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 802 op/s 2024-09-21T01:51:57.066 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:51:57.066 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:51:57.465 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 200.0 s, 3694.4 tps, lat 27.048 ms stddev 32.230 2024-09-21T01:51:57.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:57 smithi190 ceph-mon[34368]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T01:51:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:57 smithi066 ceph-mon[35001]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T01:51:58.092 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:57 smithi029 ceph-mon[31228]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.403+0000 7fabad92b640 1 -- 172.21.15.29:0/1987975874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8103160 msgr2=0x7faba8103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.403+0000 7fabad92b640 1 --2- 172.21.15.29:0/1987975874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8103160 0x7faba8103570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab98009900 tx=0x7fab9802a410 comp rx=0 tx=0).stop 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.403+0000 7fabad92b640 1 -- 172.21.15.29:0/1987975874 shutdown_connections 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.403+0000 7fabad92b640 1 --2- 172.21.15.29:0/1987975874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8105ac0 0x7faba810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.404+0000 7fabad92b640 1 --2- 172.21.15.29:0/1987975874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 0x7faba8105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.404+0000 7fabad92b640 1 --2- 172.21.15.29:0/1987975874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8103160 0x7faba8103570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.404+0000 7fabad92b640 1 -- 172.21.15.29:0/1987975874 >> 172.21.15.29:0/1987975874 conn(0x7faba8077c60 msgr2=0x7faba80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.404+0000 7fabad92b640 1 -- 172.21.15.29:0/1987975874 shutdown_connections 2024-09-21T01:51:58.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.405+0000 7fabad92b640 1 -- 172.21.15.29:0/1987975874 wait complete. 2024-09-21T01:51:58.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.406+0000 7fabad92b640 1 Processor -- start 2024-09-21T01:51:58.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.406+0000 7fabad92b640 1 -- start start 2024-09-21T01:51:58.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.407+0000 7fabad92b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.407+0000 7fabad92b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 0x7faba8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.407+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.407+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55582/0 (socket says 172.21.15.29:55582) 2024-09-21T01:51:58.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7faba6ffd640 1 -- 172.21.15.29:0/501801557 learned_addr learned my addr 172.21.15.29:0/501801557 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:58.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 0x7faba819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7faba67fc640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 0x7faba8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faba81129c0 con 0x7faba8105170 2024-09-21T01:51:58.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba8112840 con 0x7faba8103160 2024-09-21T01:51:58.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faba8112b40 con 0x7faba8105ac0 2024-09-21T01:51:58.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.408+0000 7faba77fe640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 0x7faba819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 -- 172.21.15.29:0/501801557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 msgr2=0x7faba8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 -- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 msgr2=0x7faba8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 0x7faba8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba819f2e0 con 0x7faba8105ac0 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba6ffd640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7faba77fe640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 0x7faba819ebb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7fabac929640 1 -- 172.21.15.29:0/501801557 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba000be40 con 0x7faba8105ac0 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba819f5d0 con 0x7faba8105ac0 2024-09-21T01:51:58.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.409+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba8101dc0 con 0x7faba8105ac0 2024-09-21T01:51:58.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.410+0000 7fabac929640 1 -- 172.21.15.29:0/501801557 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0017450 con 0x7faba8105ac0 2024-09-21T01:51:58.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.410+0000 7fabac929640 1 -- 172.21.15.29:0/501801557 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0005280 con 0x7faba8105ac0 2024-09-21T01:51:58.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.413+0000 7fabb3e9f740 1 Processor -- start 2024-09-21T01:51:58.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.415+0000 7fabac929640 1 -- 172.21.15.29:0/501801557 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7faba0004a40 con 0x7faba8105ac0 2024-09-21T01:51:58.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.415+0000 7fabb3e9f740 1 -- start start 2024-09-21T01:51:58.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb7e350 0x55b7bcf29840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf29d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 0x55b7bcf2a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7bce9be60 con 0x55b7bcb55490 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7bce8b290 con 0x55b7bcb7e350 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b7bce8f0a0 con 0x55b7bcb5d670 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf29d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf29d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51424/0 (socket says 172.21.15.29:51424) 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7faba6ffd640 1 -- 172.21.15.29:0/2528232580 learned_addr learned my addr 172.21.15.29:0/2528232580 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7faba67fc640 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 0x55b7bcf2a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.416+0000 7faba67fc640 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb7e350 0x55b7bcf29840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 -- 172.21.15.29:0/2528232580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb7e350 msgr2=0x55b7bcf29840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb7e350 0x55b7bcf29840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 -- 172.21.15.29:0/2528232580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 msgr2=0x55b7bcf29d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf29d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 -- 172.21.15.29:0/2528232580 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bcbe9930 con 0x55b7bcb5d670 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7fabac929640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 0x7fab900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.418+0000 7faba67fc640 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 0x55b7bcf2a2c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097e0 tx=0x7fab9c009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbe4693a3206d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.419+0000 7fabac929640 1 -- 172.21.15.29:0/501801557 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faba00e6bb0 con 0x7faba8105ac0 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.419+0000 7fab8dffb640 1 -- 172.21.15.29:0/2528232580 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004030 con 0x55b7bcb5d670 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.419+0000 7fab8dffb640 1 -- 172.21.15.29:0/2528232580 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c005020 con 0x55b7bcb5d670 2024-09-21T01:51:58.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.419+0000 7fab8dffb640 1 -- 172.21.15.29:0/2528232580 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004a90 con 0x55b7bcb5d670 2024-09-21T01:51:58.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.419+0000 7faba6ffd640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 0x7fab900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.420+0000 7fabb3e9f740 1 -- 172.21.15.29:0/2528232580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 msgr2=0x55b7bcf2a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.420+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 0x55b7bcf2a2c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097e0 tx=0x7fab9c009770 comp rx=0 tx=0).stop 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.420+0000 7faba6ffd640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 0x7fab900b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fab980096c0 tx=0x7fab980058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 -- 172.21.15.29:0/2528232580 shutdown_connections 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb5d670 0x55b7bcf2a2c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf29d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/2528232580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb7e350 0x55b7bcf29840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 -- 172.21.15.29:0/2528232580 >> 172.21.15.29:0/2528232580 conn(0x55b7bcb72740 msgr2=0x55b7bcf27730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 -- 172.21.15.29:0/2528232580 shutdown_connections 2024-09-21T01:51:58.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.421+0000 7fabb3e9f740 1 -- 172.21.15.29:0/2528232580 wait complete. 2024-09-21T01:51:58.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.422+0000 7fabb3e9f740 1 -- start start 2024-09-21T01:51:58.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.422+0000 7fabb3e9f740 1 Processor -- start 2024-09-21T01:51:58.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf24490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7bccec890 con 0x55b7bcb55490 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7bce25d40 con 0x55b7bcb5d670 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b7bcf3c400 con 0x55b7bcb7e350 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7fabb3e9f740 10 client.0 _ll_register_callbacks cb 0x55b7bce89c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7faba77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7faba77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54072/0 (socket says 172.21.15.29:54072) 2024-09-21T01:51:58.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7faba77fe640 1 -- 172.21.15.29:0/1848554549 learned_addr learned my addr 172.21.15.29:0/1848554549 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.423+0000 7faba6ffd640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf24490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.424+0000 7faba77fe640 1 -- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 msgr2=0x55b7bcf249d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.424+0000 7faba77fe640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.424+0000 7faba77fe640 1 -- 172.21.15.29:0/1848554549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 msgr2=0x55b7bcf24490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.424+0000 7faba77fe640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf24490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.424+0000 7faba77fe640 1 -- 172.21.15.29:0/1848554549 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bcf45cb0 con 0x55b7bcb7e350 2024-09-21T01:51:58.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7faba77fe640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9370 tx=0x7faba00ac600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7640bb3151cb52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fab86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0092dc0 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7faba00d00d0 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9c005020 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faba00b6710 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fabb3e9f740 10 client.25933 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.425+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7bcf48630 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c005020 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7faba00a86f0 con 0x55b7bcb7e350 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 10 client.25933 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 20 client.25933 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb3ccd0 0x55b7bcf4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 4 client.25933 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:51:58.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7bce9ddb0 con 0x55b7bcb3ccd0 2024-09-21T01:51:58.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.426+0000 7faba6ffd640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb3ccd0 0x55b7bcf4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.428+0000 7fab86ffd640 10 client.25933 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:58.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.428+0000 7faba6ffd640 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb3ccd0 0x55b7bcf4bb70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c5962770f55ffe server_cookie=89bd19dd94e82b5b in_seq=0 out_seq=0 2024-09-21T01:51:58.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.428+0000 7fab86ffd640 1 -- 172.21.15.29:0/1848554549 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7bce9ddb0 con 0x55b7bcb3ccd0 2024-09-21T01:51:58.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.428+0000 7fab86ffd640 10 client.25933 handle_command_reply: tid=0 2024-09-21T01:51:58.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.429+0000 7fabb3e9f740 1 client.25933 shutdown 2024-09-21T01:51:58.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.429+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 msgr2=0x55b7bcf456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.429+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9370 tx=0x7faba00ac600 comp rx=0 tx=0).stop 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 shutdown_connections 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb3ccd0 0x55b7bcf4bb70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf456f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf249d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1848554549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb55490 0x55b7bcf24490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.430+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 >> 172.21.15.29:0/1848554549 conn(0x55b7bcb72740 msgr2=0x55b7bcf2ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.431+0000 7fab86ffd640 0 client.25933 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:58.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.431+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 shutdown_connections 2024-09-21T01:51:58.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.431+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1848554549 wait complete. 2024-09-21T01:51:58.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.432+0000 7fabb3e9f740 20 client.25933 trim_cache size 0 max 16384 2024-09-21T01:51:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 Processor -- start 2024-09-21T01:51:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 -- start start 2024-09-21T01:51:58.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb55490 0x55b7bcf455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 0x55b7bcf22b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7faba67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 0x55b7bcf22b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7faba67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 0x55b7bcf22b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51442/0 (socket says 172.21.15.29:51442) 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb55490 0x55b7bcf455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7faba67fc640 1 -- 172.21.15.29:0/1700284596 learned_addr learned my addr 172.21.15.29:0/1700284596 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faba81ad240 con 0x55b7bcb5d670 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba81aee70 con 0x55b7bcb55490 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.463+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faba81ad910 con 0x55b7bcb7e350 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf23070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 -- 172.21.15.29:0/1700284596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb55490 msgr2=0x55b7bcf455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb55490 0x55b7bcf455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 -- 172.21.15.29:0/1700284596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 msgr2=0x55b7bcf22b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 0x55b7bcf22b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 -- 172.21.15.29:0/1700284596 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bcf2f970 con 0x55b7bcb7e350 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.464+0000 7faba77fe640 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf23070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9370 tx=0x7faba00a9520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13b0291ed633dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.465+0000 7fab8d7fa640 1 -- 172.21.15.29:0/1700284596 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00d00d0 con 0x55b7bcb7e350 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.465+0000 7fab8d7fa640 1 -- 172.21.15.29:0/1700284596 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00d0270 con 0x55b7bcb7e350 2024-09-21T01:51:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.465+0000 7fab8d7fa640 1 -- 172.21.15.29:0/1700284596 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00abda0 con 0x55b7bcb7e350 2024-09-21T01:51:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 msgr2=0x55b7bcf23070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf23070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9370 tx=0x7faba00a9520 comp rx=0 tx=0).stop 2024-09-21T01:51:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 shutdown_connections 2024-09-21T01:51:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb7e350 0x55b7bcf23070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb5d670 0x55b7bcf22b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/1700284596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb55490 0x55b7bcf455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 >> 172.21.15.29:0/1700284596 conn(0x55b7bcb72740 msgr2=0x55b7bcf39f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 shutdown_connections 2024-09-21T01:51:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.466+0000 7fabb3e9f740 1 -- 172.21.15.29:0/1700284596 wait complete. 2024-09-21T01:51:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.467+0000 7fabb3e9f740 1 -- start start 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.467+0000 7fabb3e9f740 1 Processor -- start 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 0x55b7bcf41b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb7e350 0x55b7bcf3c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faba81ae6e0 con 0x55b7bcb7e350 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba81adfe0 con 0x55b7bcb5d670 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faba81ab220 con 0x55b7bcb55490 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7fabb3e9f740 10 client.0 _ll_register_callbacks cb 0x7faba00a93f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7faba67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7faba77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb7e350 0x55b7bcf3c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7faba67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55608/0 (socket says 172.21.15.29:55608) 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7faba67fc640 1 -- 172.21.15.29:0/918647734 learned_addr learned my addr 172.21.15.29:0/918647734 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:51:58.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.468+0000 7faba6ffd640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 0x55b7bcf41b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba67fc640 1 -- 172.21.15.29:0/918647734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 msgr2=0x55b7bcf41b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba67fc640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 0x55b7bcf41b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba67fc640 1 -- 172.21.15.29:0/918647734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb7e350 msgr2=0x55b7bcf3c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba67fc640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb7e350 0x55b7bcf3c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba67fc640 1 -- 172.21.15.29:0/918647734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bcf3cf70 con 0x55b7bcb5d670 2024-09-21T01:51:58.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.469+0000 7faba6ffd640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 0x55b7bcf41b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:51:58.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.470+0000 7faba67fc640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0095f0 tx=0x7fab9c0055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2dd8cc5134c1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:51:58.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.470+0000 7fab86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:51:58.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c009b30 con 0x55b7bcb5d670 2024-09-21T01:51:58.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fab9c019070 con 0x55b7bcb5d670 2024-09-21T01:51:58.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba00d0270 con 0x55b7bcb5d670 2024-09-21T01:51:58.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fab9c00f790 con 0x55b7bcb5d670 2024-09-21T01:51:58.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fabb3e9f740 10 client.25892 fetch_fsmap learned FSMap version 33 2024-09-21T01:51:58.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.471+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7bcfb7190 con 0x55b7bcb5d670 2024-09-21T01:51:58.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c030030 con 0x55b7bcb5d670 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fab9c003a10 con 0x55b7bcb5d670 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 10 client.25892 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 20 client.25892 populate_metadata read hostname 'smithi029' 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb12080 0x55b7bcfb9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 4 client.25892 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7faba00d0270 con 0x55b7bcb12080 2024-09-21T01:51:58.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.474+0000 7faba6ffd640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb12080 0x55b7bcfb9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:51:58.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.475+0000 7fab86ffd640 10 client.25892 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:58.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.475+0000 7faba6ffd640 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb12080 0x55b7bcfb9600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9110151d2b1cad server_cookie=1698957c9c9a4451 in_seq=0 out_seq=0 2024-09-21T01:51:58.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.480+0000 7fab86ffd640 1 -- 172.21.15.29:0/918647734 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7faba00d0270 con 0x55b7bcb12080 2024-09-21T01:51:58.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.480+0000 7fab86ffd640 10 client.25892 handle_command_reply: tid=0 2024-09-21T01:51:58.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.480+0000 7fabb3e9f740 1 client.25892 shutdown 2024-09-21T01:51:58.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.480+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 msgr2=0x55b7bcf42080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0095f0 tx=0x7fab9c0055b0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 shutdown_connections 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b7bcb12080 0x55b7bcfb9600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7bcb7e350 0x55b7bcf3c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7bcb5d670 0x55b7bcf42080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 --2- 172.21.15.29:0/918647734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7bcb55490 0x55b7bcf41b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 >> 172.21.15.29:0/918647734 conn(0x55b7bcb72740 msgr2=0x55b7bcf39f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fab86ffd640 0 client.25892 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 shutdown_connections 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 1 -- 172.21.15.29:0/918647734 wait complete. 2024-09-21T01:51:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.481+0000 7fabb3e9f740 20 client.25892 trim_cache size 0 max 16384 2024-09-21T01:51:58.484 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:51:58.484 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:51:58.484 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "90602b42-4b81-4751-a9a5-c37e2d79d262", 2024-09-21T01:51:58.484 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:51:58.484 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 msgr2=0x7fab900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 0x7fab900b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fab980096c0 tx=0x7fab980058d0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 msgr2=0x7faba819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 0x7faba819ebb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba000b690 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 shutdown_connections 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab900b6590 0x7fab900b8aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faba8105ac0 0x7faba819ebb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faba8105170 0x7faba8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 --2- 172.21.15.29:0/501801557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103160 0x7faba8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:51:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.483+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 >> 172.21.15.29:0/501801557 conn(0x7faba8077c60 msgr2=0x7faba8106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:51:58.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.484+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 shutdown_connections 2024-09-21T01:51:58.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:51:58.484+0000 7fabad92b640 1 -- 172.21.15.29:0/501801557 wait complete. 2024-09-21T01:51:59.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:51:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:51:59.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:51:59 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.017 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.018 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:51:59 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:51:59 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.709+0000 7f36aa0ad640 1 -- 172.21.15.29:0/3108021302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 msgr2=0x7f36a410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.709+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/3108021302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a410d6b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009900 tx=0x7f36a002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.710+0000 7f36aa0ad640 1 -- 172.21.15.29:0/3108021302 shutdown_connections 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.710+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/3108021302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a411cba0 0x7f36a411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.710+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/3108021302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36a410dbf0 0x7f36a410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.710+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/3108021302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a410d6b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.710+0000 7f36aa0ad640 1 -- 172.21.15.29:0/3108021302 >> 172.21.15.29:0/3108021302 conn(0x7f36a406e9f0 msgr2=0x7f36a406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.713+0000 7f36aa0ad640 1 -- 172.21.15.29:0/3108021302 shutdown_connections 2024-09-21T01:52:00.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.713+0000 7f36aa0ad640 1 -- 172.21.15.29:0/3108021302 wait complete. 2024-09-21T01:52:00.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 Processor -- start 2024-09-21T01:52:00.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 -- start start 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a4113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36a411cba0 0x7f36a4113da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41226d0 con 0x7f36a411cba0 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4122550 con 0x7f36a4111cd0 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.714+0000 7f36aa0ad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4122850 con 0x7f36a410dbf0 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:54112/0 (socket says 172.21.15.29:54112) 2024-09-21T01:52:00.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 -- 172.21.15.29:0/2045814987 learned_addr learned my addr 172.21.15.29:0/2045814987 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a88aa640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a4113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 -- 172.21.15.29:0/2045814987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 msgr2=0x7f36a4113770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a4113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 -- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36a411cba0 msgr2=0x7f36a4113da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36a411cba0 0x7f36a4113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.716+0000 7f36a90ab640 1 -- 172.21.15.29:0/2045814987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4119110 con 0x7f36a410dbf0 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f36a90ab640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36a00057c0 tx=0x7f36a00045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f369a7fc640 1 -- 172.21.15.29:0/2045814987 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0040070 con 0x7f36a410dbf0 2024-09-21T01:52:00.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f369a7fc640 1 -- 172.21.15.29:0/2045814987 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a00316d0 con 0x7f36a410dbf0 2024-09-21T01:52:00.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f369a7fc640 1 -- 172.21.15.29:0/2045814987 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0040070 con 0x7f36a410dbf0 2024-09-21T01:52:00.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a4119340 con 0x7f36a410dbf0 2024-09-21T01:52:00.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.717+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a4119820 con 0x7f36a410dbf0 2024-09-21T01:52:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 Processor -- start 2024-09-21T01:52:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 -- start start 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 0x5638b994a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b957a1b0 0x5638b994ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 0x5638b994b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5638b98c0e50 con 0x5638b9582390 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5638b98b0280 con 0x5638b95a3070 2024-09-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.722+0000 7f36b1883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5638b98b4090 con 0x5638b957a1b0 2024-09-21T01:52:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a88aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 0x5638b994b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a88aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 0x5638b994b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51478/0 (socket says 172.21.15.29:51478) 2024-09-21T01:52:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 0x5638b994a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a88aa640 1 -- 172.21.15.29:0/2213274617 learned_addr learned my addr 172.21.15.29:0/2213274617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 -- 172.21.15.29:0/2213274617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b957a1b0 msgr2=0x5638b994ad50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b957a1b0 0x5638b994ad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 -- 172.21.15.29:0/2213274617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 msgr2=0x5638b994b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 0x5638b994b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.723+0000 7f36a98ac640 1 -- 172.21.15.29:0/2213274617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b960e650 con 0x5638b95a3070 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.724+0000 7f36a98ac640 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 0x5638b994a810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f369c009ce0 tx=0x7f369c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92fca1e6c3e26417 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.724+0000 7f368f7fe640 1 -- 172.21.15.29:0/2213274617 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c004b20 con 0x5638b95a3070 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.724+0000 7f368f7fe640 1 -- 172.21.15.29:0/2213274617 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f369c003e20 con 0x5638b95a3070 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.724+0000 7f368f7fe640 1 -- 172.21.15.29:0/2213274617 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0047b0 con 0x5638b95a3070 2024-09-21T01:52:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.724+0000 7f369a7fc640 1 -- 172.21.15.29:0/2045814987 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f36a0043070 con 0x7f36a410dbf0 2024-09-21T01:52:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.725+0000 7f36b1883740 1 -- 172.21.15.29:0/2213274617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 msgr2=0x5638b994a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.725+0000 7f36b1883740 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 0x5638b994a810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f369c009ce0 tx=0x7f369c00e720 comp rx=0 tx=0).stop 2024-09-21T01:52:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.725+0000 7f369a7fc640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 0x7f36900b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.726+0000 7f369a7fc640 1 -- 172.21.15.29:0/2045814987 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f36a01077e0 con 0x7f36a410dbf0 2024-09-21T01:52:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.726+0000 7f36b1883740 1 -- 172.21.15.29:0/2213274617 shutdown_connections 2024-09-21T01:52:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.726+0000 7f36b1883740 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b9582390 0x5638b994b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.726+0000 7f36b1883740 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b957a1b0 0x5638b994ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36a98ac640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 0x7f36900b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36b1883740 1 --2- 172.21.15.29:0/2213274617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b95a3070 0x5638b994a810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36b1883740 1 -- 172.21.15.29:0/2213274617 >> 172.21.15.29:0/2213274617 conn(0x5638b9597460 msgr2=0x5638b9948700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36a98ac640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 0x7f36900b8c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f369c005fd0 tx=0x7f369c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36b1883740 1 -- 172.21.15.29:0/2213274617 shutdown_connections 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.727+0000 7f36b1883740 1 -- 172.21.15.29:0/2213274617 wait complete. 2024-09-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.728+0000 7f36b1883740 1 -- start start 2024-09-21T01:52:00.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.728+0000 7f36b1883740 1 Processor -- start 2024-09-21T01:52:00.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.729+0000 7f36b1883740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.729+0000 7f36b1883740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b9582390 0x5638b9956340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.729+0000 7f36b1883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b95a3070 0x5638b9956880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36b1883740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5638b9711700 con 0x5638b957a1b0 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51490/0 (socket says 172.21.15.29:51490) 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36b1883740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5638b984b110 con 0x5638b9582390 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5638b9959090 con 0x5638b95a3070 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 -- 172.21.15.29:0/3903175666 learned_addr learned my addr 172.21.15.29:0/3903175666 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b9582390 0x5638b9956340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36b1883740 10 client.0 _ll_register_callbacks cb 0x5638b9864990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a98ac640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b95a3070 0x5638b9956880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 -- 172.21.15.29:0/3903175666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b95a3070 msgr2=0x5638b9956880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b95a3070 0x5638b9956880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 -- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b9582390 msgr2=0x5638b9956340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b9582390 0x5638b9956340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.730+0000 7f36a90ab640 1 -- 172.21.15.29:0/3903175666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b9878c40 con 0x5638b957a1b0 2024-09-21T01:52:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.731+0000 7f36a90ab640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a00cd2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bd157e78aa0004 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.731+0000 7f368cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:52:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.731+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0035040 con 0x5638b957a1b0 2024-09-21T01:52:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.731+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f36a00c9bf0 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.731+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c003e20 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.732+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36a00ce080 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.732+0000 7f36b1883740 10 client.16185 fetch_fsmap learned FSMap version 33 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.732+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5638b9964ce0 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.732+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a00d2460 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f36a003a040 con 0x5638b957a1b0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 10 client.16185 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 20 client.16185 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b95619f0 0x5638b99683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 4 client.16185 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f369c003e20 con 0x5638b95619f0 2024-09-21T01:52:00.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.733+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b95619f0 0x5638b99683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.734+0000 7f368cff9640 10 client.16185 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:00.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.734+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b95619f0 0x5638b99683f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db783021f162c170 server_cookie=b9348fb66ebeb96b in_seq=0 out_seq=0 2024-09-21T01:52:00.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.734+0000 7f368cff9640 1 -- 172.21.15.29:0/3903175666 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f369c003e20 con 0x5638b95619f0 2024-09-21T01:52:00.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.734+0000 7f368cff9640 10 client.16185 handle_command_reply: tid=0 2024-09-21T01:52:00.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.734+0000 7f36b1883740 1 client.16185 shutdown 2024-09-21T01:52:00.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.735+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 msgr2=0x5638b9955e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.735+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a00cd2d0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 shutdown_connections 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b95619f0 0x5638b99683f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b95a3070 0x5638b9956880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b9582390 0x5638b9956340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 --2- 172.21.15.29:0/3903175666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b957a1b0 0x5638b9955e00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 >> 172.21.15.29:0/3903175666 conn(0x5638b9597460 msgr2=0x5638b994d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f368cff9640 0 client.16185 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 shutdown_connections 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.736+0000 7f36b1883740 1 -- 172.21.15.29:0/3903175666 wait complete. 2024-09-21T01:52:00.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.737+0000 7f36b1883740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T01:52:00.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.763+0000 7f36b1883740 1 Processor -- start 2024-09-21T01:52:00.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.763+0000 7f36b1883740 1 -- start start 2024-09-21T01:52:00.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.763+0000 7f36b1883740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b9956730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.763+0000 7f36b1883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b994bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.763+0000 7f36b1883740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b9947cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36b1883740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b5940 con 0x5638b95a3070 2024-09-21T01:52:00.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36b1883740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b7570 con 0x5638b957a1b0 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b9956730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b9956730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55638/0 (socket says 172.21.15.29:55638) 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a90ab640 1 -- 172.21.15.29:0/1292910235 learned_addr learned my addr 172.21.15.29:0/1292910235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b994bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b6010 con 0x5638b9582390 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a98ac640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b9947cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 -- 172.21.15.29:0/1292910235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 msgr2=0x5638b9956730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b9956730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 -- 172.21.15.29:0/1292910235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 msgr2=0x5638b9947cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b9947cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 -- 172.21.15.29:0/1292910235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b9561c20 con 0x5638b9582390 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a98ac640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b9947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.764+0000 7f36a88aa640 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b994bf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5638b9968b20 tx=0x7f3694014900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3dedfbdeaed5a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.765+0000 7f368effd640 1 -- 172.21.15.29:0/1292910235 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694014190 con 0x5638b9582390 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.765+0000 7f368effd640 1 -- 172.21.15.29:0/1292910235 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36940054d0 con 0x5638b9582390 2024-09-21T01:52:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.765+0000 7f368effd640 1 -- 172.21.15.29:0/1292910235 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694012450 con 0x5638b9582390 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.766+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 msgr2=0x5638b994bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.766+0000 7f36b1883740 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b994bf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5638b9968b20 tx=0x7f3694014900 comp rx=0 tx=0).stop 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 shutdown_connections 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b9947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b994bf50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 --2- 172.21.15.29:0/1292910235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b9956730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 >> 172.21.15.29:0/1292910235 conn(0x5638b9597460 msgr2=0x5638b994b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 shutdown_connections 2024-09-21T01:52:00.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.767+0000 7f36b1883740 1 -- 172.21.15.29:0/1292910235 wait complete. 2024-09-21T01:52:00.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 -- start start 2024-09-21T01:52:00.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 Processor -- start 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b995a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b995a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b995ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b6de0 con 0x5638b95a3070 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b66e0 con 0x5638b957a1b0 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40751a0 con 0x5638b9582390 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.768+0000 7f36b1883740 10 client.0 _ll_register_callbacks cb 0x7f36a00cd750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:00.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a98ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b995ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b995a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a90ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b995a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55646/0 (socket says 172.21.15.29:55646) 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a90ab640 1 -- 172.21.15.29:0/3048600347 learned_addr learned my addr 172.21.15.29:0/3048600347 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b995a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 -- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 msgr2=0x5638b995a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b995a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 -- 172.21.15.29:0/3048600347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 msgr2=0x5638b995ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b995ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.769+0000 7f36a88aa640 1 -- 172.21.15.29:0/3048600347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b99d1d80 con 0x5638b9582390 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.770+0000 7f36a88aa640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b995a810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x5638b9968b20 tx=0x7f3694009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45564758b17436cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:00.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.770+0000 7f368cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:00.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.770+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694005020 con 0x5638b9582390 2024-09-21T01:52:00.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.770+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3694020030 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.770+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36940054d0 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36940051c0 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 10 client.25954 fetch_fsmap learned FSMap version 33 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5638b99d2fa0 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36940054d0 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f369400f040 con 0x5638b9582390 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 10 client.25954 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 20 client.25954 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b9536da0 0x5638b99d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 4 client.25954 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.771+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638b98c2da0 con 0x5638b9536da0 2024-09-21T01:52:00.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.772+0000 7f36a90ab640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b9536da0 0x5638b99d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.772+0000 7f36a90ab640 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b9536da0 0x5638b99d56a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8909c5149caecd89 server_cookie=c861d21afacdead6 in_seq=0 out_seq=0 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f368cff9640 10 client.25954 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f368cff9640 1 -- 172.21.15.29:0/3048600347 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x5638b98c2da0 con 0x5638b9536da0 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f368cff9640 10 client.25954 handle_command_reply: tid=0 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f36b1883740 1 client.25954 shutdown 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 msgr2=0x5638b995a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.773+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b995a810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x5638b9968b20 tx=0x7f3694009f30 comp rx=0 tx=0).stop 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 shutdown_connections 2024-09-21T01:52:00.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5638b9536da0 0x5638b99d56a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638b95a3070 0x5638b995ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5638b9582390 0x5638b995a810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 --2- 172.21.15.29:0/3048600347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638b957a1b0 0x5638b995a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 >> 172.21.15.29:0/3048600347 conn(0x5638b9597460 msgr2=0x5638b994e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f368cff9640 0 client.25954 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.774+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 shutdown_connections 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:52:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "90602b42-4b81-4751-a9a5-c37e2d79d262", 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36b1883740 1 -- 172.21.15.29:0/3048600347 wait complete. 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36b1883740 20 client.25954 trim_cache size 0 max 16384 2024-09-21T01:52:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 msgr2=0x7f36900b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 0x7f36900b8c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f369c005fd0 tx=0x7f369c004a40 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 msgr2=0x7f36a411a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.775+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36a00057c0 tx=0x7f36a00045e0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 shutdown_connections 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f36900b6710 0x7f36900b8c20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36a411cba0 0x7f36a4113da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a4111cd0 0x7f36a4113770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 --2- 172.21.15.29:0/2045814987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36a410dbf0 0x7f36a411a4a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 >> 172.21.15.29:0/2045814987 conn(0x7f36a406e9f0 msgr2=0x7f36a411b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 shutdown_connections 2024-09-21T01:52:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:00.776+0000 7f36aa0ad640 1 -- 172.21.15.29:0/2045814987 wait complete. 2024-09-21T01:52:01.772 INFO:tasks.cephfs.filesystem:scrub status for tag:90602b42-4b81-4751-a9a5-c37e2d79d262 - {'path': '/', 'tag': '90602b42-4b81-4751-a9a5-c37e2d79d262', 'options': 'recursive,force'} 2024-09-21T01:52:01.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:01 smithi190 ceph-mon[34368]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.8 MiB/s wr, 922 op/s 2024-09-21T01:52:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:01 smithi066 ceph-mon[35001]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.8 MiB/s wr, 922 op/s 2024-09-21T01:52:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:01 smithi029 ceph-mon[31228]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.8 MiB/s wr, 922 op/s 2024-09-21T01:52:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 205.0 s, 3674.6 tps, lat 27.253 ms stddev 30.042 2024-09-21T01:52:02.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:02 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:03.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:02 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:03.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:02 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:03 smithi190 ceph-mon[34368]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.3 MiB/s wr, 848 op/s 2024-09-21T01:52:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:03 smithi066 ceph-mon[35001]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.3 MiB/s wr, 848 op/s 2024-09-21T01:52:04.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:03 smithi029 ceph-mon[31228]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.3 MiB/s wr, 848 op/s 2024-09-21T01:52:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:05 smithi190 ceph-mon[34368]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 851 op/s 2024-09-21T01:52:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:52:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:52:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:52:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:05 smithi066 ceph-mon[35001]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 851 op/s 2024-09-21T01:52:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:52:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:52:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:52:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:05 smithi029 ceph-mon[31228]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 851 op/s 2024-09-21T01:52:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:52:06.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:52:06.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:52:06.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:06 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:52:06.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:06 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:52:07.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:06 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:52:07.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:06 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:52:07.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:06 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:52:07.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:06 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:52:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 210.0 s, 4215.8 tps, lat 23.561 ms stddev 25.751 2024-09-21T01:52:07.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:07 smithi029 ceph-mon[31228]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 958 op/s 2024-09-21T01:52:07.974 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:07 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:07 smithi190 ceph-mon[34368]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 958 op/s 2024-09-21T01:52:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:07 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:08.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:07 smithi066 ceph-mon[35001]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 958 op/s 2024-09-21T01:52:08.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:07 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 -- 172.21.15.29:0/2372509602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103640 msgr2=0x7f618c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 --2- 172.21.15.29:0/2372509602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103640 0x7f618c103a50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f617c009900 tx=0x7f617c02a410 comp rx=0 tx=0).stop 2024-09-21T01:52:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 -- 172.21.15.29:0/2372509602 shutdown_connections 2024-09-21T01:52:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 --2- 172.21.15.29:0/2372509602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c103fe0 0x7f618c10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 --2- 172.21.15.29:0/2372509602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103640 0x7f618c103a50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 --2- 172.21.15.29:0/2372509602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c1025c0 0x7f618c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.517+0000 7f6193f20640 1 -- 172.21.15.29:0/2372509602 >> 172.21.15.29:0/2372509602 conn(0x7f618c0fdf00 msgr2=0x7f618c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 -- 172.21.15.29:0/2372509602 shutdown_connections 2024-09-21T01:52:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 -- 172.21.15.29:0/2372509602 wait complete. 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 Processor -- start 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 -- start start 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 0x7f618c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.518+0000 7f6193f20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103fe0 0x7f618c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6193f20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10e1f0 con 0x7f618c103fe0 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6193f20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10e070 con 0x7f618c103640 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6193f20640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10e370 con 0x7f618c1025c0 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103fe0 0x7f618c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 0x7f618c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34426/0 (socket says 172.21.15.29:34426) 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6191cbc640 1 -- 172.21.15.29:0/3642965740 learned_addr learned my addr 172.21.15.29:0/3642965740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 -- 172.21.15.29:0/3642965740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 msgr2=0x7f618c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 -- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103fe0 msgr2=0x7f618c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103fe0 0x7f618c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 -- 172.21.15.29:0/3642965740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c19ba10 con 0x7f618c103640 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6191cbc640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f61914bb640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 0x7f618c19a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.519+0000 7f6182ffd640 1 -- 172.21.15.29:0/3642965740 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c03f070 con 0x7f618c103640 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.520+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c19bca0 con 0x7f618c103640 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.520+0000 7f6182ffd640 1 -- 172.21.15.29:0/3642965740 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f617c003ca0 con 0x7f618c103640 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.520+0000 7f6182ffd640 1 -- 172.21.15.29:0/3642965740 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c035070 con 0x7f618c103640 2024-09-21T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.520+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a23f0 con 0x7f618c103640 2024-09-21T01:52:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 Processor -- start 2024-09-21T01:52:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 -- start start 2024-09-21T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f73090a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f73087b6f0 con 0x55f7305611b0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f73086ab20 con 0x55f7305382f0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.523+0000 7f619ae90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f73086e930 con 0x55f7305404d0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6182ffd640 1 -- 172.21.15.29:0/3642965740 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f617c03b4f0 con 0x7f618c103640 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33228/0 (socket says 172.21.15.29:33228) 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41632/0 (socket says 172.21.15.29:41632) 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61924bd640 1 -- 172.21.15.29:0/1787224666 learned_addr learned my addr 172.21.15.29:0/1787224666 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61914bb640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f73090a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 -- 172.21.15.29:0/1787224666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 msgr2=0x55f73090a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f73090a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 -- 172.21.15.29:0/1787224666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 msgr2=0x55f730908e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 -- 172.21.15.29:0/1787224666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f730830530 con 0x55f7305382f0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61914bb640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f73090a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f61924bd640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6188011870 tx=0x7f6188011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d1bfed51da73a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6177fff640 1 -- 172.21.15.29:0/1787224666 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880112f0 con 0x55f7305382f0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.524+0000 7f6177fff640 1 -- 172.21.15.29:0/1787224666 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188003e20 con 0x55f7305382f0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f6177fff640 1 -- 172.21.15.29:0/1787224666 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880043d0 con 0x55f7305382f0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f6182ffd640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 0x7f61780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f61924bd640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 0x7f61780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f6182ffd640 1 -- 172.21.15.29:0/3642965740 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f617c107700 con 0x7f618c103640 2024-09-21T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f619ae90740 1 -- 172.21.15.29:0/1787224666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 msgr2=0x55f7309094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f61924bd640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 0x7f61780b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f618400b9f0 tx=0x7f6184005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.525+0000 7f619ae90740 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6188011870 tx=0x7f6188011d30 comp rx=0 tx=0).stop 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 -- 172.21.15.29:0/1787224666 shutdown_connections 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f73090a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f7309094c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 --2- 172.21.15.29:0/1787224666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730908e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 -- 172.21.15.29:0/1787224666 >> 172.21.15.29:0/1787224666 conn(0x55f730555300 msgr2=0x55f7309048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 -- 172.21.15.29:0/1787224666 shutdown_connections 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 -- 172.21.15.29:0/1787224666 wait complete. 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 -- start start 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.526+0000 7f619ae90740 1 Processor -- start 2024-09-21T01:52:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730912bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730913720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f7307a2ab0 con 0x55f7305611b0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f730810730 con 0x55f7305404d0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f73090ab30 con 0x55f7305382f0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34450/0 (socket says 172.21.15.29:34450) 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f6191cbc640 1 -- 172.21.15.29:0/2312045330 learned_addr learned my addr 172.21.15.29:0/2312045330 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730912bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f619ae90740 10 client.0 _ll_register_callbacks cb 0x55f7308646b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61924bd640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730913720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 -- 172.21.15.29:0/2312045330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 msgr2=0x55f7309124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 -- 172.21.15.29:0/2312045330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 msgr2=0x55f730913720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730913720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 -- 172.21.15.29:0/2312045330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7305cc790 con 0x55f7305404d0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f6191cbc640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61914bb640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730912bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f617c105260 tx=0x7f617c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9131c4046fefdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.527+0000 7f61757fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c048040 con 0x55f7305404d0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f617c0cebd0 con 0x55f7305404d0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6188003e20 con 0x55f7305404d0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f617c0cd720 con 0x55f7305404d0 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 10 client.25919 fetch_fsmap learned FSMap version 35 2024-09-21T01:52:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f730645620 con 0x55f7305404d0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188003e20 con 0x55f7305404d0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 35) ==== 8398+0+0 (secure 0 0 0) 0x7f617c04b060 con 0x55f7305404d0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 10 client.25919 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 20 client.25919 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f73051fd80 0x55f73091fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 4 client.25919 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.528+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73087d640 con 0x55f73051fd80 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.529+0000 7f6191cbc640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f73051fd80 0x55f73091fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.529+0000 7f6191cbc640 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f73051fd80 0x55f73091fde0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cb448eaedc7a7 server_cookie=60618527c3555f06 in_seq=0 out_seq=0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.529+0000 7f61757fa640 10 client.25919 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f61757fa640 1 -- 172.21.15.29:0/2312045330 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f73087d640 con 0x55f73051fd80 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f61757fa640 10 client.25919 handle_command_reply: tid=0 2024-09-21T01:52:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 client.25919 shutdown 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 msgr2=0x55f730912bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730912bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f617c105260 tx=0x7f617c000bc0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 shutdown_connections 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f73051fd80 0x55f73091fde0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730913720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f61757fa640 0 client.25919 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730912bb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 --2- 172.21.15.29:0/2312045330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f7309124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.530+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 >> 172.21.15.29:0/2312045330 conn(0x55f730555300 msgr2=0x55f730902ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.531+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 shutdown_connections 2024-09-21T01:52:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.531+0000 7f619ae90740 1 -- 172.21.15.29:0/2312045330 wait complete. 2024-09-21T01:52:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.531+0000 7f619ae90740 20 client.25919 trim_cache size 0 max 16384 2024-09-21T01:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 Processor -- start 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 -- start start 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f730904220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730908df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1a8d30 con 0x55f7305611b0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1aaa20 con 0x55f7305404d0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f619ae90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1a9400 con 0x55f7305382f0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f61914bb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730908df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33254/0 (socket says 172.21.15.29:33254) 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f61924bd640 1 -- 172.21.15.29:0/2609052336 learned_addr learned my addr 172.21.15.29:0/2609052336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.552+0000 7f6191cbc640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f730904220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 -- 172.21.15.29:0/2609052336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 msgr2=0x55f730904220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f730904220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 -- 172.21.15.29:0/2609052336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 msgr2=0x55f730909330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 -- 172.21.15.29:0/2609052336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f730910680 con 0x55f7305404d0 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f6191cbc640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f730904220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61924bd640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61914bb640 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730908df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f617c005ab0 tx=0x7f617c009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52f3777e90bec43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61777fe640 1 -- 172.21.15.29:0/2609052336 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c048030 con 0x55f7305404d0 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61777fe640 1 -- 172.21.15.29:0/2609052336 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f617c0b7020 con 0x55f7305404d0 2024-09-21T01:52:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.553+0000 7f61777fe640 1 -- 172.21.15.29:0/2609052336 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0c90c0 con 0x55f7305404d0 2024-09-21T01:52:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- 172.21.15.29:0/2609052336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 msgr2=0x55f730908df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730908df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f617c005ab0 tx=0x7f617c009230 comp rx=0 tx=0).stop 2024-09-21T01:52:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- 172.21.15.29:0/2609052336 shutdown_connections 2024-09-21T01:52:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f730909330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305404d0 0x55f730908df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 --2- 172.21.15.29:0/2609052336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305382f0 0x55f730904220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- 172.21.15.29:0/2609052336 >> 172.21.15.29:0/2609052336 conn(0x55f730555300 msgr2=0x55f7308ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- 172.21.15.29:0/2609052336 shutdown_connections 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- 172.21.15.29:0/2609052336 wait complete. 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.554+0000 7f619ae90740 1 -- start start 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 Processor -- start 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f730919020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f7309195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1aa290 con 0x55f7305611b0 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1a9b90 con 0x55f7305382f0 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1a6d10 con 0x55f7305404d0 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f619ae90740 10 client.0 _ll_register_callbacks cb 0x7f617c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f7309195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f61924bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f7309195c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33266/0 (socket says 172.21.15.29:33266) 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f61924bd640 1 -- 172.21.15.29:0/1455450245 learned_addr learned my addr 172.21.15.29:0/1455450245 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.555+0000 7f6191cbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41660/0 (socket says 172.21.15.29:41660) 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f61914bb640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f730919020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 -- 172.21.15.29:0/1455450245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 msgr2=0x55f730919020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f730919020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 -- 172.21.15.29:0/1455450245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 msgr2=0x55f7309195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f7309195c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 -- 172.21.15.29:0/1455450245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f730919d50 con 0x55f7305382f0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f61914bb640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f730919020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f6191cbc640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f7309204e0 tx=0x7f6188005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e23f19c3b075db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f61757fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618800fdc0 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617c0b7020 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.556+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6188011050 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6188011270 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 10 client.25925 fetch_fsmap learned FSMap version 35 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f730993020 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f617c0b7020 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 35) ==== 8398+0+0 (secure 0 0 0) 0x7f618802e430 con 0x55f7305382f0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 10 client.25925 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 20 client.25925 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f7304f51c0 0x55f730995aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 4 client.25925 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7308790f0 con 0x55f7304f51c0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.557+0000 7f61914bb640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f7304f51c0 0x55f730995aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.558+0000 7f61757fa640 10 client.25925 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.558+0000 7f61914bb640 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f7304f51c0 0x55f730995aa0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4065b588ad0c76 server_cookie=71fb8a09aa5a8068 in_seq=0 out_seq=0 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f61757fa640 1 -- 172.21.15.29:0/1455450245 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7308790f0 con 0x55f7304f51c0 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f61757fa640 10 client.25925 handle_command_reply: tid=0 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 client.25925 shutdown 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 msgr2=0x55f73091a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f7309204e0 tx=0x7f6188005a50 comp rx=0 tx=0).stop 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 shutdown_connections 2024-09-21T01:52:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f7304f51c0 0x55f730995aa0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f7305611b0 0x55f7309195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f61757fa640 0 client.25925 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7305404d0 0x55f730919020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 --2- 172.21.15.29:0/1455450245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7305382f0 0x55f73091a360 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 >> 172.21.15.29:0/1455450245 conn(0x55f730555300 msgr2=0x55f730910890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 shutdown_connections 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 1 -- 172.21.15.29:0/1455450245 wait complete. 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.667+0000 7f619ae90740 20 client.25925 trim_cache size 0 max 16384 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:52:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 msgr2=0x7f61780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 0x7f61780b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f618400b9f0 tx=0x7f6184005cc0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 msgr2=0x7f618c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 0x7f618c19a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c004270 comp rx=0 tx=0).stop 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 shutdown_connections 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f61780b6510 0x7f61780b8a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f618c103fe0 0x7f618c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c103640 0x7f618c19a640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 --2- 172.21.15.29:0/3642965740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f618c1025c0 0x7f618c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.668+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 >> 172.21.15.29:0/3642965740 conn(0x7f618c0fdf00 msgr2=0x7f618c06b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.669+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 shutdown_connections 2024-09-21T01:52:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:08.669+0000 7f6193f20640 1 -- 172.21.15.29:0/3642965740 wait complete. 2024-09-21T01:52:09.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:09 smithi190 ceph-mon[34368]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.7 MiB/s wr, 739 op/s 2024-09-21T01:52:10.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:09 smithi066 ceph-mon[35001]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.7 MiB/s wr, 739 op/s 2024-09-21T01:52:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:09 smithi029 ceph-mon[31228]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.7 MiB/s wr, 739 op/s 2024-09-21T01:52:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:11 smithi190 ceph-mon[34368]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-21T01:52:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:11 smithi066 ceph-mon[35001]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-21T01:52:12.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:11 smithi029 ceph-mon[31228]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-21T01:52:12.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 215.0 s, 3853.4 tps, lat 26.075 ms stddev 28.145 2024-09-21T01:52:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:13 smithi190 ceph-mon[34368]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 794 op/s 2024-09-21T01:52:14.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:13 smithi066 ceph-mon[35001]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 794 op/s 2024-09-21T01:52:14.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:13 smithi029 ceph-mon[31228]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 794 op/s 2024-09-21T01:52:15.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:15 smithi190 ceph-mon[34368]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 739 op/s 2024-09-21T01:52:16.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:15 smithi066 ceph-mon[35001]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 739 op/s 2024-09-21T01:52:16.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:15 smithi029 ceph-mon[31228]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 739 op/s 2024-09-21T01:52:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 220.0 s, 4413.6 tps, lat 22.577 ms stddev 33.134 2024-09-21T01:52:17.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:17 smithi190 ceph-mon[34368]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T01:52:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:17 smithi066 ceph-mon[35001]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T01:52:18.080 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:17 smithi029 ceph-mon[31228]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T01:52:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:19 smithi066 ceph-mon[35001]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 764 op/s 2024-09-21T01:52:20.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:19 smithi029 ceph-mon[31228]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 764 op/s 2024-09-21T01:52:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:19 smithi190 ceph-mon[34368]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 764 op/s 2024-09-21T01:52:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.824+0000 7f8670a0e640 1 -- 172.21.15.29:0/655675196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c103fc0 msgr2=0x7f866c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.824+0000 7f8670a0e640 1 --2- 172.21.15.29:0/655675196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c103fc0 0x7f866c10a960 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8664009c90 tx=0x7f866402c780 comp rx=0 tx=0).stop 2024-09-21T01:52:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 -- 172.21.15.29:0/655675196 shutdown_connections 2024-09-21T01:52:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 --2- 172.21.15.29:0/655675196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c103fc0 0x7f866c10a960 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 --2- 172.21.15.29:0/655675196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 --2- 172.21.15.29:0/655675196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1025a0 0x7f866c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 -- 172.21.15.29:0/655675196 >> 172.21.15.29:0/655675196 conn(0x7f866c0fdee0 msgr2=0x7f866c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 -- 172.21.15.29:0/655675196 shutdown_connections 2024-09-21T01:52:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.825+0000 7f8670a0e640 1 -- 172.21.15.29:0/655675196 wait complete. 2024-09-21T01:52:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 Processor -- start 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 -- start start 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 0x7f866c19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c103fc0 0x7f866c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10d120 con 0x7f866c1025a0 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10cfa0 con 0x7f866c103620 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.826+0000 7f8670a0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10d2a0 con 0x7f866c103fc0 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f866a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 0x7f866c19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46812/0 (socket says 172.21.15.29:46812) 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 -- 172.21.15.29:0/2041923992 learned_addr learned my addr 172.21.15.29:0/2041923992 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f866ad9d640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c103fc0 0x7f866c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 -- 172.21.15.29:0/2041923992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c103fc0 msgr2=0x7f866c19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c103fc0 0x7f866c19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 -- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 msgr2=0x7f866c19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 0x7f866c19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 -- 172.21.15.29:0/2041923992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c19db80 con 0x7f866c103620 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f866a59c640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 0x7f866c19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.827+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f866000d950 tx=0x7f866000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.828+0000 7f865b7fe640 1 -- 172.21.15.29:0/2041923992 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660017070 con 0x7f866c103620 2024-09-21T01:52:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.828+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c19de70 con 0x7f866c103620 2024-09-21T01:52:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.828+0000 7f865b7fe640 1 -- 172.21.15.29:0/2041923992 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8660009c30 con 0x7f866c103620 2024-09-21T01:52:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.828+0000 7f865b7fe640 1 -- 172.21.15.29:0/2041923992 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866000f620 con 0x7f866c103620 2024-09-21T01:52:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.828+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c1a4560 con 0x7f866c103620 2024-09-21T01:52:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.832+0000 7f865b7fe640 1 -- 172.21.15.29:0/2041923992 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f866000f7c0 con 0x7f866c103620 2024-09-21T01:52:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.833+0000 7f865b7fe640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 0x7f86540b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.834+0000 7f865b7fe640 1 -- 172.21.15.29:0/2041923992 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f86600e6de0 con 0x7f866c103620 2024-09-21T01:52:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.834+0000 7f866a59c640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 0x7f86540b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.835+0000 7f866a59c640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 0x7f86540b8b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f865c00bfd0 tx=0x7f865c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.836+0000 7f8677a55740 1 Processor -- start 2024-09-21T01:52:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.836+0000 7f8677a55740 1 -- start start 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.836+0000 7f8677a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63eea1b0 0x55ce64291e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8677a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8677a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce64293010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8677a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ce642046f0 con 0x55ce63ec94d0 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8677a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ce641f3b20 con 0x55ce63eea1b0 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8677a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ce641f7930 con 0x55ce63ec12f0 2024-09-21T01:52:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f866ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce64293010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37344/0 (socket says 172.21.15.29:37344) 2024-09-21T01:52:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 -- 172.21.15.29:0/889661908 learned_addr learned my addr 172.21.15.29:0/889661908 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f866a59c640 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63eea1b0 0x55ce64291e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 -- 172.21.15.29:0/889661908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63eea1b0 msgr2=0x55ce64291e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63eea1b0 0x55ce64291e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 -- 172.21.15.29:0/889661908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 msgr2=0x55ce64293010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce64293010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.837+0000 7f8669d9b640 1 -- 172.21.15.29:0/889661908 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce641b9530 con 0x55ce63ec12f0 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.838+0000 7f8669d9b640 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86600accd0 tx=0x7f86600a8f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6667a847a514ec74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.838+0000 7f8658ff9640 1 -- 172.21.15.29:0/889661908 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660020070 con 0x55ce63ec12f0 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.838+0000 7f8658ff9640 1 -- 172.21.15.29:0/889661908 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86600b1a00 con 0x55ce63ec12f0 2024-09-21T01:52:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.838+0000 7f8658ff9640 1 -- 172.21.15.29:0/889661908 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660092e60 con 0x55ce63ec12f0 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 -- 172.21.15.29:0/889661908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 msgr2=0x55ce642924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86600accd0 tx=0x7f86600a8f80 comp rx=0 tx=0).stop 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 -- 172.21.15.29:0/889661908 shutdown_connections 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce64293010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce642924c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 --2- 172.21.15.29:0/889661908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63eea1b0 0x55ce64291e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 -- 172.21.15.29:0/889661908 >> 172.21.15.29:0/889661908 conn(0x55ce63ede300 msgr2=0x55ce6428d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 -- 172.21.15.29:0/889661908 shutdown_connections 2024-09-21T01:52:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.839+0000 7f8677a55740 1 -- 172.21.15.29:0/889661908 wait complete. 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 -- start start 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 Processor -- start 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6429c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ce6412bab0 con 0x55ce63eea1b0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ce64199730 con 0x55ce63ec94d0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ce64293b30 con 0x55ce63ec12f0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.840+0000 7f8677a55740 10 client.0 _ll_register_callbacks cb 0x55ce641ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f866a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f866ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6429c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f866a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37360/0 (socket says 172.21.15.29:37360) 2024-09-21T01:52:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f866ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6429c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46634/0 (socket says 172.21.15.29:46634) 2024-09-21T01:52:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f866a59c640 1 -- 172.21.15.29:0/2421883451 learned_addr learned my addr 172.21.15.29:0/2421883451 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 -- 172.21.15.29:0/2421883451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 msgr2=0x55ce6429b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 -- 172.21.15.29:0/2421883451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 msgr2=0x55ce6429c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6429c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.841+0000 7f8669d9b640 1 -- 172.21.15.29:0/2421883451 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce63f55790 con 0x55ce63ec94d0 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.842+0000 7f8669d9b640 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429bd60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86600accd0 tx=0x7f8660010690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7a75e8bbbfd262 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.842+0000 7f8651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.842+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86600d4070 con 0x55ce63ec94d0 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.843+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8660098170 con 0x55ce63ec94d0 2024-09-21T01:52:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.843+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86600b1a00 con 0x55ce63ec94d0 2024-09-21T01:52:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.843+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8660020050 con 0x55ce63ec94d0 2024-09-21T01:52:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.843+0000 7f8677a55740 10 client.25937 fetch_fsmap learned FSMap version 35 2024-09-21T01:52:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce63fce620 con 0x55ce63ec94d0 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86600983b0 con 0x55ce63ec94d0 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 35) ==== 8398+0+0 (secure 0 0 0) 0x7f86600abe40 con 0x55ce63ec94d0 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8677a55740 10 client.25937 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8677a55740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.844+0000 7f8677a55740 20 client.25937 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.845+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63ea8d80 0x55ce642b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.845+0000 7f8677a55740 4 client.25937 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.845+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86600b1a00 con 0x55ce63ea8d80 2024-09-21T01:52:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.845+0000 7f866a59c640 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63ea8d80 0x55ce642b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.846+0000 7f866a59c640 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63ea8d80 0x55ce642b1150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a1e997c7580ccb server_cookie=23ac4b3efa4fa51a in_seq=0 out_seq=0 2024-09-21T01:52:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.846+0000 7f8651ffb640 10 client.25937 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.846+0000 7f8651ffb640 1 -- 172.21.15.29:0/2421883451 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f86600b1a00 con 0x55ce63ea8d80 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.846+0000 7f8651ffb640 10 client.25937 handle_command_reply: tid=0 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.846+0000 7f8677a55740 1 client.25937 shutdown 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 msgr2=0x55ce6429bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429bd60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86600accd0 tx=0x7f8660010690 comp rx=0 tx=0).stop 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 shutdown_connections 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63ea8d80 0x55ce642b1150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6429c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429bd60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 --2- 172.21.15.29:0/2421883451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 >> 172.21.15.29:0/2421883451 conn(0x55ce63ede300 msgr2=0x55ce6428bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8651ffb640 0 client.25937 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 shutdown_connections 2024-09-21T01:52:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 1 -- 172.21.15.29:0/2421883451 wait complete. 2024-09-21T01:52:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.847+0000 7f8677a55740 20 client.25937 trim_cache size 0 max 16384 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 Processor -- start 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 -- start start 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6428de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00e540 con 0x55ce63eea1b0 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f864c010230 con 0x55ce63ec94d0 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8677a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00ec10 con 0x55ce63ec12f0 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f866ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6428de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f866ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6428de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46636/0 (socket says 172.21.15.29:46636) 2024-09-21T01:52:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f866ad9d640 1 -- 172.21.15.29:0/96331829 learned_addr learned my addr 172.21.15.29:0/96331829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f866a59c640 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.870+0000 7f8669d9b640 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.871+0000 7f866a59c640 1 -- 172.21.15.29:0/96331829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 msgr2=0x55ce6429ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.871+0000 7f866a59c640 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.871+0000 7f866a59c640 1 -- 172.21.15.29:0/96331829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 msgr2=0x55ce6428de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.871+0000 7f866a59c640 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6428de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.871+0000 7f866a59c640 1 -- 172.21.15.29:0/96331829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce64299830 con 0x55ce63ec12f0 2024-09-21T01:52:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.873+0000 7f866a59c640 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429a8c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b5f0 tx=0x7f865c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5890676f7ead562c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.873+0000 7f8653fff640 1 -- 172.21.15.29:0/96331829 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0200d0 con 0x55ce63ec12f0 2024-09-21T01:52:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.873+0000 7f8653fff640 1 -- 172.21.15.29:0/96331829 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c020270 con 0x55ce63ec12f0 2024-09-21T01:52:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.873+0000 7f8653fff640 1 -- 172.21.15.29:0/96331829 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c029c30 con 0x55ce63ec12f0 2024-09-21T01:52:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- 172.21.15.29:0/96331829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 msgr2=0x55ce6429a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429a8c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b5f0 tx=0x7f865c013000 comp rx=0 tx=0).stop 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- 172.21.15.29:0/96331829 shutdown_connections 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63eea1b0 0x55ce6428de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec94d0 0x55ce6429ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 --2- 172.21.15.29:0/96331829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63ec12f0 0x55ce6429a8c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- 172.21.15.29:0/96331829 >> 172.21.15.29:0/96331829 conn(0x55ce63ede300 msgr2=0x55ce64292c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- 172.21.15.29:0/96331829 shutdown_connections 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- 172.21.15.29:0/96331829 wait complete. 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 -- start start 2024-09-21T01:52:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.875+0000 7f8677a55740 1 Processor -- start 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 0x55ce642a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce642a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63eea1b0 0x55ce642a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00faa0 con 0x55ce63ec94d0 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00f3a0 con 0x55ce63ec12f0 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00c520 con 0x55ce63eea1b0 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8677a55740 10 client.0 _ll_register_callbacks cb 0x7f8664009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce642a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 0x55ce642a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8669d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce642a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46652/0 (socket says 172.21.15.29:46652) 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f8669d9b640 1 -- 172.21.15.29:0/334040902 learned_addr learned my addr 172.21.15.29:0/334040902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866ad9d640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63eea1b0 0x55ce642a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 -- 172.21.15.29:0/334040902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63eea1b0 msgr2=0x55ce642a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63eea1b0 0x55ce642a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 -- 172.21.15.29:0/334040902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 msgr2=0x55ce642a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce642a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.876+0000 7f866a59c640 1 -- 172.21.15.29:0/334040902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce642a2050 con 0x55ce63ec12f0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f866a59c640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 0x55ce642a06c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b5f0 tx=0x7f865c020140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18909ee01b1ae799 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c004020 con 0x55ce63ec12f0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f865c0041c0 con 0x55ce63ec12f0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c020270 con 0x55ce63ec12f0 2024-09-21T01:52:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f865c0043c0 con 0x55ce63ec12f0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8677a55740 10 client.25943 fetch_fsmap learned FSMap version 35 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.877+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce642a3370 con 0x55ce63ec12f0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c020270 con 0x55ce63ec12f0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 35) ==== 8398+0+0 (secure 0 0 0) 0x7f865c042070 con 0x55ce63ec12f0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 10 client.25943 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 20 client.25943 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63e7e1c0 0x55ce6431db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 4 client.25943 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce64206640 con 0x55ce63e7e1c0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8669d9b640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63e7e1c0 0x55ce6431db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8651ffb640 10 client.25943 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.878+0000 7f8669d9b640 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63e7e1c0 0x55ce6431db90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219337f40a30ae45 server_cookie=254f322ecf92f682 in_seq=0 out_seq=0 2024-09-21T01:52:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.894+0000 7f8651ffb640 1 -- 172.21.15.29:0/334040902 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce64206640 con 0x55ce63e7e1c0 2024-09-21T01:52:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.894+0000 7f8651ffb640 10 client.25943 handle_command_reply: tid=0 2024-09-21T01:52:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.894+0000 7f8677a55740 1 client.25943 shutdown 2024-09-21T01:52:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 msgr2=0x55ce642a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 0x55ce642a06c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b5f0 tx=0x7f865c020140 comp rx=0 tx=0).stop 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 shutdown_connections 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ce63e7e1c0 0x55ce6431db90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce63eea1b0 0x55ce642a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ce63ec94d0 0x55ce642a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 --2- 172.21.15.29:0/334040902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce63ec12f0 0x55ce642a06c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 >> 172.21.15.29:0/334040902 conn(0x55ce63ede300 msgr2=0x55ce64292c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8651ffb640 0 client.25943 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 shutdown_connections 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 1 -- 172.21.15.29:0/334040902 wait complete. 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.895+0000 7f8677a55740 20 client.25943 trim_cache size 0 max 16384 2024-09-21T01:52:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:52:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:52:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:52:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:52:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 msgr2=0x7f86540b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 0x7f86540b8b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f865c00bfd0 tx=0x7f865c00b390 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 msgr2=0x7f866c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f866000d950 tx=0x7f866000de10 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 shutdown_connections 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86540b6610 0x7f86540b8b20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c103fc0 0x7f866c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c103620 0x7f866c19c7b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 --2- 172.21.15.29:0/2041923992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866c1025a0 0x7f866c19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 >> 172.21.15.29:0/2041923992 conn(0x7f866c0fdee0 msgr2=0x7f866c1927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 shutdown_connections 2024-09-21T01:52:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:21.896+0000 7f8670a0e640 1 -- 172.21.15.29:0/2041923992 wait complete. 2024-09-21T01:52:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:21 smithi066 ceph-mon[35001]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T01:52:22.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:21 smithi029 ceph-mon[31228]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T01:52:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:21 smithi190 ceph-mon[34368]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T01:52:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 225.0 s, 3124.0 tps, lat 32.228 ms stddev 36.064 2024-09-21T01:52:24.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:23 smithi066 ceph-mon[35001]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-09-21T01:52:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:23 smithi029 ceph-mon[31228]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-09-21T01:52:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:23 smithi190 ceph-mon[34368]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-09-21T01:52:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:25 smithi066 ceph-mon[35001]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 778 op/s 2024-09-21T01:52:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:25 smithi029 ceph-mon[31228]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 778 op/s 2024-09-21T01:52:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:25 smithi190 ceph-mon[34368]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 778 op/s 2024-09-21T01:52:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 230.0 s, 3229.0 tps, lat 30.747 ms stddev 38.030 2024-09-21T01:52:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:27 smithi066 ceph-mon[35001]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 950 op/s 2024-09-21T01:52:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:27 smithi029 ceph-mon[31228]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 950 op/s 2024-09-21T01:52:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:27 smithi190 ceph-mon[34368]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 950 op/s 2024-09-21T01:52:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:29 smithi066 ceph-mon[35001]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-21T01:52:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:29 smithi029 ceph-mon[31228]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-21T01:52:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:29 smithi190 ceph-mon[34368]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-21T01:52:31.773 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:52:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:31 smithi066 ceph-mon[35001]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 829 op/s 2024-09-21T01:52:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:31 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:31 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:52:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:31 smithi029 ceph-mon[31228]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 829 op/s 2024-09-21T01:52:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:31 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:32.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:31 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:52:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:31 smithi190 ceph-mon[34368]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 829 op/s 2024-09-21T01:52:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:31 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:31 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:52:32.312 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.926+0000 7f677ca87640 1 -- 172.21.15.29:0/2148446321 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740318a0 con 0x7f67781049b0 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.928+0000 7f677fced640 1 -- 172.21.15.29:0/2148446321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 msgr2=0x7f6778104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.928+0000 7f677fced640 1 --2- 172.21.15.29:0/2148446321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f6778104dc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6774009900 tx=0x7f677402a410 comp rx=0 tx=0).stop 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 -- 172.21.15.29:0/2148446321 shutdown_connections 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 --2- 172.21.15.29:0/2148446321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f677810de10 0x7f6778110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 --2- 172.21.15.29:0/2148446321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6778105300 0x7f677810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 --2- 172.21.15.29:0/2148446321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f6778104dc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 -- 172.21.15.29:0/2148446321 >> 172.21.15.29:0/2148446321 conn(0x7f6778079730 msgr2=0x7f6778079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 -- 172.21.15.29:0/2148446321 shutdown_connections 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.929+0000 7f677fced640 1 -- 172.21.15.29:0/2148446321 wait complete. 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.930+0000 7f677fced640 1 Processor -- start 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 -- start start 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f677819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677810de10 0x7f67781a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6778103240 con 0x7f67781049b0 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67781030c0 con 0x7f6778105300 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677fced640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67781033c0 con 0x7f677810de10 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677d288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677d288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35760/0 (socket says 172.21.15.29:35760) 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677d288640 1 -- 172.21.15.29:0/2366471250 learned_addr learned my addr 172.21.15.29:0/2366471250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677da89640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f677819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.931+0000 7f677e28a640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677810de10 0x7f67781a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 -- 172.21.15.29:0/2366471250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677810de10 msgr2=0x7f67781a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677810de10 0x7f67781a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 -- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 msgr2=0x7f677819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f677819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 -- 172.21.15.29:0/2366471250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67781a2640 con 0x7f6778105300 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677d288640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f676800e9c0 tx=0x7f676800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f676effd640 1 -- 172.21.15.29:0/2366471250 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800ccc0 con 0x7f6778105300 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781a2930 con 0x7f6778105300 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781a9090 con 0x7f6778105300 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f676effd640 1 -- 172.21.15.29:0/2366471250 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67680054d0 con 0x7f6778105300 2024-09-21T01:52:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.932+0000 7f676effd640 1 -- 172.21.15.29:0/2366471250 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768010430 con 0x7f6778105300 2024-09-21T01:52:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.937+0000 7f676effd640 1 -- 172.21.15.29:0/2366471250 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f67680105d0 con 0x7f6778105300 2024-09-21T01:52:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.937+0000 7f6786261740 1 Processor -- start 2024-09-21T01:52:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.937+0000 7f6786261740 1 -- start start 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d7fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29b7390 0x5614e2d800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614e2cf5e50 con 0x5614e29af1b0 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614e2ce5280 con 0x5614e29d8070 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f6786261740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5614e2ce9090 con 0x5614e29b7390 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35768/0 (socket says 172.21.15.29:35768) 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 -- 172.21.15.29:0/2674904381 learned_addr learned my addr 172.21.15.29:0/2674904381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677da89640 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d7fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677d288640 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29b7390 0x5614e2d800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 -- 172.21.15.29:0/2674904381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29b7390 msgr2=0x5614e2d800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29b7390 0x5614e2d800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 -- 172.21.15.29:0/2674904381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 msgr2=0x5614e2d7fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d7fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.938+0000 7f677e28a640 1 -- 172.21.15.29:0/2674904381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e2a43650 con 0x5614e29d8070 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f677e28a640 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fc0 tx=0x7f6770010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bfc8e8273efb03c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f676effd640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 0x7f67640b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f6763fff640 1 -- 172.21.15.29:0/2674904381 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770003b70 con 0x5614e29d8070 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f6763fff640 1 -- 172.21.15.29:0/2674904381 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6770003d10 con 0x5614e29d8070 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f6763fff640 1 -- 172.21.15.29:0/2674904381 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770011a40 con 0x5614e29d8070 2024-09-21T01:52:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f677da89640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 0x7f67640b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.939+0000 7f676effd640 1 -- 172.21.15.29:0/2366471250 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f67680e7770 con 0x7f6778105300 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f677da89640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 0x7f67640b8aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6774000c00 tx=0x7f6774005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 -- 172.21.15.29:0/2674904381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 msgr2=0x5614e2d7f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fc0 tx=0x7f6770010860 comp rx=0 tx=0).stop 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 -- 172.21.15.29:0/2674904381 shutdown_connections 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29b7390 0x5614e2d800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d7fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 --2- 172.21.15.29:0/2674904381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29d8070 0x5614e2d7f650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.940+0000 7f6786261740 1 -- 172.21.15.29:0/2674904381 >> 172.21.15.29:0/2674904381 conn(0x5614e29cc460 msgr2=0x5614e2d7d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.941+0000 7f6786261740 1 -- 172.21.15.29:0/2674904381 shutdown_connections 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.941+0000 7f6786261740 1 -- 172.21.15.29:0/2674904381 wait complete. 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.941+0000 7f6786261740 1 -- start start 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.941+0000 7f6786261740 1 Processor -- start 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.942+0000 7f6786261740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.942+0000 7f6786261740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.942+0000 7f6786261740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 0x5614e2d97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.942+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f6786261740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614e2b46700 con 0x5614e29af1b0 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d8ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56894/0 (socket says 172.21.15.29:56894) 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677da89640 1 -- 172.21.15.29:0/2036538694 learned_addr learned my addr 172.21.15.29:0/2036538694 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614e2c80110 con 0x5614e29b7390 2024-09-21T01:52:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5614e2d8de30 con 0x5614e29d8070 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 0x5614e2d97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f6786261740 10 client.0 _ll_register_callbacks cb 0x5614e2c99990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 -- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 msgr2=0x5614e2d8b2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d8b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 -- 172.21.15.29:0/2036538694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 msgr2=0x5614e2d8ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d8ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.943+0000 7f677e28a640 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e2cadc40 con 0x5614e29d8070 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.944+0000 7f677e28a640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 0x5614e2d97120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6770006010 tx=0x7f6770010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9dffef75a3b25e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.944+0000 7f67617fa640 10 client.26002 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.944+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6770003d10 con 0x5614e29d8070 2024-09-21T01:52:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.944+0000 7f67617fa640 1 -- 172.21.15.29:0/2036538694 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677002a060 con 0x5614e29d8070 2024-09-21T01:52:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.944+0000 7f67617fa640 1 -- 172.21.15.29:0/2036538694 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6770011740 con 0x5614e29d8070 2024-09-21T01:52:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f67617fa640 1 -- 172.21.15.29:0/2036538694 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6770011960 con 0x5614e29d8070 2024-09-21T01:52:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 10 client.26002 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=36}) -- 0x5614e2d99a40 con 0x5614e29d8070 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f67617fa640 1 -- 172.21.15.29:0/2036538694 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f6770011b00 con 0x5614e29d8070 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 10 client.26002 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 20 client.26002 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29969f0 0x5614e2d9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 4 client.26002 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.945+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6770003d10 con 0x5614e29969f0 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.946+0000 7f677da89640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29969f0 0x5614e2d9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.946+0000 7f67617fa640 10 client.26002 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.946+0000 7f677da89640 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29969f0 0x5614e2d9cda0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a694236b74f4f7a2 server_cookie=21fee381c219ae1a in_seq=0 out_seq=0 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f67617fa640 1 -- 172.21.15.29:0/2036538694 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6770003d10 con 0x5614e29969f0 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f67617fa640 10 client.26002 handle_command_reply: tid=0 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 client.26002 shutdown 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 msgr2=0x5614e2d97120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 0x5614e2d97120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6770006010 tx=0x7f6770010fd0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 shutdown_connections 2024-09-21T01:52:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29969f0 0x5614e2d9cda0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29d8070 0x5614e2d97120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f67617fa640 0 client.26002 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d8b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 --2- 172.21.15.29:0/2036538694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29af1b0 0x5614e2d8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.947+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 >> 172.21.15.29:0/2036538694 conn(0x5614e29cc460 msgr2=0x5614e2d82830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.948+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 shutdown_connections 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.948+0000 7f6786261740 1 -- 172.21.15.29:0/2036538694 wait complete. 2024-09-21T01:52:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.948+0000 7f6786261740 20 client.26002 trim_cache size 0 max 16384 2024-09-21T01:52:32.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 235.0 s, 3886.0 tps, lat 25.729 ms stddev 30.191 2024-09-21T01:52:32.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 Processor -- start 2024-09-21T01:52:32.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 -- start start 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d80d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67781b1610 con 0x5614e29969f0 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67781b32a0 con 0x5614e29b7390 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f6786261740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67781b1ce0 con 0x5614e29af1b0 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56910/0 (socket says 172.21.15.29:56910) 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.975+0000 7f677da89640 1 -- 172.21.15.29:0/2482483333 learned_addr learned my addr 172.21.15.29:0/2482483333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677e28a640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677d288640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d80d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 -- 172.21.15.29:0/2482483333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 msgr2=0x5614e2d80d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d80d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 -- 172.21.15.29:0/2482483333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 msgr2=0x5614e2d7cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d7cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 -- 172.21.15.29:0/2482483333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e2d80470 con 0x5614e29969f0 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677d288640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d80d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.976+0000 7f677da89640 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6774006010 tx=0x7f677403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550d2df65455ca98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.978+0000 7f67637fe640 1 -- 172.21.15.29:0/2482483333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774035040 con 0x5614e29969f0 2024-09-21T01:52:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.978+0000 7f67637fe640 1 -- 172.21.15.29:0/2482483333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6774055070 con 0x5614e29969f0 2024-09-21T01:52:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.978+0000 7f67637fe640 1 -- 172.21.15.29:0/2482483333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774004920 con 0x5614e29969f0 2024-09-21T01:52:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- 172.21.15.29:0/2482483333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 msgr2=0x5614e2d89c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6774006010 tx=0x7f677403e000 comp rx=0 tx=0).stop 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- 172.21.15.29:0/2482483333 shutdown_connections 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d7cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d80d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 --2- 172.21.15.29:0/2482483333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2d89c80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- 172.21.15.29:0/2482483333 >> 172.21.15.29:0/2482483333 conn(0x5614e29cc460 msgr2=0x5614e2d80680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- 172.21.15.29:0/2482483333 shutdown_connections 2024-09-21T01:52:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- 172.21.15.29:0/2482483333 wait complete. 2024-09-21T01:52:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 -- start start 2024-09-21T01:52:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.979+0000 7f6786261740 1 Processor -- start 2024-09-21T01:52:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d9e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67781b2b70 con 0x5614e29969f0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67781b2470 con 0x5614e29b7390 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67781af5f0 con 0x5614e29af1b0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f6786261740 10 client.0 _ll_register_callbacks cb 0x7f67700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f677da89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56922/0 (socket says 172.21.15.29:56922) 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f677da89640 1 -- 172.21.15.29:0/304154624 learned_addr learned my addr 172.21.15.29:0/304154624 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f677d288640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.980+0000 7f677e28a640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d9e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 -- 172.21.15.29:0/304154624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 msgr2=0x5614e2d9dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d9dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 -- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 msgr2=0x5614e2d9e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d9e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 -- 172.21.15.29:0/304154624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e2e06520 con 0x5614e29969f0 2024-09-21T01:52:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677e28a640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d9e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f677da89640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6774009650 tx=0x7f67740041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe3f72c1f6303cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f67617fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:52:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.981+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677405d020 con 0x5614e29969f0 2024-09-21T01:52:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6774005040 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6774055070 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6774035070 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 10 client.16245 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5614e2e07600 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6774055070 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f67740035f0 con 0x5614e29969f0 2024-09-21T01:52:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 10 client.16245 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 20 client.16245 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29d8070 0x5614e2e09d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 4 client.16245 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614e2cf7da0 con 0x5614e29d8070 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.982+0000 7f677d288640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29d8070 0x5614e2e09d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.983+0000 7f67617fa640 10 client.16245 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.983+0000 7f677d288640 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29d8070 0x5614e2e09d70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=289c283725b871da server_cookie=bde3fa8c0d1f2216 in_seq=0 out_seq=0 2024-09-21T01:52:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.983+0000 7f67617fa640 1 -- 172.21.15.29:0/304154624 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5614e2cf7da0 con 0x5614e29d8070 2024-09-21T01:52:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.983+0000 7f67617fa640 10 client.16245 handle_command_reply: tid=0 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 client.16245 shutdown 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 msgr2=0x5614e2da0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6774009650 tx=0x7f67740041c0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 shutdown_connections 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5614e29d8070 0x5614e2e09d70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e29b7390 0x5614e2d9e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614e29af1b0 0x5614e2d9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 --2- 172.21.15.29:0/304154624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614e29969f0 0x5614e2da0740 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f67617fa640 0 client.16245 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.984+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 >> 172.21.15.29:0/304154624 conn(0x5614e29cc460 msgr2=0x5614e2d8c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.985+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 shutdown_connections 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.985+0000 7f6786261740 1 -- 172.21.15.29:0/304154624 wait complete. 2024-09-21T01:52:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.986+0000 7f6786261740 20 client.16245 trim_cache size 0 max 16384 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:32.991 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:52:32.992 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:32.993 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 msgr2=0x7f67640b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 0x7f67640b8aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6774000c00 tx=0x7f6774005a20 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 msgr2=0x7f677819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f676800e9c0 tx=0x7f676800ee80 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 shutdown_connections 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f67640b6590 0x7f67640b8aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677810de10 0x7f67781a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6778105300 0x7f677819cc70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 --2- 172.21.15.29:0/2366471250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67781049b0 0x7f677819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:32.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 >> 172.21.15.29:0/2366471250 conn(0x7f6778079730 msgr2=0x7f677810f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:32.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 shutdown_connections 2024-09-21T01:52:32.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:32.987+0000 7f677fced640 1 -- 172.21.15.29:0/2366471250 wait complete. 2024-09-21T01:52:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:52:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:52:33.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:52:33.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:52:33.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:52:33.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:52:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:52:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:52:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:52:33.991 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:33 smithi029 ceph-mon[31228]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-09-21T01:52:34.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:52:34.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:52:34.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:33 smithi066 ceph-mon[35001]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-09-21T01:52:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:33 smithi190 ceph-mon[34368]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-09-21T01:52:34.487 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 -- 172.21.15.29:0/1392783568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962410b5e0 msgr2=0x7f962410daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 --2- 172.21.15.29:0/1392783568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962410b5e0 0x7f962410daf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f961c009bd0 tx=0x7f961c02a6a0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 -- 172.21.15.29:0/1392783568 shutdown_connections 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 --2- 172.21.15.29:0/1392783568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962410b5e0 0x7f962410daf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 --2- 172.21.15.29:0/1392783568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9624108c80 0x7f962410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 --2- 172.21.15.29:0/1392783568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962406b4e0 0x7f9624108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 -- 172.21.15.29:0/1392783568 >> 172.21.15.29:0/1392783568 conn(0x7f96240fdee0 msgr2=0x7f9624100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 -- 172.21.15.29:0/1392783568 shutdown_connections 2024-09-21T01:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.041+0000 7f9628a9b640 1 -- 172.21.15.29:0/1392783568 wait complete. 2024-09-21T01:52:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.042+0000 7f9628a9b640 1 Processor -- start 2024-09-21T01:52:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.043+0000 7f9628a9b640 1 -- start start 2024-09-21T01:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.043+0000 7f9628a9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962406b4e0 0x7f962419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.043+0000 7f9628a9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9624108c80 0x7f962419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9628a9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9628a9b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9624111520 con 0x7f962406b4e0 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9628a9b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96241113a0 con 0x7f962410b5e0 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9628a9b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96241116a0 con 0x7f9624108c80 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35786/0 (socket says 172.21.15.29:35786) 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 -- 172.21.15.29:0/2390054900 learned_addr learned my addr 172.21.15.29:0/2390054900 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f962259c640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962406b4e0 0x7f962419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9621d9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9624108c80 0x7f962419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 -- 172.21.15.29:0/2390054900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9624108c80 msgr2=0x7f962419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9624108c80 0x7f962419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.044+0000 7f9622d9d640 1 -- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962406b4e0 msgr2=0x7f962419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.045+0000 7f9622d9d640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962406b4e0 0x7f962419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.045+0000 7f9622d9d640 1 -- 172.21.15.29:0/2390054900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962419d9e0 con 0x7f962410b5e0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.045+0000 7f9622d9d640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f961c009af0 tx=0x7f961c003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.045+0000 7f96137fe640 1 -- 172.21.15.29:0/2390054900 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c040020 con 0x7f962410b5e0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.045+0000 7f96137fe640 1 -- 172.21.15.29:0/2390054900 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961c030de0 con 0x7f962410b5e0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.046+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962419dc70 con 0x7f962410b5e0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.046+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962419df80 con 0x7f962410b5e0 2024-09-21T01:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.046+0000 7f96137fe640 1 -- 172.21.15.29:0/2390054900 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c036040 con 0x7f962410b5e0 2024-09-21T01:52:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.049+0000 7f962fa09740 1 Processor -- start 2024-09-21T01:52:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.049+0000 7f96137fe640 1 -- 172.21.15.29:0/2390054900 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f961c043020 con 0x7f962410b5e0 2024-09-21T01:52:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.049+0000 7f962fa09740 1 -- start start 2024-09-21T01:52:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 0x56142d287e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cebf4d0 0x56142d289010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56142d1fa6f0 con 0x56142cebf4d0 2024-09-21T01:52:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56142d1e9b20 con 0x56142cee01b0 2024-09-21T01:52:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962fa09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56142d1ed930 con 0x56142ceb72f0 2024-09-21T01:52:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52150/0 (socket says 172.21.15.29:52150) 2024-09-21T01:52:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.050+0000 7f962259c640 1 -- 172.21.15.29:0/927913829 learned_addr learned my addr 172.21.15.29:0/927913829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f9621d9b640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cebf4d0 0x56142d289010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f9621d9b640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 0x56142d287e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f96137fe640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 0x7f960c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f962259c640 1 -- 172.21.15.29:0/927913829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 msgr2=0x56142d287e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f962259c640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 0x56142d287e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f962259c640 1 -- 172.21.15.29:0/927913829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cebf4d0 msgr2=0x56142d289010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f962259c640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cebf4d0 0x56142d289010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f96137fe640 1 -- 172.21.15.29:0/2390054900 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f961c107fb0 con 0x7f962410b5e0 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f962259c640 1 -- 172.21.15.29:0/927913829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142d1af530 con 0x56142ceb72f0 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.051+0000 7f9621d9b640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 0x56142d287e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.052+0000 7f9622d9d640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 0x7f960c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.052+0000 7f962259c640 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9614009cf0 tx=0x7f9614011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67cd89c4aeeb1305 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.052+0000 7f9622d9d640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 0x7f960c0b8ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f961c009ba0 tx=0x7f961c004100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.053+0000 7f9610ff9640 1 -- 172.21.15.29:0/927913829 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614004030 con 0x56142ceb72f0 2024-09-21T01:52:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.053+0000 7f9610ff9640 1 -- 172.21.15.29:0/927913829 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96140041d0 con 0x56142ceb72f0 2024-09-21T01:52:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.053+0000 7f9610ff9640 1 -- 172.21.15.29:0/927913829 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614019770 con 0x56142ceb72f0 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 -- 172.21.15.29:0/927913829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 msgr2=0x56142d2884c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9614009cf0 tx=0x7f9614011810 comp rx=0 tx=0).stop 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 -- 172.21.15.29:0/927913829 shutdown_connections 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cebf4d0 0x56142d289010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2884c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 --2- 172.21.15.29:0/927913829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cee01b0 0x56142d287e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.054+0000 7f962fa09740 1 -- 172.21.15.29:0/927913829 >> 172.21.15.29:0/927913829 conn(0x56142ced4300 msgr2=0x56142d2838e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.055+0000 7f962fa09740 1 -- 172.21.15.29:0/927913829 shutdown_connections 2024-09-21T01:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.055+0000 7f962fa09740 1 -- 172.21.15.29:0/927913829 wait complete. 2024-09-21T01:52:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 -- start start 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 Processor -- start 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d291d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56142d121ab0 con 0x56142cee01b0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56142d18f730 con 0x56142cebf4d0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56142d289b80 con 0x56142ceb72f0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962fa09740 10 client.0 _ll_register_callbacks cb 0x56142d1e36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f9621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f962259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d291d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f9621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56934/0 (socket says 172.21.15.29:56934) 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f9621d9b640 1 -- 172.21.15.29:0/1943914273 learned_addr learned my addr 172.21.15.29:0/1943914273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.056+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 -- 172.21.15.29:0/1943914273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 msgr2=0x56142d2916c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2916c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 -- 172.21.15.29:0/1943914273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 msgr2=0x56142d2928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 -- 172.21.15.29:0/1943914273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142cf4b790 con 0x56142cebf4d0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f962259c640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d291d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9614009cf0 tx=0x7f9614003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547ba5c5319358d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f9609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.057+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614027020 con 0x56142cebf4d0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96140041d0 con 0x56142cebf4d0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9614005040 con 0x56142cebf4d0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9614005240 con 0x56142cebf4d0 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 10 client.25970 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56142cfc4620 con 0x56142cebf4d0 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96140041d0 con 0x56142cebf4d0 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f9614019490 con 0x56142cebf4d0 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 10 client.25970 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 20 client.25970 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce9ed80 0x56142d29f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 4 client.25970 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.058+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142d1fc640 con 0x56142ce9ed80 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.059+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce9ed80 0x56142d29f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.059+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce9ed80 0x56142d29f2a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4b3e7142b795e server_cookie=3f5f72c4a952c040 in_seq=0 out_seq=0 2024-09-21T01:52:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.059+0000 7f9609ffb640 10 client.25970 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.060+0000 7f9609ffb640 1 -- 172.21.15.29:0/1943914273 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56142d1fc640 con 0x56142ce9ed80 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.060+0000 7f9609ffb640 10 client.25970 handle_command_reply: tid=0 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.060+0000 7f962fa09740 1 client.25970 shutdown 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 msgr2=0x56142d291d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d291d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9614009cf0 tx=0x7f9614003ba0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 shutdown_connections 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce9ed80 0x56142d29f2a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d291d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 --2- 172.21.15.29:0/1943914273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d2916c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 >> 172.21.15.29:0/1943914273 conn(0x56142ced4300 msgr2=0x56142d281ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.061+0000 7f9609ffb640 0 client.25970 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.062+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 shutdown_connections 2024-09-21T01:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.062+0000 7f962fa09740 1 -- 172.21.15.29:0/1943914273 wait complete. 2024-09-21T01:52:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.062+0000 7f962fa09740 20 client.25970 trim_cache size 0 max 16384 2024-09-21T01:52:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 Processor -- start 2024-09-21T01:52:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 -- start start 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d290110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142cee01b0 0x56142d290650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b1480 con 0x56142ceb72f0 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b3110 con 0x56142cebf4d0 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962fa09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b1b50 con 0x56142cee01b0 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f9621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f962259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d290110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f9621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56936/0 (socket says 172.21.15.29:56936) 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.084+0000 7f9621d9b640 1 -- 172.21.15.29:0/3574750590 learned_addr learned my addr 172.21.15.29:0/3574750590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f9621d9b640 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142cee01b0 0x56142d290650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 -- 172.21.15.29:0/3574750590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142cee01b0 msgr2=0x56142d290650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142cee01b0 0x56142d290650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 -- 172.21.15.29:0/3574750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 msgr2=0x56142d293000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 -- 172.21.15.29:0/3574750590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142d292080 con 0x56142cebf4d0 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f9621d9b640 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f962259c640 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d290110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96140199d0 tx=0x7f9614011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f1f94b4c871b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f960bfff640 1 -- 172.21.15.29:0/3574750590 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614027040 con 0x56142cebf4d0 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f960bfff640 1 -- 172.21.15.29:0/3574750590 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614003e10 con 0x56142cebf4d0 2024-09-21T01:52:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.085+0000 7f960bfff640 1 -- 172.21.15.29:0/3574750590 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614005640 con 0x56142cebf4d0 2024-09-21T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.086+0000 7f962fa09740 1 -- 172.21.15.29:0/3574750590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 msgr2=0x56142d290110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.086+0000 7f962fa09740 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d290110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96140199d0 tx=0x7f9614011810 comp rx=0 tx=0).stop 2024-09-21T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.087+0000 7f962fa09740 1 -- 172.21.15.29:0/3574750590 shutdown_connections 2024-09-21T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.087+0000 7f962fa09740 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142cee01b0 0x56142d290650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.087+0000 7f962fa09740 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d290110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.087+0000 7f962fa09740 1 --2- 172.21.15.29:0/3574750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142ceb72f0 0x56142d293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.087+0000 7f962fa09740 1 -- 172.21.15.29:0/3574750590 >> 172.21.15.29:0/3574750590 conn(0x56142ced4300 msgr2=0x56142d2928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.088+0000 7f962fa09740 1 -- 172.21.15.29:0/3574750590 shutdown_connections 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.088+0000 7f962fa09740 1 -- 172.21.15.29:0/3574750590 wait complete. 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.088+0000 7f962fa09740 1 -- start start 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.088+0000 7f962fa09740 1 Processor -- start 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d295fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b29e0 con 0x56142cee01b0 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b22e0 con 0x56142cebf4d0 2024-09-21T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96241af460 con 0x56142ceb72f0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962fa09740 10 client.0 _ll_register_callbacks cb 0x7f9614009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f9621d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d295fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35818/0 (socket says 172.21.15.29:35818) 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f962259c640 1 -- 172.21.15.29:0/1648799081 learned_addr learned my addr 172.21.15.29:0/1648799081 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.089+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 -- 172.21.15.29:0/1648799081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 msgr2=0x56142d295fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d295fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 -- 172.21.15.29:0/1648799081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 msgr2=0x56142d2971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 -- 172.21.15.29:0/1648799081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142d297940 con 0x56142cebf4d0 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962259c640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9614003dc0 tx=0x7f9614005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ea8219985f5c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f9609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961403e070 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9614023870 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.090+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9614003e10 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f961404a050 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 10 client.25976 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56142d298c60 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614003e10 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f9614035020 con 0x56142cebf4d0 2024-09-21T01:52:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 10 client.25976 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 20 client.25976 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce741c0 0x56142d2996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.091+0000 7f962fa09740 4 client.25976 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.092+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56142d1f80f0 con 0x56142ce741c0 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.092+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce741c0 0x56142d2996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.093+0000 7f9609ffb640 10 client.25976 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.093+0000 7f9621d9b640 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce741c0 0x56142d2996c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55889eb422b55a5d server_cookie=9471d7f02a6a9f2f in_seq=0 out_seq=0 2024-09-21T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.108+0000 7f9609ffb640 1 -- 172.21.15.29:0/1648799081 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56142d1f80f0 con 0x56142ce741c0 2024-09-21T01:52:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.108+0000 7f9609ffb640 10 client.25976 handle_command_reply: tid=0 2024-09-21T01:52:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.108+0000 7f962fa09740 1 client.25976 shutdown 2024-09-21T01:52:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.108+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 msgr2=0x56142d296670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.108+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9614003dc0 tx=0x7f9614005990 comp rx=0 tx=0).stop 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 shutdown_connections 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56142ce741c0 0x56142d2996c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56142cee01b0 0x56142d2971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56142cebf4d0 0x56142d296670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f9609ffb640 0 client.25976 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 --2- 172.21.15.29:0/1648799081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56142ceb72f0 0x56142d295fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 >> 172.21.15.29:0/1648799081 conn(0x56142ced4300 msgr2=0x56142d284050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 shutdown_connections 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 1 -- 172.21.15.29:0/1648799081 wait complete. 2024-09-21T01:52:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.109+0000 7f962fa09740 20 client.25976 trim_cache size 0 max 16384 2024-09-21T01:52:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:52:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:52:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:52:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 msgr2=0x7f960c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 0x7f960c0b8ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f961c009ba0 tx=0x7f961c004100 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 msgr2=0x7f962419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f961c009af0 tx=0x7f961c003f20 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 shutdown_connections 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f960c0b65d0 0x7f960c0b8ae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f962410b5e0 0x7f962419d1c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9624108c80 0x7f962419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 --2- 172.21.15.29:0/2390054900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962406b4e0 0x7f962419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 >> 172.21.15.29:0/2390054900 conn(0x7f96240fdee0 msgr2=0x7f9624109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.110+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 shutdown_connections 2024-09-21T01:52:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:35.111+0000 7f9628a9b640 1 -- 172.21.15.29:0/2390054900 wait complete. 2024-09-21T01:52:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:35 smithi066 ceph-mon[35001]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-09-21T01:52:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:35 smithi029 ceph-mon[31228]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-09-21T01:52:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:35 smithi190 ceph-mon[34368]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 -- 172.21.15.29:0/406033425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 msgr2=0x7fab681087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 --2- 172.21.15.29:0/406033425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab681087b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009900 tx=0x7fab5c02a410 comp rx=0 tx=0).stop 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 -- 172.21.15.29:0/406033425 shutdown_connections 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 --2- 172.21.15.29:0/406033425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68108cf0 0x7fab6810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 --2- 172.21.15.29:0/406033425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab681087b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:36.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 --2- 172.21.15.29:0/406033425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68102a60 0x7fab68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:36.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 -- 172.21.15.29:0/406033425 >> 172.21.15.29:0/406033425 conn(0x7fab68079730 msgr2=0x7fab68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:36.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 -- 172.21.15.29:0/406033425 shutdown_connections 2024-09-21T01:52:36.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.656+0000 7fab701c0640 1 -- 172.21.15.29:0/406033425 wait complete. 2024-09-21T01:52:36.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.657+0000 7fab701c0640 1 Processor -- start 2024-09-21T01:52:36.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.657+0000 7fab701c0640 1 -- start start 2024-09-21T01:52:36.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab6806f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68108cf0 0x7fab6806f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806d6a0 con 0x7fab68108cf0 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806d520 con 0x7fab681083a0 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab701c0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806d820 con 0x7fab68102a60 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6e75d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68108cf0 0x7fab6806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52190/0 (socket says 172.21.15.29:52190) 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 -- 172.21.15.29:0/875888188 learned_addr learned my addr 172.21.15.29:0/875888188 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6d75b640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab6806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 -- 172.21.15.29:0/875888188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 msgr2=0x7fab6806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab6806f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 -- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68108cf0 msgr2=0x7fab6806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:36.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68108cf0 0x7fab6806f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:36.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.658+0000 7fab6df5c640 1 -- 172.21.15.29:0/875888188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6806ff50 con 0x7fab68102a60 2024-09-21T01:52:36.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.659+0000 7fab6df5c640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:36.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.660+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6000cd80 con 0x7fab68102a60 2024-09-21T01:52:36.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.660+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab600108e0 con 0x7fab68102a60 2024-09-21T01:52:36.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.660+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6810faf0 con 0x7fab68102a60 2024-09-21T01:52:36.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.660+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60018980 con 0x7fab68102a60 2024-09-21T01:52:36.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.660+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6810ff40 con 0x7fab68102a60 2024-09-21T01:52:36.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.661+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68102e70 con 0x7fab68102a60 2024-09-21T01:52:36.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.663+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fab60010430 con 0x7fab68102a60 2024-09-21T01:52:36.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.664+0000 7fab66ffd640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 0x7fab500b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:36.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.665+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fab600e7860 con 0x7fab68102a60 2024-09-21T01:52:36.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.665+0000 7fab6d75b640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 0x7fab500b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:36.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.666+0000 7fab6d75b640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 0x7fab500b8c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006010 tx=0x7fab5c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:36.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:36.667+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab600ae120 con 0x7fab68102a60 2024-09-21T01:52:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.049+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab68070470 con 0x7fab68102a60 2024-09-21T01:52:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.050+0000 7fab66ffd640 1 -- 172.21.15.29:0/875888188 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11981 (secure 0 0 0) 0x7fab68070470 con 0x7fab68102a60 2024-09-21T01:52:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:52:37.051 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":36,"btime":"2024-09-21T01:52:30:751660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:52:30.234816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:52:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 36 2024-09-21T01:52:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 msgr2=0x7fab500b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 0x7fab500b8c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006010 tx=0x7fab5c031400 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 msgr2=0x7fab68074eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 shutdown_connections 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fab500b6720 0x7fab500b8c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab68108cf0 0x7fab6806f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab681083a0 0x7fab6806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 --2- 172.21.15.29:0/875888188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab68102a60 0x7fab68074eb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 >> 172.21.15.29:0/875888188 conn(0x7fab68079730 msgr2=0x7fab68104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 shutdown_connections 2024-09-21T01:52:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:37.054+0000 7fab701c0640 1 -- 172.21.15.29:0/875888188 wait complete. 2024-09-21T01:52:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 240.0 s, 2933.0 tps, lat 34.077 ms stddev 45.343 2024-09-21T01:52:38.045 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:37 smithi029 ceph-mon[31228]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 869 op/s 2024-09-21T01:52:38.046 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:37 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/875888188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:52:38.048 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:37 smithi066 ceph-mon[35001]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 869 op/s 2024-09-21T01:52:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:37 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/875888188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:52:38.231 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:37 smithi190 ceph-mon[34368]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 869 op/s 2024-09-21T01:52:38.231 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:37 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/875888188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:52:38.819 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:52:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 -- 172.21.15.29:0/1610670688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa41083a0 msgr2=0x7feaa41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 --2- 172.21.15.29:0/1610670688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa41083a0 0x7feaa41087b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea9802a410 comp rx=0 tx=0).stop 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 -- 172.21.15.29:0/1610670688 shutdown_connections 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 --2- 172.21.15.29:0/1610670688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 --2- 172.21.15.29:0/1610670688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa41083a0 0x7feaa41087b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 --2- 172.21.15.29:0/1610670688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa4102a60 0x7feaa4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 -- 172.21.15.29:0/1610670688 >> 172.21.15.29:0/1610670688 conn(0x7feaa4079730 msgr2=0x7feaa4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 -- 172.21.15.29:0/1610670688 shutdown_connections 2024-09-21T01:52:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.778+0000 7feaa92dc640 1 -- 172.21.15.29:0/1610670688 wait complete. 2024-09-21T01:52:39.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 Processor -- start 2024-09-21T01:52:39.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 -- start start 2024-09-21T01:52:39.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa41a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4115c60 con 0x7feaa4102a60 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4115ae0 con 0x7feaa41083a0 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.780+0000 7feaa92dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4115de0 con 0x7feaa4108cf0 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56980/0 (socket says 172.21.15.29:56980) 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35864/0 (socket says 172.21.15.29:35864) 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 -- 172.21.15.29:0/3484730107 learned_addr learned my addr 172.21.15.29:0/3484730107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa41a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 -- 172.21.15.29:0/3484730107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 msgr2=0x7feaa41a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa41a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 -- 172.21.15.29:0/3484730107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 msgr2=0x7feaa419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 -- 172.21.15.29:0/3484730107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a2680 con 0x7feaa4102a60 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa27fc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7feaa2ffd640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9f0 tx=0x7fea9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.781+0000 7fea8bfff640 1 -- 172.21.15.29:0/3484730107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400cd80 con 0x7feaa4102a60 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.782+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a2970 con 0x7feaa4102a60 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.782+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41a90d0 con 0x7feaa4102a60 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.782+0000 7fea8bfff640 1 -- 172.21.15.29:0/3484730107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea940108e0 con 0x7feaa4102a60 2024-09-21T01:52:39.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.782+0000 7fea8bfff640 1 -- 172.21.15.29:0/3484730107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94018980 con 0x7feaa4102a60 2024-09-21T01:52:39.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.785+0000 7feaaf850740 1 Processor -- start 2024-09-21T01:52:39.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 -- start start 2024-09-21T01:52:39.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff18f8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff18f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dff1865ea0 con 0x55dff151f1b0 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff18552d0 con 0x55dff1527390 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaaf850740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dff18590e0 con 0x55dff1548070 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52228/0 (socket says 172.21.15.29:52228) 2024-09-21T01:52:39.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.786+0000 7feaa37fe640 1 -- 172.21.15.29:0/3758027085 learned_addr learned my addr 172.21.15.29:0/3758027085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 -- 172.21.15.29:0/3758027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 msgr2=0x55dff18f8550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff18f8550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 -- 172.21.15.29:0/3758027085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 msgr2=0x55dff18f8010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff18f8010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 -- 172.21.15.29:0/3758027085 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff15b3650 con 0x55dff1548070 2024-09-21T01:52:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea9c011790 tx=0x7fea9c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4a94a26b9e7f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.787+0000 7fea8bfff640 1 -- 172.21.15.29:0/3484730107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fea94010430 con 0x7feaa4102a60 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.789+0000 7fea8bfff640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 0x7fea8c0b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.789+0000 7fea8bfff640 1 -- 172.21.15.29:0/3484730107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fea940e7600 con 0x7feaa4102a60 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.790+0000 7fea897fa640 1 -- 172.21.15.29:0/3758027085 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c003c30 con 0x55dff1548070 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.790+0000 7fea897fa640 1 -- 172.21.15.29:0/3758027085 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c00fe50 con 0x55dff1548070 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.790+0000 7fea897fa640 1 -- 172.21.15.29:0/3758027085 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0193e0 con 0x55dff1548070 2024-09-21T01:52:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.790+0000 7feaa2ffd640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 0x7fea8c0b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- 172.21.15.29:0/3758027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 msgr2=0x55dff18f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea9c011790 tx=0x7fea9c011c50 comp rx=0 tx=0).stop 2024-09-21T01:52:39.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaa2ffd640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 0x7fea8c0b8b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fea940ad4f0 tx=0x7fea9400cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- 172.21.15.29:0/3758027085 shutdown_connections 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff18f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff18f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 --2- 172.21.15.29:0/3758027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1548070 0x55dff18f7ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- 172.21.15.29:0/3758027085 >> 172.21.15.29:0/3758027085 conn(0x55dff153c460 msgr2=0x55dff18f59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- 172.21.15.29:0/3758027085 shutdown_connections 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- 172.21.15.29:0/3758027085 wait complete. 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.791+0000 7feaaf850740 1 -- start start 2024-09-21T01:52:39.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 Processor -- start 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff1903290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1548070 0x55dff190f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dff16b6700 con 0x55dff151f1b0 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff17f0160 con 0x55dff1548070 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaaf850740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dff19062b0 con 0x55dff1527390 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.792+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52244/0 (socket says 172.21.15.29:52244) 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 -- 172.21.15.29:0/2913416402 learned_addr learned my addr 172.21.15.29:0/2913416402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:39.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa37fe640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff1903290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaaf850740 10 client.0 _ll_register_callbacks cb 0x55dff18099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 -- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1548070 msgr2=0x55dff190f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1548070 0x55dff190f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 -- 172.21.15.29:0/2913416402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 msgr2=0x55dff1903290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff1903290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.793+0000 7feaa27fc640 1 -- 172.21.15.29:0/2913416402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff181dc90 con 0x55dff1527390 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.794+0000 7feaa27fc640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea980045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52eae3fa2a902fea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.794+0000 7fea86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.794+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98035020 con 0x55dff1527390 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.794+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fea98031400 con 0x55dff1527390 2024-09-21T01:52:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.794+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c00fe50 con 0x55dff1527390 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fea98004990 con 0x55dff1527390 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 10 client.26044 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dff19120c0 con 0x55dff1527390 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c00fe50 con 0x55dff1527390 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7fea9803a040 con 0x55dff1527390 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 10 client.26044 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 10 client.26044 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:52:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 20 client.26044 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff15069f0 0x55dff19155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 4 client.26044 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.795+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff1867df0 con 0x55dff15069f0 2024-09-21T01:52:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.796+0000 7feaa37fe640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff15069f0 0x55dff19155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.796+0000 7fea86ff5640 10 client.26044 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:52:39.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.796+0000 7feaa37fe640 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff15069f0 0x55dff19155d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34add20c2cbd1ec server_cookie=bfa96e411a683539 in_seq=0 out_seq=0 2024-09-21T01:52:39.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.797+0000 7fea86ff5640 1 -- 172.21.15.29:0/2913416402 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dff1867df0 con 0x55dff15069f0 2024-09-21T01:52:39.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.797+0000 7fea86ff5640 10 client.26044 handle_command_reply: tid=0 2024-09-21T01:52:39.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.797+0000 7feaaf850740 1 client.26044 shutdown 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.797+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 msgr2=0x55dff19037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea980045d0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 shutdown_connections 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff15069f0 0x55dff19155d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7fea86ff5640 0 client.26044 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1548070 0x55dff190f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff1527390 0x55dff19037d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 --2- 172.21.15.29:0/2913416402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff151f1b0 0x55dff1903290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 >> 172.21.15.29:0/2913416402 conn(0x55dff153c460 msgr2=0x55dff18fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 shutdown_connections 2024-09-21T01:52:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 1 -- 172.21.15.29:0/2913416402 wait complete. 2024-09-21T01:52:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.798+0000 7feaaf850740 20 client.26044 trim_cache size 0 max 16384 2024-09-21T01:52:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.825+0000 7feaaf850740 1 Processor -- start 2024-09-21T01:52:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 -- start start 2024-09-21T01:52:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1900480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ad320 con 0x55dff15069f0 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41aefb0 con 0x55dff1527390 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.829+0000 7feaaf850740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ad9f0 con 0x55dff151f1b0 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.832+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1900480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.832+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.832+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57002/0 (socket says 172.21.15.29:57002) 2024-09-21T01:52:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.832+0000 7feaa37fe640 1 -- 172.21.15.29:0/3982913197 learned_addr learned my addr 172.21.15.29:0/3982913197 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.834+0000 7feaa27fc640 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 -- 172.21.15.29:0/3982913197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 msgr2=0x55dff190f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 -- 172.21.15.29:0/3982913197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 msgr2=0x55dff1900480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1900480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 -- 172.21.15.29:0/3982913197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff18fa030 con 0x55dff15069f0 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.836+0000 7feaa37fe640 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea9c014040 tx=0x7fea9c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad821019b616405 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.837+0000 7fea88ff9640 1 -- 172.21.15.29:0/3982913197 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c01f040 con 0x55dff15069f0 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.837+0000 7fea88ff9640 1 -- 172.21.15.29:0/3982913197 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c011250 con 0x55dff15069f0 2024-09-21T01:52:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.837+0000 7fea88ff9640 1 -- 172.21.15.29:0/3982913197 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004cf0 con 0x55dff15069f0 2024-09-21T01:52:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.838+0000 7feaaf850740 1 -- 172.21.15.29:0/3982913197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 msgr2=0x55dff1903750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea9c014040 tx=0x7fea9c005ab0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 -- 172.21.15.29:0/3982913197 shutdown_connections 2024-09-21T01:52:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1900480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 --2- 172.21.15.29:0/3982913197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff1903750 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.839+0000 7feaaf850740 1 -- 172.21.15.29:0/3982913197 >> 172.21.15.29:0/3982913197 conn(0x55dff153c460 msgr2=0x55dff1903040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.841+0000 7feaaf850740 1 -- 172.21.15.29:0/3982913197 shutdown_connections 2024-09-21T01:52:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.841+0000 7feaaf850740 1 -- 172.21.15.29:0/3982913197 wait complete. 2024-09-21T01:52:39.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.841+0000 7feaaf850740 1 -- start start 2024-09-21T01:52:39.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.841+0000 7feaaf850740 1 Processor -- start 2024-09-21T01:52:39.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff190b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1979080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ae880 con 0x55dff15069f0 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ae180 con 0x55dff1527390 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.842+0000 7feaaf850740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ab300 con 0x55dff151f1b0 2024-09-21T01:52:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.843+0000 7feaaf850740 10 client.0 _ll_register_callbacks cb 0x7fea9c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.844+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.844+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52254/0 (socket says 172.21.15.29:52254) 2024-09-21T01:52:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.844+0000 7feaa27fc640 1 -- 172.21.15.29:0/343500789 learned_addr learned my addr 172.21.15.29:0/343500789 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.844+0000 7feaa37fe640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1979080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.844+0000 7feaa37fe640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff190b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.845+0000 7feaa27fc640 1 -- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 msgr2=0x55dff1979080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.845+0000 7feaa27fc640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1979080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.845+0000 7feaa27fc640 1 -- 172.21.15.29:0/343500789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 msgr2=0x55dff190b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.845+0000 7feaa27fc640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff190b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.845+0000 7feaa27fc640 1 -- 172.21.15.29:0/343500789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff190c0b0 con 0x55dff151f1b0 2024-09-21T01:52:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.846+0000 7feaa27fc640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea980059d0 tx=0x7fea98005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b6877b2217641e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.846+0000 7fea86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.846+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98035040 con 0x55dff151f1b0 2024-09-21T01:52:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.846+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c011250 con 0x55dff151f1b0 2024-09-21T01:52:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.847+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fea98003e30 con 0x55dff151f1b0 2024-09-21T01:52:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.848+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fea980043e0 con 0x55dff151f1b0 2024-09-21T01:52:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.848+0000 7feaaf850740 10 client.26050 fetch_fsmap learned FSMap version 36 2024-09-21T01:52:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.848+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dff19913d0 con 0x55dff151f1b0 2024-09-21T01:52:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.850+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c011250 con 0x55dff151f1b0 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7fea9805a480 con 0x55dff151f1b0 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 10 client.26050 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 10 client.26050 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 20 client.26050 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff1548070 0x55dff1993bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 4 client.26050 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dff18638a0 con 0x55dff1548070 2024-09-21T01:52:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.851+0000 7feaa37fe640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff1548070 0x55dff1993bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.852+0000 7feaa37fe640 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff1548070 0x55dff1993bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa4815449bbaef28 server_cookie=c135c4a4dd2e5bb9 in_seq=0 out_seq=0 2024-09-21T01:52:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7fea86ff5640 10 client.26050 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:52:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7fea86ff5640 1 -- 172.21.15.29:0/343500789 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dff18638a0 con 0x55dff1548070 2024-09-21T01:52:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7fea86ff5640 10 client.26050 handle_command_reply: tid=0 2024-09-21T01:52:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7feaaf850740 1 client.26050 shutdown 2024-09-21T01:52:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 msgr2=0x55dff190ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.853+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea980059d0 tx=0x7fea98005fd0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 shutdown_connections 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55dff1548070 0x55dff1993bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff1527390 0x55dff1979080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dff151f1b0 0x55dff190ba70 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea980059d0 tx=0x7fea98005fd0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 --2- 172.21.15.29:0/343500789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dff15069f0 0x55dff190b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 >> 172.21.15.29:0/343500789 conn(0x55dff153c460 msgr2=0x55dff18f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.854+0000 7fea86ff5640 0 client.26050 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:52:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.855+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 shutdown_connections 2024-09-21T01:52:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.855+0000 7feaaf850740 1 -- 172.21.15.29:0/343500789 wait complete. 2024-09-21T01:52:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.855+0000 7feaaf850740 20 client.26050 trim_cache size 0 max 16384 2024-09-21T01:52:39.858 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 msgr2=0x7fea8c0b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 0x7fea8c0b8b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fea940ad4f0 tx=0x7fea9400cf20 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 msgr2=0x7feaa419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9f0 tx=0x7fea9400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 shutdown_connections 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea8c0b6600 0x7fea8c0b8b10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaa4108cf0 0x7feaa41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41083a0 0x7feaa419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 --2- 172.21.15.29:0/3484730107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa4102a60 0x7feaa419c710 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 >> 172.21.15.29:0/3484730107 conn(0x7feaa4079730 msgr2=0x7feaa4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 shutdown_connections 2024-09-21T01:52:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:39.857+0000 7feaa92dc640 1 -- 172.21.15.29:0/3484730107 wait complete. 2024-09-21T01:52:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:39 smithi066 ceph-mon[35001]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 623 op/s 2024-09-21T01:52:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:39 smithi029 ceph-mon[31228]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 623 op/s 2024-09-21T01:52:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:39 smithi190 ceph-mon[34368]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 623 op/s 2024-09-21T01:52:40.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:52:41.144 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 -- 172.21.15.29:0/1398566192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde24111f40 msgr2=0x7fde2410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 --2- 172.21.15.29:0/1398566192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde24111f40 0x7fde2410a3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fde1401a910 tx=0x7fde1403b430 comp rx=0 tx=0).stop 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 -- 172.21.15.29:0/1398566192 shutdown_connections 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 --2- 172.21.15.29:0/1398566192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 0x7fde2411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 --2- 172.21.15.29:0/1398566192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2410a9e0 0x7fde2410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 --2- 172.21.15.29:0/1398566192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde24111f40 0x7fde2410a3c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 -- 172.21.15.29:0/1398566192 >> 172.21.15.29:0/1398566192 conn(0x7fde24070ec0 msgr2=0x7fde240712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 -- 172.21.15.29:0/1398566192 shutdown_connections 2024-09-21T01:52:41.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.779+0000 7fde28ea2640 1 -- 172.21.15.29:0/1398566192 wait complete. 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.780+0000 7fde28ea2640 1 Processor -- start 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.780+0000 7fde28ea2640 1 -- start start 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde2410a9e0 0x7fde2411c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 0x7fde24118c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114c10 con 0x7fde24111f40 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114a90 con 0x7fde2411cc30 2024-09-21T01:52:41.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde28ea2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114d90 con 0x7fde2410a9e0 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57032/0 (socket says 172.21.15.29:57032) 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde21d9b640 1 -- 172.21.15.29:0/3694803139 learned_addr learned my addr 172.21.15.29:0/3694803139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde22d9d640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 0x7fde24118c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.781+0000 7fde2259c640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde2410a9e0 0x7fde2411c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 -- 172.21.15.29:0/3694803139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde2410a9e0 msgr2=0x7fde2411c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde2410a9e0 0x7fde2411c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 -- 172.21.15.29:0/3694803139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 msgr2=0x7fde24118c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 0x7fde24118c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 -- 172.21.15.29:0/3694803139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde24119420 con 0x7fde24111f40 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.782+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e9f0 tx=0x7fde1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.783+0000 7fde137fe640 1 -- 172.21.15.29:0/3694803139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800ccc0 con 0x7fde24111f40 2024-09-21T01:52:41.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.783+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241ace30 con 0x7fde24111f40 2024-09-21T01:52:41.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.783+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241ad300 con 0x7fde24111f40 2024-09-21T01:52:41.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.783+0000 7fde137fe640 1 -- 172.21.15.29:0/3694803139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde180044d0 con 0x7fde24111f40 2024-09-21T01:52:41.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.784+0000 7fde137fe640 1 -- 172.21.15.29:0/3694803139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18021430 con 0x7fde24111f40 2024-09-21T01:52:41.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 Processor -- start 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 -- start start 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde137fe640 1 -- 172.21.15.29:0/3694803139 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fde180215d0 con 0x7fde24111f40 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb989b390 0x561bb9c64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561bb9bd9ea0 con 0x561bb98931b0 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561bb9bc92d0 con 0x561bb989b390 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.787+0000 7fde2f416740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561bb9bcd0e0 con 0x561bb98bc070 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde22d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57048/0 (socket says 172.21.15.29:57048) 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 -- 172.21.15.29:0/3751025295 learned_addr learned my addr 172.21.15.29:0/3751025295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb989b390 0x561bb9c64120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 -- 172.21.15.29:0/3751025295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 msgr2=0x561bb9c636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 -- 172.21.15.29:0/3751025295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb989b390 msgr2=0x561bb9c64120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb989b390 0x561bb9c64120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.788+0000 7fde2259c640 1 -- 172.21.15.29:0/3751025295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb9927650 con 0x561bb98931b0 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.789+0000 7fde2259c640 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fde1403b9a0 tx=0x7fde140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9653ff256fda5715 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.789+0000 7fde137fe640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 0x7fde0c0b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.789+0000 7fde137fe640 1 -- 172.21.15.29:0/3694803139 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde18003d00 con 0x7fde24111f40 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.790+0000 7fde10ff9640 1 -- 172.21.15.29:0/3751025295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14046070 con 0x561bb98931b0 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.790+0000 7fde10ff9640 1 -- 172.21.15.29:0/3751025295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14004970 con 0x561bb98931b0 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.790+0000 7fde10ff9640 1 -- 172.21.15.29:0/3751025295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140054d0 con 0x561bb98931b0 2024-09-21T01:52:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.791+0000 7fde2f416740 1 -- 172.21.15.29:0/3751025295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 msgr2=0x561bb9c63be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.791+0000 7fde2f416740 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fde1403b9a0 tx=0x7fde140046c0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.791+0000 7fde22d9d640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 0x7fde0c0b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 -- 172.21.15.29:0/3751025295 shutdown_connections 2024-09-21T01:52:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb989b390 0x561bb9c64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb98931b0 0x561bb9c63be0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 --2- 172.21.15.29:0/3751025295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 -- 172.21.15.29:0/3751025295 >> 172.21.15.29:0/3751025295 conn(0x561bb98b0460 msgr2=0x561bb9c61590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 -- 172.21.15.29:0/3751025295 shutdown_connections 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde22d9d640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 0x7fde0c0b8c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde2411cb70 tx=0x7fde1c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.792+0000 7fde2f416740 1 -- 172.21.15.29:0/3751025295 wait complete. 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 -- start start 2024-09-21T01:52:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 Processor -- start 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c6f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561bb9a2a700 con 0x561bb989b390 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561bb9b64160 con 0x561bb98931b0 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.793+0000 7fde2f416740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561bb9c71e20 con 0x561bb98bc070 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2f416740 10 client.0 _ll_register_callbacks cb 0x561bb9b7d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde22d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c6f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35950/0 (socket says 172.21.15.29:35950) 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 -- 172.21.15.29:0/3756481777 learned_addr learned my addr 172.21.15.29:0/3756481777 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 -- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 msgr2=0x561bb9c6f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c6f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 -- 172.21.15.29:0/3756481777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 msgr2=0x561bb9c6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.794+0000 7fde2259c640 1 -- 172.21.15.29:0/3756481777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb9b91c90 con 0x561bb98931b0 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.795+0000 7fde2259c640 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde14000c00 tx=0x7fde14004b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70a091aab928dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.796+0000 7fde09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.796+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14046070 con 0x561bb98931b0 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.796+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde14005250 con 0x561bb98931b0 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.796+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde14004970 con 0x561bb98931b0 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.800+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde14005030 con 0x561bb98931b0 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.800+0000 7fde2f416740 10 client.26006 fetch_fsmap learned FSMap version 37 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.800+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bb9c7da80 con 0x561bb98931b0 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14004970 con 0x561bb98931b0 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fde1404b040 con 0x561bb98931b0 2024-09-21T01:52:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 10 client.26006 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 10 client.26006 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 20 client.26006 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb987a9f0 0x561bb9c80de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 4 client.26006 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bb9bdbdf0 con 0x561bb987a9f0 2024-09-21T01:52:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.801+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb987a9f0 0x561bb9c80de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.803+0000 7fde21d9b640 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb987a9f0 0x561bb9c80de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c276ff189af29e8b server_cookie=b1f4696182961e69 in_seq=0 out_seq=0 2024-09-21T01:52:41.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.803+0000 7fde09ffb640 10 client.26006 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:52:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.804+0000 7fde09ffb640 1 -- 172.21.15.29:0/3756481777 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bb9bdbdf0 con 0x561bb987a9f0 2024-09-21T01:52:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.804+0000 7fde09ffb640 10 client.26006 handle_command_reply: tid=0 2024-09-21T01:52:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.804+0000 7fde2f416740 1 client.26006 shutdown 2024-09-21T01:52:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.804+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 msgr2=0x561bb9c6ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.804+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde14000c00 tx=0x7fde14004b20 comp rx=0 tx=0).stop 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 shutdown_connections 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb987a9f0 0x561bb9c80de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98bc070 0x561bb9c6f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde09ffb640 0 client.26006 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 --2- 172.21.15.29:0/3756481777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6ec80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.806+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 >> 172.21.15.29:0/3756481777 conn(0x561bb98b0460 msgr2=0x561bb9c66830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.807+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 shutdown_connections 2024-09-21T01:52:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.807+0000 7fde2f416740 1 -- 172.21.15.29:0/3756481777 wait complete. 2024-09-21T01:52:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.807+0000 7fde2f416740 20 client.26006 trim_cache size 0 max 16384 2024-09-21T01:52:41.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 Processor -- start 2024-09-21T01:52:41.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 -- start start 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb987a9f0 0x561bb9c7b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400e4e0 con 0x561bb989b390 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde04010140 con 0x561bb98931b0 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.826+0000 7fde2f416740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400ebb0 con 0x561bb987a9f0 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde22d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36750/0 (socket says 172.21.15.29:36750) 2024-09-21T01:52:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 -- 172.21.15.29:0/301976378 learned_addr learned my addr 172.21.15.29:0/301976378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde2259c640 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb987a9f0 0x561bb9c7b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 -- 172.21.15.29:0/301976378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb987a9f0 msgr2=0x561bb9c7b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb987a9f0 0x561bb9c7b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 -- 172.21.15.29:0/301976378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 msgr2=0x561bb9c6c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 -- 172.21.15.29:0/301976378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb9c6ff40 con 0x561bb98931b0 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.827+0000 7fde21d9b640 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c6a0 tx=0x7fde18000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37094401a67dfae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.828+0000 7fde0bfff640 1 -- 172.21.15.29:0/301976378 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180acd50 con 0x561bb98931b0 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.828+0000 7fde0bfff640 1 -- 172.21.15.29:0/301976378 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde180ad0f0 con 0x561bb98931b0 2024-09-21T01:52:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.828+0000 7fde0bfff640 1 -- 172.21.15.29:0/301976378 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18021430 con 0x561bb98931b0 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 -- 172.21.15.29:0/301976378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 msgr2=0x561bb9c6be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c6a0 tx=0x7fde18000bc0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 -- 172.21.15.29:0/301976378 shutdown_connections 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9c6c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb98931b0 0x561bb9c6be70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 --2- 172.21.15.29:0/301976378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb987a9f0 0x561bb9c7b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 -- 172.21.15.29:0/301976378 >> 172.21.15.29:0/301976378 conn(0x561bb98b0460 msgr2=0x561bb9c65090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 -- 172.21.15.29:0/301976378 shutdown_connections 2024-09-21T01:52:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.829+0000 7fde2f416740 1 -- 172.21.15.29:0/301976378 wait complete. 2024-09-21T01:52:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.830+0000 7fde2f416740 1 -- start start 2024-09-21T01:52:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.830+0000 7fde2f416740 1 Processor -- start 2024-09-21T01:52:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98931b0 0x561bb9c826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9cea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400fa40 con 0x561bb989b390 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400f340 con 0x561bb987a9f0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400c4c0 con 0x561bb98931b0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36756/0 (socket says 172.21.15.29:36756) 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 -- 172.21.15.29:0/1751282874 learned_addr learned my addr 172.21.15.29:0/1751282874 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2f416740 10 client.0 _ll_register_callbacks cb 0x7fde1c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde21d9b640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98931b0 0x561bb9c826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde22d9d640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9cea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 -- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98931b0 msgr2=0x561bb9c826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98931b0 0x561bb9c826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 -- 172.21.15.29:0/1751282874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 msgr2=0x561bb9cea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9cea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde2259c640 1 -- 172.21.15.29:0/1751282874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb9ceaaa0 con 0x561bb987a9f0 2024-09-21T01:52:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.831+0000 7fde22d9d640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9cea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.832+0000 7fde2259c640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde1401a860 tx=0x7fde1401a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4689066a820f75f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.832+0000 7fde09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1406e060 con 0x561bb987a9f0 2024-09-21T01:52:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde140622c0 con 0x561bb987a9f0 2024-09-21T01:52:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde180ad0f0 con 0x561bb987a9f0 2024-09-21T01:52:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde14005030 con 0x561bb987a9f0 2024-09-21T01:52:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde2f416740 10 client.26012 fetch_fsmap learned FSMap version 37 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.833+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bb9cebb60 con 0x561bb987a9f0 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1405b030 con 0x561bb987a9f0 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fde140040d0 con 0x561bb987a9f0 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 10 client.26012 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 10 client.26012 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 20 client.26012 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb98bc070 0x561bb9cee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 4 client.26012 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.834+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fde180ad0f0 con 0x561bb98bc070 2024-09-21T01:52:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.835+0000 7fde21d9b640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb98bc070 0x561bb9cee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.835+0000 7fde21d9b640 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb98bc070 0x561bb9cee2f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d530908b7a0f9ce server_cookie=fd02b3cc04cd32a1 in_seq=0 out_seq=0 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.835+0000 7fde09ffb640 10 client.26012 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde09ffb640 1 -- 172.21.15.29:0/1751282874 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fde180ad0f0 con 0x561bb98bc070 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde09ffb640 10 client.26012 handle_command_reply: tid=0 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 client.26012 shutdown 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 msgr2=0x561bb9c82180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde1401a860 tx=0x7fde1401a570 comp rx=0 tx=0).stop 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 shutdown_connections 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561bb989b390 0x561bb9cea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561bb98bc070 0x561bb9cee2f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb98931b0 0x561bb9c826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde09ffb640 0 client.26012 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 --2- 172.21.15.29:0/1751282874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561bb987a9f0 0x561bb9c82180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 >> 172.21.15.29:0/1751282874 conn(0x561bb98b0460 msgr2=0x561bb9c70ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 shutdown_connections 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.836+0000 7fde2f416740 1 -- 172.21.15.29:0/1751282874 wait complete. 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.837+0000 7fde2f416740 20 client.26012 trim_cache size 0 max 16384 2024-09-21T01:52:41.840 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.838+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 msgr2=0x7fde0c0b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.838+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 0x7fde0c0b8c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde2411cb70 tx=0x7fde1c009500 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.838+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 msgr2=0x7fde241186c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.838+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e9f0 tx=0x7fde1800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 shutdown_connections 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde0c0b6750 0x7fde0c0b8c60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2411cc30 0x7fde24118c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde24111f40 0x7fde241186c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 --2- 172.21.15.29:0/3694803139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde2410a9e0 0x7fde2411c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 >> 172.21.15.29:0/3694803139 conn(0x7fde24070ec0 msgr2=0x7fde2410ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 shutdown_connections 2024-09-21T01:52:41.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:41.839+0000 7fde28ea2640 1 -- 172.21.15.29:0/3694803139 wait complete. 2024-09-21T01:52:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:41 smithi066 ceph-mon[35001]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 869 op/s 2024-09-21T01:52:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:41 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:41 smithi029 ceph-mon[31228]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 869 op/s 2024-09-21T01:52:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:41 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:41 smithi190 ceph-mon[34368]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 869 op/s 2024-09-21T01:52:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:41 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:42.758 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:52:42.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 245.0 s, 3276.4 tps, lat 30.675 ms stddev 39.707 2024-09-21T01:52:43.180 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:43 smithi066 ceph-mon[35001]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 742 op/s 2024-09-21T01:52:44.053 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:43 smithi029 ceph-mon[31228]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 742 op/s 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 -- 172.21.15.29:0/978195579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 msgr2=0x7ff500102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 --2- 172.21.15.29:0/978195579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff500102bb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009930 tx=0x7ff4fc02a480 comp rx=0 tx=0).stop 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 -- 172.21.15.29:0/978195579 shutdown_connections 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 --2- 172.21.15.29:0/978195579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff500107be0 0x7ff5001080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 --2- 172.21.15.29:0/978195579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5001030f0 0x7ff500107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 --2- 172.21.15.29:0/978195579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff500102bb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.195+0000 7ff507490640 1 -- 172.21.15.29:0/978195579 >> 172.21.15.29:0/978195579 conn(0x7ff500079730 msgr2=0x7ff500079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.196+0000 7ff507490640 1 -- 172.21.15.29:0/978195579 shutdown_connections 2024-09-21T01:52:44.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.196+0000 7ff507490640 1 -- 172.21.15.29:0/978195579 wait complete. 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.197+0000 7ff507490640 1 Processor -- start 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.197+0000 7ff507490640 1 -- start start 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 0x7ff5001a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 0x7ff5001a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff50011a1b0 con 0x7ff5001030f0 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff50011a030 con 0x7ff5001027a0 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff507490640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff50011a330 con 0x7ff500107be0 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff50522c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff50522c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36784/0 (socket says 172.21.15.29:36784) 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff50522c640 1 -- 172.21.15.29:0/1588837469 learned_addr learned my addr 172.21.15.29:0/1588837469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:44.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 0x7ff5001a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.198+0000 7ff504a2b640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 0x7ff5001a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 -- 172.21.15.29:0/1588837469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 msgr2=0x7ff5001a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 0x7ff5001a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 -- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 msgr2=0x7ff5001a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 0x7ff5001a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 -- 172.21.15.29:0/1588837469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5001a6b50 con 0x7ff5001027a0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 0x7ff5001a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff504a2b640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 0x7ff5001a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff50522c640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a9a0 tx=0x7ff4fc003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff4f67fc640 1 -- 172.21.15.29:0/1588837469 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc03f070 con 0x7ff5001027a0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff4f67fc640 1 -- 172.21.15.29:0/1588837469 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc03bd20 con 0x7ff5001027a0 2024-09-21T01:52:44.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff4f67fc640 1 -- 172.21.15.29:0/1588837469 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc035030 con 0x7ff5001027a0 2024-09-21T01:52:44.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.199+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5001a6de0 con 0x7ff5001027a0 2024-09-21T01:52:44.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.200+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5001a7230 con 0x7ff5001027a0 2024-09-21T01:52:44.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.204+0000 7ff4f67fc640 1 -- 172.21.15.29:0/1588837469 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff4fc042070 con 0x7ff5001027a0 2024-09-21T01:52:44.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.204+0000 7ff50da04740 1 Processor -- start 2024-09-21T01:52:44.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.204+0000 7ff50da04740 1 -- start start 2024-09-21T01:52:44.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabb6390 0x5574aaf7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5574aaef4ea0 con 0x5574aabb6390 2024-09-21T01:52:44.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5574aaee42d0 con 0x5574aabd7070 2024-09-21T01:52:44.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50da04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574aaee80e0 con 0x5574aabae1b0 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50522c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50522c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51440/0 (socket says 172.21.15.29:51440) 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff50522c640 1 -- 172.21.15.29:0/3382805591 learned_addr learned my addr 172.21.15.29:0/3382805591 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.205+0000 7ff504a2b640 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabb6390 0x5574aaf7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 -- 172.21.15.29:0/3382805591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 msgr2=0x5574aaf7e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 -- 172.21.15.29:0/3382805591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabb6390 msgr2=0x5574aaf7ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabb6390 0x5574aaf7ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 -- 172.21.15.29:0/3382805591 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aac42650 con 0x5574aabae1b0 2024-09-21T01:52:44.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.206+0000 7ff50522c640 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc10bda0 tx=0x7ff4fc10be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b14af4149de0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.207+0000 7ff4f67fc640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 0x7ff4ec0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.207+0000 7ff4eb7fe640 1 -- 172.21.15.29:0/3382805591 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc117070 con 0x5574aabae1b0 2024-09-21T01:52:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.207+0000 7ff4eb7fe640 1 -- 172.21.15.29:0/3382805591 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc110bf0 con 0x5574aabae1b0 2024-09-21T01:52:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.208+0000 7ff4eb7fe640 1 -- 172.21.15.29:0/3382805591 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc112630 con 0x5574aabae1b0 2024-09-21T01:52:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.208+0000 7ff4f67fc640 1 -- 172.21.15.29:0/1588837469 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff4fc107c70 con 0x7ff5001027a0 2024-09-21T01:52:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.208+0000 7ff50da04740 1 -- 172.21.15.29:0/3382805591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 msgr2=0x5574aaf7e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.208+0000 7ff50da04740 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc10bda0 tx=0x7ff4fc10be90 comp rx=0 tx=0).stop 2024-09-21T01:52:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 0x7ff4ec0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff50da04740 1 -- 172.21.15.29:0/3382805591 shutdown_connections 2024-09-21T01:52:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff50da04740 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabb6390 0x5574aaf7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff50da04740 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabae1b0 0x5574aaf7e800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff50da04740 1 --2- 172.21.15.29:0/3382805591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.209+0000 7ff50da04740 1 -- 172.21.15.29:0/3382805591 >> 172.21.15.29:0/3382805591 conn(0x5574aabcb460 msgr2=0x5574aaf7c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.210+0000 7ff50da04740 1 -- 172.21.15.29:0/3382805591 shutdown_connections 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.210+0000 7ff50da04740 1 -- 172.21.15.29:0/3382805591 wait complete. 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.210+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 0x7ff4ec0b8b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009e80 tx=0x7ff4f8009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.210+0000 7ff50da04740 1 -- start start 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.210+0000 7ff50da04740 1 Processor -- start 2024-09-21T01:52:44.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aaf898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabb6390 0x5574aaf89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5574aad45700 con 0x5574aabae1b0 2024-09-21T01:52:44.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5574aae7f160 con 0x5574aabb6390 2024-09-21T01:52:44.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.211+0000 7ff50da04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574aaf9d870 con 0x5574aabd7070 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51450/0 (socket says 172.21.15.29:51450) 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 -- 172.21.15.29:0/2623103053 learned_addr learned my addr 172.21.15.29:0/2623103053 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff504a2b640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabb6390 0x5574aaf89e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff50da04740 10 client.0 _ll_register_callbacks cb 0x5574aae989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 -- 172.21.15.29:0/2623103053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabb6390 msgr2=0x5574aaf89e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabb6390 0x5574aaf89e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 -- 172.21.15.29:0/2623103053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 msgr2=0x5574aaf898e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aaf898e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff505a2d640 1 -- 172.21.15.29:0/2623103053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aaeacc90 con 0x5574aabd7070 2024-09-21T01:52:44.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.212+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc110bf0 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff505a2d640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8018c10 tx=0x7ff4f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b74162ad710525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff4e8ff9640 10 client.26077 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/2623103053 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8028060 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/2623103053 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff4f8012030 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/2623103053 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff4f8018dc0 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff50da04740 10 client.26077 fetch_fsmap learned FSMap version 37 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.213+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5574aafa9560 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/2623103053 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7ff4f8016820 con 0x5574aabd7070 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 10 client.26077 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T01:52:44.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 10 client.26077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:52:44.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 20 client.26077 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:44.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab959f0 0x5574aafac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 4 client.26077 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:44.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574aaef6df0 con 0x5574aab959f0 2024-09-21T01:52:44.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.214+0000 7ff50522c640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab959f0 0x5574aafac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.215+0000 7ff50522c640 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab959f0 0x5574aafac970 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e546105aa0a2c6f9 server_cookie=b9c6ef44fff444b2 in_seq=0 out_seq=0 2024-09-21T01:52:44.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.215+0000 7ff4e8ff9640 10 client.26077 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:52:44.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/2623103053 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574aaef6df0 con 0x5574aab959f0 2024-09-21T01:52:44.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff4e8ff9640 10 client.26077 handle_command_reply: tid=0 2024-09-21T01:52:44.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 client.26077 shutdown 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 msgr2=0x5574aaf8a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8018c10 tx=0x7ff4f8004a40 comp rx=0 tx=0).stop 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 shutdown_connections 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab959f0 0x5574aafac970 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabd7070 0x5574aaf8a380 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabb6390 0x5574aaf89e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 --2- 172.21.15.29:0/2623103053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aaf898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 >> 172.21.15.29:0/2623103053 conn(0x5574aabcb460 msgr2=0x5574aaf81450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff4e8ff9640 0 client.26077 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 shutdown_connections 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.216+0000 7ff50da04740 1 -- 172.21.15.29:0/2623103053 wait complete. 2024-09-21T01:52:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.217+0000 7ff50da04740 20 client.26077 trim_cache size 0 max 16384 2024-09-21T01:52:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:43 smithi190 ceph-mon[34368]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 742 op/s 2024-09-21T01:52:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.238+0000 7ff50da04740 1 Processor -- start 2024-09-21T01:52:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 -- start start 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aaf7f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 0x5574aaf7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001121b0 con 0x5574aabd7070 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001aeef0 con 0x5574aabae1b0 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50da04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001ad960 con 0x5574aabb6390 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50522c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50522c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36818/0 (socket says 172.21.15.29:36818) 2024-09-21T01:52:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.239+0000 7ff50522c640 1 -- 172.21.15.29:0/383885235 learned_addr learned my addr 172.21.15.29:0/383885235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff505a2d640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 0x5574aaf7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 -- 172.21.15.29:0/383885235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 msgr2=0x5574aaf7f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff504a2b640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aaf7f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aaf7f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 -- 172.21.15.29:0/383885235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 msgr2=0x5574aaf7fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 0x5574aaf7fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 -- 172.21.15.29:0/383885235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aaf8a980 con 0x5574aabae1b0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff505a2d640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 0x5574aaf7fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff504a2b640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aaf7f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff50522c640 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc1106c0 tx=0x7ff4fc113ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2225bae53005c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff4eaffd640 1 -- 172.21.15.29:0/383885235 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc049070 con 0x5574aabae1b0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.240+0000 7ff4eaffd640 1 -- 172.21.15.29:0/383885235 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc0cd8c0 con 0x5574aabae1b0 2024-09-21T01:52:44.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.241+0000 7ff4eaffd640 1 -- 172.21.15.29:0/383885235 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc117040 con 0x5574aabae1b0 2024-09-21T01:52:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 -- 172.21.15.29:0/383885235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 msgr2=0x5574aaf8b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc1106c0 tx=0x7ff4fc113ab0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 -- 172.21.15.29:0/383885235 shutdown_connections 2024-09-21T01:52:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabd7070 0x5574aaf7fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aaf7f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 --2- 172.21.15.29:0/383885235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabae1b0 0x5574aaf8b7c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 -- 172.21.15.29:0/383885235 >> 172.21.15.29:0/383885235 conn(0x5574aabcb460 msgr2=0x5574aaf7f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 -- 172.21.15.29:0/383885235 shutdown_connections 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.242+0000 7ff50da04740 1 -- 172.21.15.29:0/383885235 wait complete. 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.243+0000 7ff50da04740 1 -- start start 2024-09-21T01:52:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.243+0000 7ff50da04740 1 Processor -- start 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aafa2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aafa35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001ae7f0 con 0x5574aabae1b0 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001ae0f0 con 0x5574aabd7070 2024-09-21T01:52:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a1d10 con 0x5574aabb6390 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50da04740 10 client.0 _ll_register_callbacks cb 0x7ff4f800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36832/0 (socket says 172.21.15.29:36832) 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 -- 172.21.15.29:0/1836574968 learned_addr learned my addr 172.21.15.29:0/1836574968 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff504a2b640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aafa35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff50522c640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aafa2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 -- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 msgr2=0x5574aafa35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aafa35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 -- 172.21.15.29:0/1836574968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 msgr2=0x5574aafa2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aafa2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.244+0000 7ff505a2d640 1 -- 172.21.15.29:0/1836574968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aaf9ee40 con 0x5574aabd7070 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff505a2d640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800a010 tx=0x7ff4f8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4fa4e7fa6c3908 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff4e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:44.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f803c060 con 0x5574aabd7070 2024-09-21T01:52:44.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff4f80236f0 con 0x5574aabd7070 2024-09-21T01:52:44.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0cd8c0 con 0x5574aabd7070 2024-09-21T01:52:44.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.245+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff4f8028040 con 0x5574aabd7070 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.246+0000 7ff50da04740 10 client.26027 fetch_fsmap learned FSMap version 37 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.246+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5574ab016df0 con 0x5574aabd7070 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc0cd8c0 con 0x5574aabd7070 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7ff4f802d950 con 0x5574aabd7070 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 10 client.26027 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 10 client.26027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:52:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 20 client.26027 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab6ada0 0x5574ab019640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 4 client.26027 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574aaef28a0 con 0x5574aab6ada0 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.247+0000 7ff50522c640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab6ada0 0x5574ab019640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff4e8ff9640 10 client.26027 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50522c640 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab6ada0 0x5574ab019640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b8004a2f5611dfa server_cookie=9bf4a99c47868841 in_seq=0 out_seq=0 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff4e8ff9640 1 -- 172.21.15.29:0/1836574968 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574aaef28a0 con 0x5574aab6ada0 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff4e8ff9640 10 client.26027 handle_command_reply: tid=0 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 client.26027 shutdown 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 msgr2=0x5574aaf9e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800a010 tx=0x7ff4f8013000 comp rx=0 tx=0).stop 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 shutdown_connections 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5574aab6ada0 0x5574ab019640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5574aabd7070 0x5574aaf9e6b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574aabb6390 0x5574aafa35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 --2- 172.21.15.29:0/1836574968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574aabae1b0 0x5574aafa2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.248+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 >> 172.21.15.29:0/1836574968 conn(0x5574aabcb460 msgr2=0x5574aaf7fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.249+0000 7ff4e8ff9640 0 client.26027 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.249+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 shutdown_connections 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.249+0000 7ff50da04740 1 -- 172.21.15.29:0/1836574968 wait complete. 2024-09-21T01:52:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.249+0000 7ff50da04740 20 client.26027 trim_cache size 0 max 16384 2024-09-21T01:52:44.253 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:52:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.250+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 msgr2=0x7ff4ec0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.250+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 0x7ff4ec0b8b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009e80 tx=0x7ff4f8009510 comp rx=0 tx=0).stop 2024-09-21T01:52:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.250+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 msgr2=0x7ff5001a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.250+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a9a0 tx=0x7ff4fc003d40 comp rx=0 tx=0).stop 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 shutdown_connections 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4ec0b6660 0x7ff4ec0b8b70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff500107be0 0x7ff5001a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5001030f0 0x7ff5001a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 --2- 172.21.15.29:0/1588837469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5001027a0 0x7ff5001a0be0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 >> 172.21.15.29:0/1588837469 conn(0x7ff500079730 msgr2=0x7ff5001055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 shutdown_connections 2024-09-21T01:52:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:44.251+0000 7ff507490640 1 -- 172.21.15.29:0/1588837469 wait complete. 2024-09-21T01:52:45.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:52:45.623 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:46.071 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:45 smithi029 ceph-mon[31228]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-21T01:52:46.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 -- 172.21.15.29:0/1516890203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9941083a0 msgr2=0x7fc9941087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 --2- 172.21.15.29:0/1516890203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9941083a0 0x7fc9941087b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc988009900 tx=0x7fc98802a410 comp rx=0 tx=0).stop 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 -- 172.21.15.29:0/1516890203 shutdown_connections 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 --2- 172.21.15.29:0/1516890203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994108cf0 0x7fc99410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 --2- 172.21.15.29:0/1516890203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9941083a0 0x7fc9941087b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 --2- 172.21.15.29:0/1516890203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc994102a60 0x7fc994102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 -- 172.21.15.29:0/1516890203 >> 172.21.15.29:0/1516890203 conn(0x7fc994079730 msgr2=0x7fc994079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 -- 172.21.15.29:0/1516890203 shutdown_connections 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.209+0000 7fc999a10640 1 -- 172.21.15.29:0/1516890203 wait complete. 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.210+0000 7fc999a10640 1 Processor -- start 2024-09-21T01:52:46.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.210+0000 7fc999a10640 1 -- start start 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.210+0000 7fc999a10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994102a60 0x7fc99419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.210+0000 7fc999a10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc999a10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc994108cf0 0x7fc9941a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc999a10640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc994112960 con 0x7fc994102a60 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc999a10640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941127e0 con 0x7fc9941083a0 2024-09-21T01:52:46.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc999a10640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc994112ae0 con 0x7fc994108cf0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9937fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc994108cf0 0x7fc9941a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36846/0 (socket says 172.21.15.29:36846) 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 -- 172.21.15.29:0/3397796281 learned_addr learned my addr 172.21.15.29:0/3397796281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc992ffd640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994102a60 0x7fc99419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 -- 172.21.15.29:0/3397796281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc994108cf0 msgr2=0x7fc9941a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc994108cf0 0x7fc9941a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 -- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994102a60 msgr2=0x7fc99419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994102a60 0x7fc99419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 -- 172.21.15.29:0/3397796281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941a2680 con 0x7fc9941083a0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.211+0000 7fc9927fc640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc988032fd0 tx=0x7fc988031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.212+0000 7fc998a0e640 1 -- 172.21.15.29:0/3397796281 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98803f030 con 0x7fc9941083a0 2024-09-21T01:52:46.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.212+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941a2910 con 0x7fc9941083a0 2024-09-21T01:52:46.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.212+0000 7fc998a0e640 1 -- 172.21.15.29:0/3397796281 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc988004490 con 0x7fc9941083a0 2024-09-21T01:52:46.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.212+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941093b0 con 0x7fc9941083a0 2024-09-21T01:52:46.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.212+0000 7fc998a0e640 1 -- 172.21.15.29:0/3397796281 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988035070 con 0x7fc9941083a0 2024-09-21T01:52:46.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.215+0000 7fc99ff84740 1 Processor -- start 2024-09-21T01:52:46.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 -- start start 2024-09-21T01:52:46.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409ceb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409cec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f409c61ea0 con 0x55f409923390 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f409c512d0 con 0x55f409944070 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc99ff84740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f409c550e0 con 0x55f40991b1b0 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc9937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409ceb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc9937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409ceb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36852/0 (socket says 172.21.15.29:36852) 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc9937fe640 1 -- 172.21.15.29:0/1454881832 learned_addr learned my addr 172.21.15.29:0/1454881832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.216+0000 7fc992ffd640 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 -- 172.21.15.29:0/1454881832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 msgr2=0x55f409ceb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409ceb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 -- 172.21.15.29:0/1454881832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 msgr2=0x55f409cec120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:52:46.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409cec120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 -- 172.21.15.29:0/1454881832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4099af650 con 0x55f40991b1b0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.217+0000 7fc992ffd640 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cebbe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc984009c00 tx=0x7fc984011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb400c1486c5010 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.218+0000 7fc998a0e640 1 -- 172.21.15.29:0/3397796281 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc988042030 con 0x7fc9941083a0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.218+0000 7fc979ffb640 1 -- 172.21.15.29:0/1454881832 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004230 con 0x55f40991b1b0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.218+0000 7fc979ffb640 1 -- 172.21.15.29:0/1454881832 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9840043d0 con 0x55f40991b1b0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.218+0000 7fc979ffb640 1 -- 172.21.15.29:0/1454881832 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004cc0 con 0x55f40991b1b0 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc998a0e640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 0x7fc97c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 -- 172.21.15.29:0/1454881832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 msgr2=0x55f409cebbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cebbe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc984009c00 tx=0x7fc984011810 comp rx=0 tx=0).stop 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc9937fe640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 0x7fc97c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc998a0e640 1 -- 172.21.15.29:0/3397796281 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc988107500 con 0x7fc9941083a0 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 -- 172.21.15.29:0/1454881832 shutdown_connections 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409cec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cebbe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 --2- 172.21.15.29:0/1454881832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409ceb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 -- 172.21.15.29:0/1454881832 >> 172.21.15.29:0/1454881832 conn(0x55f409938460 msgr2=0x55f409ce9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 -- 172.21.15.29:0/1454881832 shutdown_connections 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.219+0000 7fc99ff84740 1 -- 172.21.15.29:0/1454881832 wait complete. 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc9937fe640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 0x7fc97c0b8aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005fd0 tx=0x7fc98c018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 -- start start 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 Processor -- start 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 0x55f409cf6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 0x55f409cf71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f409ab2700 con 0x55f40991b1b0 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f409bec160 con 0x55f409944070 2024-09-21T01:52:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.220+0000 7fc99ff84740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f409d0af30 con 0x55f409923390 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc99ff84740 10 client.0 _ll_register_callbacks cb 0x55f409c059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9927fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 0x55f409cf71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 0x55f409cf6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36854/0 (socket says 172.21.15.29:36854) 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9927fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 0x55f409cf71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51522/0 (socket says 172.21.15.29:51522) 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9937fe640 1 -- 172.21.15.29:0/2340715027 learned_addr learned my addr 172.21.15.29:0/2340715027 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 -- 172.21.15.29:0/2340715027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 msgr2=0x55f409cf71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 0x55f409cf71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 -- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 msgr2=0x55f409cf7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc992ffd640 1 -- 172.21.15.29:0/2340715027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f409c19c90 con 0x55f40991b1b0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.221+0000 7fc9937fe640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 0x55f409cf6c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc984004920 tx=0x7fc984019bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d320ccfcd0642130 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984028020 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9840043d0 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc984024620 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc984024820 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc99ff84740 10 client.16311 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.222+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f409d16b90 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9840043d0 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fc984018070 con 0x55f40991b1b0 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 10 client.16311 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:52:46.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 20 client.16311 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4099029f0 0x55f409d1a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 4 client.16311 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f409c63df0 con 0x55f4099029f0 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.223+0000 7fc9927fc640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4099029f0 0x55f409d1a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.224+0000 7fc972ffd640 10 client.16311 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.224+0000 7fc9927fc640 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4099029f0 0x55f409d1a430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e32f81a67b1768cd server_cookie=3ed1e4d290c4e1bb in_seq=0 out_seq=0 2024-09-21T01:52:46.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.225+0000 7fc972ffd640 1 -- 172.21.15.29:0/2340715027 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f409c63df0 con 0x55f4099029f0 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.225+0000 7fc972ffd640 10 client.16311 handle_command_reply: tid=0 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.225+0000 7fc99ff84740 1 client.16311 shutdown 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 msgr2=0x55f409cf6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 0x55f409cf6c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc984004920 tx=0x7fc984019bb0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 shutdown_connections 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4099029f0 0x55f409d1a430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409cf7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f409923390 0x55f409cf71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2340715027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f40991b1b0 0x55f409cf6c80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 >> 172.21.15.29:0/2340715027 conn(0x55f409938460 msgr2=0x55f409ced090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.226+0000 7fc972ffd640 0 client.16311 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:52:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:45 smithi190 ceph-mon[34368]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-21T01:52:46.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.228+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 shutdown_connections 2024-09-21T01:52:46.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.228+0000 7fc99ff84740 1 -- 172.21.15.29:0/2340715027 wait complete. 2024-09-21T01:52:46.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.228+0000 7fc99ff84740 20 client.16311 trim_cache size 0 max 16384 2024-09-21T01:52:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.247+0000 7fc99ff84740 1 Processor -- start 2024-09-21T01:52:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 -- start start 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409923390 0x55f409cf8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409944070 0x55f409cecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc98000e420 con 0x55f409944070 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc980010050 con 0x55f409923390 2024-09-21T01:52:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.248+0000 7fc99ff84740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc98000eaf0 con 0x55f40991b1b0 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51526/0 (socket says 172.21.15.29:51526) 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 -- 172.21.15.29:0/3981120393 learned_addr learned my addr 172.21.15.29:0/3981120393 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc9927fc640 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409923390 0x55f409cf8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc9937fe640 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409944070 0x55f409cecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 -- 172.21.15.29:0/3981120393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409923390 msgr2=0x55f409cf8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409923390 0x55f409cf8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 -- 172.21.15.29:0/3981120393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409944070 msgr2=0x55f409cecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409944070 0x55f409cecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 -- 172.21.15.29:0/3981120393 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f409cedd90 con 0x55f40991b1b0 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.249+0000 7fc992ffd640 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9840057c0 tx=0x7fc984012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947a58cb86d0312b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.250+0000 7fc9797fa640 1 -- 172.21.15.29:0/3981120393 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984028070 con 0x55f40991b1b0 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.250+0000 7fc9797fa640 1 -- 172.21.15.29:0/3981120393 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98401f040 con 0x55f40991b1b0 2024-09-21T01:52:46.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.250+0000 7fc9797fa640 1 -- 172.21.15.29:0/3981120393 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984024680 con 0x55f40991b1b0 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 -- 172.21.15.29:0/3981120393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 msgr2=0x55f409cf2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9840057c0 tx=0x7fc984012f90 comp rx=0 tx=0).stop 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 -- 172.21.15.29:0/3981120393 shutdown_connections 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409944070 0x55f409cecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409923390 0x55f409cf8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.251+0000 7fc99ff84740 1 --2- 172.21.15.29:0/3981120393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409cf2430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.252+0000 7fc99ff84740 1 -- 172.21.15.29:0/3981120393 >> 172.21.15.29:0/3981120393 conn(0x55f409938460 msgr2=0x55f409cec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.252+0000 7fc99ff84740 1 -- 172.21.15.29:0/3981120393 shutdown_connections 2024-09-21T01:52:46.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.252+0000 7fc99ff84740 1 -- 172.21.15.29:0/3981120393 wait complete. 2024-09-21T01:52:46.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.252+0000 7fc99ff84740 1 -- start start 2024-09-21T01:52:46.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.253+0000 7fc99ff84740 1 Processor -- start 2024-09-21T01:52:46.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.253+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409d0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409d0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc98000f980 con 0x55f409923390 2024-09-21T01:52:46.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc98000f280 con 0x55f409944070 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc98000c400 con 0x55f40991b1b0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc99ff84740 10 client.0 _ll_register_callbacks cb 0x7fc984009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51534/0 (socket says 172.21.15.29:51534) 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 -- 172.21.15.29:0/2588017859 learned_addr learned my addr 172.21.15.29:0/2588017859 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc9937fe640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409d0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc9927fc640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409d0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 -- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 msgr2=0x55f409d0dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409d0dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 -- 172.21.15.29:0/2588017859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 msgr2=0x55f409d0d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409d0d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.254+0000 7fc992ffd640 1 -- 172.21.15.29:0/2588017859 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f409d82df0 con 0x55f40991b1b0 2024-09-21T01:52:46.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.255+0000 7fc992ffd640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc984024650 tx=0x7fc984024680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7a4686216ff9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:46.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.255+0000 7fc972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:46.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.255+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984028040 con 0x55f40991b1b0 2024-09-21T01:52:46.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.255+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc984024dd0 con 0x55f40991b1b0 2024-09-21T01:52:46.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.255+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98401f040 con 0x55f40991b1b0 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.256+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc984004fe0 con 0x55f40991b1b0 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.257+0000 7fc99ff84740 10 client.26098 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.257+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f409d83fd0 con 0x55f40991b1b0 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.257+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98401f040 con 0x55f40991b1b0 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.257+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fc984036030 con 0x55f40991b1b0 2024-09-21T01:52:46.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 10 client.26098 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 10 client.26098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 20 client.26098 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4098d7da0 0x55f409d866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 4 client.26098 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.258+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f409c5f8a0 con 0x55f4098d7da0 2024-09-21T01:52:46.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.259+0000 7fc9927fc640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4098d7da0 0x55f409d866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.259+0000 7fc9927fc640 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4098d7da0 0x55f409d866f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c11b68bed8891922 server_cookie=8acf83c8507018a in_seq=0 out_seq=0 2024-09-21T01:52:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.259+0000 7fc972ffd640 10 client.26098 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:52:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.259+0000 7fc972ffd640 1 -- 172.21.15.29:0/2588017859 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f409c5f8a0 con 0x55f4098d7da0 2024-09-21T01:52:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.259+0000 7fc972ffd640 10 client.26098 handle_command_reply: tid=0 2024-09-21T01:52:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 client.26098 shutdown 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 msgr2=0x55f409d0d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc984024650 tx=0x7fc984024680 comp rx=0 tx=0).stop 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 shutdown_connections 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f4098d7da0 0x55f409d866f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc972ffd640 0 client.26098 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f409944070 0x55f409d0dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f409923390 0x55f409d0d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 --2- 172.21.15.29:0/2588017859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f40991b1b0 0x55f409d0d160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 >> 172.21.15.29:0/2588017859 conn(0x55f409938460 msgr2=0x55f409d13b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 shutdown_connections 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 1 -- 172.21.15.29:0/2588017859 wait complete. 2024-09-21T01:52:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.260+0000 7fc99ff84740 20 client.26098 trim_cache size 0 max 16384 2024-09-21T01:52:46.264 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:52:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 msgr2=0x7fc97c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 0x7fc97c0b8aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005fd0 tx=0x7fc98c018040 comp rx=0 tx=0).stop 2024-09-21T01:52:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 msgr2=0x7fc99419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc988032fd0 tx=0x7fc988031a20 comp rx=0 tx=0).stop 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 shutdown_connections 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc97c0b6590 0x7fc97c0b8aa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc994108cf0 0x7fc9941a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9941083a0 0x7fc99419ccb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 --2- 172.21.15.29:0/3397796281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc994102a60 0x7fc99419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 >> 172.21.15.29:0/3397796281 conn(0x7fc994079730 msgr2=0x7fc99406afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 shutdown_connections 2024-09-21T01:52:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:46.261+0000 7fc999a10640 1 -- 172.21.15.29:0/3397796281 wait complete. 2024-09-21T01:52:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:45 smithi066 ceph-mon[35001]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-21T01:52:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:46 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:47.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:46 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:47.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:52:47.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:46 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:52:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 250.0 s, 2606.4 tps, lat 38.154 ms stddev 43.259 2024-09-21T01:52:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:47 smithi029 ceph-mon[31228]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 849 op/s 2024-09-21T01:52:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:47 smithi190 ceph-mon[34368]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 849 op/s 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 -- 172.21.15.29:0/3098440723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 msgr2=0x7f30f0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 --2- 172.21.15.29:0/3098440723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f0102a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009900 tx=0x7f30e002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 -- 172.21.15.29:0/3098440723 shutdown_connections 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 --2- 172.21.15.29:0/3098440723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 --2- 172.21.15.29:0/3098440723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 --2- 172.21.15.29:0/3098440723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f0102a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 -- 172.21.15.29:0/3098440723 >> 172.21.15.29:0/3098440723 conn(0x7f30f00fdee0 msgr2=0x7f30f0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 -- 172.21.15.29:0/3098440723 shutdown_connections 2024-09-21T01:52:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.254+0000 7f30f5670640 1 -- 172.21.15.29:0/3098440723 wait complete. 2024-09-21T01:52:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.255+0000 7f30f5670640 1 Processor -- start 2024-09-21T01:52:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.255+0000 7f30f5670640 1 -- start start 2024-09-21T01:52:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f01a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f01a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010d100 con 0x7f30f0103620 2024-09-21T01:52:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010cf80 con 0x7f30f0103fc0 2024-09-21T01:52:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30f5670640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010d280 con 0x7f30f01025a0 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36888/0 (socket says 172.21.15.29:36888) 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.256+0000 7f30ef7fe640 1 -- 172.21.15.29:0/2633141782 learned_addr learned my addr 172.21.15.29:0/2633141782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f01a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f01a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 -- 172.21.15.29:0/2633141782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 msgr2=0x7f30f01a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f01a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 -- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 msgr2=0x7f30f01a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f01a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 -- 172.21.15.29:0/2633141782 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f01a1fd0 con 0x7f30f0103fc0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f01a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.257+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f30e800eb40 tx=0x7f30e800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.258+0000 7f30d7fff640 1 -- 172.21.15.29:0/2633141782 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8010430 con 0x7f30f0103fc0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.258+0000 7f30d7fff640 1 -- 172.21.15.29:0/2633141782 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e8010a80 con 0x7f30f0103fc0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.258+0000 7f30d7fff640 1 -- 172.21.15.29:0/2633141782 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8018b60 con 0x7f30f0103fc0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.258+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f01a2200 con 0x7f30f0103fc0 2024-09-21T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.259+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01a8790 con 0x7f30f0103fc0 2024-09-21T01:52:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.262+0000 7f30fc5dd740 1 Processor -- start 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 -- start start 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30d7fff640 1 -- 172.21.15.29:0/2633141782 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f30e8018d00 con 0x7f30f0103fc0 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x55908876ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x55908876c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5590886dd6f0 con 0x5590883c31b0 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5590886ccb20 con 0x55908839a2f0 2024-09-21T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5590886d0930 con 0x5590883a24d0 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x55908876ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x55908876ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41390/0 (socket says 172.21.15.29:41390) 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36904/0 (socket says 172.21.15.29:36904) 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30ee7fc640 1 -- 172.21.15.29:0/2701851718 learned_addr learned my addr 172.21.15.29:0/2701851718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.264+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x55908876c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 -- 172.21.15.29:0/2701851718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 msgr2=0x55908876c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x55908876c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 -- 172.21.15.29:0/2701851718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 msgr2=0x55908876ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x55908876ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 -- 172.21.15.29:0/2701851718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559088692530 con 0x55908839a2f0 2024-09-21T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30e002a9e0 tx=0x7f30e00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1ffd7949f4d126 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30d57fa640 1 -- 172.21.15.29:0/2701851718 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0035030 con 0x55908839a2f0 2024-09-21T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30d57fa640 1 -- 172.21.15.29:0/2701851718 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e00044b0 con 0x55908839a2f0 2024-09-21T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.265+0000 7f30d57fa640 1 -- 172.21.15.29:0/2701851718 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0004e20 con 0x55908839a2f0 2024-09-21T01:52:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.269+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2701851718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 msgr2=0x55908876b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.269+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30e002a9e0 tx=0x7f30e00040c0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.269+0000 7f30d7fff640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 0x7f30d80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.269+0000 7f30d7fff640 1 -- 172.21.15.29:0/2633141782 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30e80e7900 con 0x7f30f0103fc0 2024-09-21T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.270+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2701851718 shutdown_connections 2024-09-21T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.270+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x55908876c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.270+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x55908876b4c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.270+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2701851718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x55908876ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.270+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2701851718 >> 172.21.15.29:0/2701851718 conn(0x5590883b7300 msgr2=0x5590887668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.276+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 0x7f30d80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.278+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2701851718 shutdown_connections 2024-09-21T01:52:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.279+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2701851718 wait complete. 2024-09-21T01:52:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.280+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 0x7f30d80b8bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30e4012640 tx=0x7f30e4014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.280+0000 7f30fc5dd740 1 -- start start 2024-09-21T01:52:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.280+0000 7f30fc5dd740 1 Processor -- start 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x559088774c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088775790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36918/0 (socket says 172.21.15.29:36918) 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x559088774c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30eeffd640 1 -- 172.21.15.29:0/2516652164 learned_addr learned my addr 172.21.15.29:0/2516652164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559088604ab0 con 0x5590883c31b0 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559088672730 con 0x55908839a2f0 2024-09-21T01:52:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55908876cad0 con 0x5590883a24d0 2024-09-21T01:52:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088775790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.281+0000 7f30fc5dd740 10 client.0 _ll_register_callbacks cb 0x5590886c66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 -- 172.21.15.29:0/2516652164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 msgr2=0x559088774c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x559088774c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 -- 172.21.15.29:0/2516652164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 msgr2=0x559088775790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088775790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908842e790 con 0x55908839a2f0 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088775790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30eeffd640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30e0003a80 tx=0x7f30e00049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e147a060c5eb504 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.282+0000 7f30d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.283+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0051060 con 0x55908839a2f0 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.283+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e00044b0 con 0x55908839a2f0 2024-09-21T01:52:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.283+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30e00315a0 con 0x55908839a2f0 2024-09-21T01:52:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.288+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f30e0035040 con 0x55908839a2f0 2024-09-21T01:52:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.288+0000 7f30fc5dd740 10 client.26057 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.288+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5590884a7620 con 0x55908839a2f0 2024-09-21T01:52:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e00044b0 con 0x55908839a2f0 2024-09-21T01:52:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f30e004b030 con 0x55908839a2f0 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 10 client.26057 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 20 client.26057 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559088381d80 0x559088781850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 4 client.26057 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590886df640 con 0x559088381d80 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.292+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559088381d80 0x559088781850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.294+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559088381d80 0x559088781850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958e13b2ef2974b server_cookie=b0d28c90dacd977b in_seq=0 out_seq=0 2024-09-21T01:52:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30d2ff5640 10 client.26057 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30d2ff5640 1 -- 172.21.15.29:0/2516652164 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590886df640 con 0x559088381d80 2024-09-21T01:52:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30d2ff5640 10 client.26057 handle_command_reply: tid=0 2024-09-21T01:52:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 client.26057 shutdown 2024-09-21T01:52:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:47 smithi066 ceph-mon[35001]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 849 op/s 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 msgr2=0x559088774560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30e0003a80 tx=0x7f30e00049d0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 shutdown_connections 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559088381d80 0x559088781850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088775790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883a24d0 0x559088774c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/2516652164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55908839a2f0 0x559088774560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 >> 172.21.15.29:0/2516652164 conn(0x5590883b7300 msgr2=0x55908876bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30d2ff5640 0 client.26057 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 shutdown_connections 2024-09-21T01:52:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.297+0000 7f30fc5dd740 1 -- 172.21.15.29:0/2516652164 wait complete. 2024-09-21T01:52:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.298+0000 7f30fc5dd740 20 client.26057 trim_cache size 0 max 16384 2024-09-21T01:52:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.327+0000 7f30fc5dd740 1 Processor -- start 2024-09-21T01:52:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.327+0000 7f30fc5dd740 1 -- start start 2024-09-21T01:52:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55908839a2f0 0x55908876f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088765860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01b1470 con 0x5590883c31b0 2024-09-21T01:52:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01b30a0 con 0x5590883a24d0 2024-09-21T01:52:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01b1b40 con 0x55908839a2f0 2024-09-21T01:52:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36932/0 (socket says 172.21.15.29:36932) 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30ee7fc640 1 -- 172.21.15.29:0/3647319881 learned_addr learned my addr 172.21.15.29:0/3647319881 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.328+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088765860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 -- 172.21.15.29:0/3647319881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55908839a2f0 msgr2=0x55908876f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55908839a2f0 0x55908876f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 -- 172.21.15.29:0/3647319881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 msgr2=0x559088765860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088765860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 -- 172.21.15.29:0/3647319881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908876cc00 con 0x5590883a24d0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x559088781ef0 tx=0x7f30e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9b48786efa5608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30d4ff9640 1 -- 172.21.15.29:0/3647319881 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e400fad0 con 0x5590883a24d0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30d4ff9640 1 -- 172.21.15.29:0/3647319881 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e4030070 con 0x5590883a24d0 2024-09-21T01:52:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.329+0000 7f30d4ff9640 1 -- 172.21.15.29:0/3647319881 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4025dc0 con 0x5590883a24d0 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.331+0000 7f30fc5dd740 1 -- 172.21.15.29:0/3647319881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 msgr2=0x55908877c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.331+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x559088781ef0 tx=0x7f30e4005f90 comp rx=0 tx=0).stop 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 -- 172.21.15.29:0/3647319881 shutdown_connections 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590883c31b0 0x559088765860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877c650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/3647319881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55908839a2f0 0x55908876f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 -- 172.21.15.29:0/3647319881 >> 172.21.15.29:0/3647319881 conn(0x5590883b7300 msgr2=0x559088772af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 -- 172.21.15.29:0/3647319881 shutdown_connections 2024-09-21T01:52:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.332+0000 7f30fc5dd740 1 -- 172.21.15.29:0/3647319881 wait complete. 2024-09-21T01:52:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.333+0000 7f30fc5dd740 1 -- start start 2024-09-21T01:52:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.333+0000 7f30fc5dd740 1 Processor -- start 2024-09-21T01:52:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.333+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 0x55908877c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 0x55908877b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01b29d0 con 0x55908839a2f0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01b22d0 con 0x5590883a24d0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01af450 con 0x5590883c31b0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36934/0 (socket says 172.21.15.29:36934) 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 -- 172.21.15.29:0/1689023106 learned_addr learned my addr 172.21.15.29:0/1689023106 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30fc5dd740 10 client.0 _ll_register_callbacks cb 0x7f30e4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30eeffd640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 0x55908877c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 0x55908877b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 -- 172.21.15.29:0/1689023106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 msgr2=0x55908877b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 0x55908877b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 -- 172.21.15.29:0/1689023106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 msgr2=0x55908877c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 0x55908877c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ee7fc640 1 -- 172.21.15.29:0/1689023106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908877bc90 con 0x5590883a24d0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30eeffd640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 0x55908877c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.334+0000 7f30ef7fe640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 0x55908877b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30ee7fc640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30e4014010 tx=0x7f30e4025d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42342407d1be44b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4038020 con 0x5590883a24d0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4030070 con 0x5590883a24d0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30e4005040 con 0x5590883a24d0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f30e4005d00 con 0x5590883a24d0 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30fc5dd740 10 client.26063 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5590887ec150 con 0x5590883a24d0 2024-09-21T01:52:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.335+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e4030070 con 0x5590883a24d0 2024-09-21T01:52:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f30e4022040 con 0x5590883a24d0 2024-09-21T01:52:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 10 client.26063 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 20 client.26063 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5590883571c0 0x5590887ee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 4 client.26063 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590886db0f0 con 0x5590883571c0 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.336+0000 7f30eeffd640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5590883571c0 0x5590887ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.338+0000 7f30d2ff5640 10 client.26063 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.338+0000 7f30eeffd640 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5590883571c0 0x5590887ee930 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697f90b545ee27c7 server_cookie=7d0beb85bef96f3f in_seq=0 out_seq=0 2024-09-21T01:52:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.354+0000 7f30d2ff5640 1 -- 172.21.15.29:0/1689023106 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590886db0f0 con 0x5590883571c0 2024-09-21T01:52:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.354+0000 7f30d2ff5640 10 client.26063 handle_command_reply: tid=0 2024-09-21T01:52:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.354+0000 7f30fc5dd740 1 client.26063 shutdown 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 msgr2=0x55908877afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30e4014010 tx=0x7f30e4025d50 comp rx=0 tx=0).stop 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 shutdown_connections 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5590883571c0 0x5590887ee930 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30d2ff5640 0 client.26063 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590883c31b0 0x55908877b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590883a24d0 0x55908877afe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 --2- 172.21.15.29:0/1689023106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55908839a2f0 0x55908877c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 >> 172.21.15.29:0/1689023106 conn(0x5590883b7300 msgr2=0x559088785f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 shutdown_connections 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.355+0000 7f30fc5dd740 1 -- 172.21.15.29:0/1689023106 wait complete. 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30fc5dd740 20 client.26063 trim_cache size 0 max 16384 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:52:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:52:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:52:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 msgr2=0x7f30d80b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 0x7f30d80b8bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30e4012640 tx=0x7f30e4014040 comp rx=0 tx=0).stop 2024-09-21T01:52:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 msgr2=0x7f30f01a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f30e800eb40 tx=0x7f30e800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 shutdown_connections 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30d80b66e0 0x7f30d80b8bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f0103fc0 0x7f30f01a17b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0103620 0x7f30f01a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 --2- 172.21.15.29:0/2633141782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30f01025a0 0x7f30f01a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.356+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 >> 172.21.15.29:0/2633141782 conn(0x7f30f00fdee0 msgr2=0x7f30f00ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.357+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 shutdown_connections 2024-09-21T01:52:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:52:48.357+0000 7f30f5670640 1 -- 172.21.15.29:0/2633141782 wait complete. 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.544+0000 7ff82dab6640 1 -- 172.21.15.29:0/3070458994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200a2010 msgr2=0x7ff8200a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 --2- 172.21.15.29:0/3070458994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200a2010 0x7ff8200a2420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff814009860 tx=0x7ff81402a340 comp rx=0 tx=0).stop 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 -- 172.21.15.29:0/3070458994 shutdown_connections 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 --2- 172.21.15.29:0/3070458994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200a2960 0x7ff8200a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 --2- 172.21.15.29:0/3070458994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200a2010 0x7ff8200a2420 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff814009860 tx=0x7ff81402a340 comp rx=0 tx=0).stop 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 --2- 172.21.15.29:0/3070458994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82009c920 0x7ff82009cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.545+0000 7ff82dab6640 1 -- 172.21.15.29:0/3070458994 >> 172.21.15.29:0/3070458994 conn(0x7ff82000a6a0 msgr2=0x7ff82000aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.546+0000 7ff82dab6640 1 -- 172.21.15.29:0/3070458994 shutdown_connections 2024-09-21T01:52:48.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.546+0000 7ff82dab6640 1 -- 172.21.15.29:0/3070458994 wait complete. 2024-09-21T01:52:48.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.547+0000 7ff82dab6640 1 Processor -- start 2024-09-21T01:52:48.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.547+0000 7ff82dab6640 1 -- start start 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.548+0000 7ff82dab6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 0x7ff8201331f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.548+0000 7ff82dab6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a2960 0x7ff820133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 0x7ff8201331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 0x7ff8201331f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51600/0 (socket says 172.21.15.29:51600) 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff82dab6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 0x7ff820138d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a2960 0x7ff820133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff8277fe640 1 -- 172.21.15.29:0/4111090442 learned_addr learned my addr 172.21.15.29:0/4111090442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff827fff640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 0x7ff820138d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200ac640 con 0x7ff8200a2960 2024-09-21T01:52:48.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.549+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200ac4c0 con 0x7ff820138970 2024-09-21T01:52:48.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200ac7c0 con 0x7ff82009c920 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff827fff640 1 -- 172.21.15.29:0/4111090442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 msgr2=0x7ff8201331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff827fff640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 0x7ff8201331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff827fff640 1 -- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a2960 msgr2=0x7ff820133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff827fff640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a2960 0x7ff820133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.550+0000 7ff827fff640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff820139550 con 0x7ff820138970 2024-09-21T01:52:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.551+0000 7ff827fff640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 0x7ff820138d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d680 tx=0x7ff81c00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.551+0000 7ff824ff9640 1 -- 172.21.15.29:0/4111090442 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c017070 con 0x7ff820138970 2024-09-21T01:52:48.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.551+0000 7ff824ff9640 1 -- 172.21.15.29:0/4111090442 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81c009820 con 0x7ff820138970 2024-09-21T01:52:48.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.551+0000 7ff824ff9640 1 -- 172.21.15.29:0/4111090442 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004ce0 con 0x7ff820138970 2024-09-21T01:52:48.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.551+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82013faf0 con 0x7ff820138970 2024-09-21T01:52:48.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.552+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82013fe90 con 0x7ff820138970 2024-09-21T01:52:48.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.557+0000 7ff83402a740 1 Processor -- start 2024-09-21T01:52:48.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 -- start start 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f7f070 0x555e75326860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 0x555e753272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555e7529cea0 con 0x555e74f561b0 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555e7528c2d0 con 0x555e74f7f070 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff824ff9640 1 -- 172.21.15.29:0/4111090442 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff81c0099d0 con 0x7ff820138970 2024-09-21T01:52:48.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff83402a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555e752900e0 con 0x555e74f5e390 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.559+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41420/0 (socket says 172.21.15.29:41420) 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 -- 172.21.15.29:0/2903509710 learned_addr learned my addr 172.21.15.29:0/2903509710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff826ffd640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f7f070 0x555e75326860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff826ffd640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 0x555e753272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 -- 172.21.15.29:0/2903509710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 msgr2=0x555e753272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 0x555e753272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 -- 172.21.15.29:0/2903509710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f7f070 msgr2=0x555e75326860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f7f070 0x555e75326860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff8277fe640 1 -- 172.21.15.29:0/2903509710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e74fea650 con 0x555e74f561b0 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.560+0000 7ff826ffd640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 0x555e753272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff824ff9640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 0x7ff8180b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff8277fe640 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff828051d10 tx=0x7ff8280759c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6833195544bd84b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff824ff9640 1 -- 172.21.15.29:0/4111090442 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff81c0e6b40 con 0x7ff820138970 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff811ffb640 1 -- 172.21.15.29:0/2903509710 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82806a7d0 con 0x555e74f561b0 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff811ffb640 1 -- 172.21.15.29:0/2903509710 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82806ae20 con 0x555e74f561b0 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.561+0000 7ff811ffb640 1 -- 172.21.15.29:0/2903509710 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828080e40 con 0x555e74f561b0 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.562+0000 7ff826ffd640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 0x7ff8180b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.563+0000 7ff83402a740 1 -- 172.21.15.29:0/2903509710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 msgr2=0x555e75326da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.563+0000 7ff83402a740 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff828051d10 tx=0x7ff8280759c0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff826ffd640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 0x7ff8180b8c30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff814008000 tx=0x7ff814000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 -- 172.21.15.29:0/2903509710 shutdown_connections 2024-09-21T01:52:48.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f5e390 0x555e753272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f561b0 0x555e75326da0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 --2- 172.21.15.29:0/2903509710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f7f070 0x555e75326860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 -- 172.21.15.29:0/2903509710 >> 172.21.15.29:0/2903509710 conn(0x555e74f73460 msgr2=0x555e75324750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 -- 172.21.15.29:0/2903509710 shutdown_connections 2024-09-21T01:52:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.564+0000 7ff83402a740 1 -- 172.21.15.29:0/2903509710 wait complete. 2024-09-21T01:52:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.565+0000 7ff83402a740 1 -- start start 2024-09-21T01:52:48.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.565+0000 7ff83402a740 1 Processor -- start 2024-09-21T01:52:48.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e75331e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555e750ed700 con 0x555e74f7f070 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555e75227160 con 0x555e74f5e390 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555e75335040 con 0x555e74f561b0 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff83402a740 10 client.0 _ll_register_callbacks cb 0x555e752409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e75331e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36968/0 (socket says 172.21.15.29:36968) 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff826ffd640 1 -- 172.21.15.29:0/3144975060 learned_addr learned my addr 172.21.15.29:0/3144975060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.566+0000 7ff827fff640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 -- 172.21.15.29:0/3144975060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 msgr2=0x555e75331e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e75331e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 -- 172.21.15.29:0/3144975060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 msgr2=0x555e753328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 -- 172.21.15.29:0/3144975060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e75254c90 con 0x555e74f5e390 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff826ffd640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff814009860 tx=0x7ff81402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160443eb0bfd3b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.567+0000 7ff806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81404d070 con 0x555e74f5e390 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff814004ad0 con 0x555e74f5e390 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82806ae20 con 0x555e74f5e390 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff814037040 con 0x555e74f5e390 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff83402a740 10 client.26072 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555e75340c10 con 0x555e74f5e390 2024-09-21T01:52:48.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.568+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82806ae20 con 0x555e74f5e390 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff814048070 con 0x555e74f5e390 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 10 client.26072 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 10 client.26072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 20 client.26072 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f3d9f0 0x555e75344670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 4 client.26072 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e7529edf0 con 0x555e74f3d9f0 2024-09-21T01:52:48.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.569+0000 7ff8277fe640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f3d9f0 0x555e75344670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.570+0000 7ff806ffd640 10 client.26072 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.570+0000 7ff8277fe640 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f3d9f0 0x555e75344670 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5e1164f6186701 server_cookie=ee6a3d3ac80eaabf in_seq=0 out_seq=0 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.571+0000 7ff806ffd640 1 -- 172.21.15.29:0/3144975060 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e7529edf0 con 0x555e74f3d9f0 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.571+0000 7ff806ffd640 10 client.26072 handle_command_reply: tid=0 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.571+0000 7ff83402a740 1 client.26072 shutdown 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 msgr2=0x555e75332380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff814009860 tx=0x7ff81402a340 comp rx=0 tx=0).stop 2024-09-21T01:52:48.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 shutdown_connections 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f3d9f0 0x555e75344670 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75332380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 --2- 172.21.15.29:0/3144975060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e75331e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 >> 172.21.15.29:0/3144975060 conn(0x555e74f73460 msgr2=0x555e753299f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff806ffd640 0 client.26072 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 shutdown_connections 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 1 -- 172.21.15.29:0/3144975060 wait complete. 2024-09-21T01:52:48.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.572+0000 7ff83402a740 20 client.26072 trim_cache size 0 max 16384 2024-09-21T01:52:48.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.595+0000 7ff83402a740 1 Processor -- start 2024-09-21T01:52:48.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.595+0000 7ff83402a740 1 -- start start 2024-09-21T01:52:48.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 0x555e7532f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 0x555e7532f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36982/0 (socket says 172.21.15.29:36982) 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 0x555e7532f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41440/0 (socket says 172.21.15.29:41440) 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff8277fe640 1 -- 172.21.15.29:0/4188861714 learned_addr learned my addr 172.21.15.29:0/4188861714 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f7f070 0x555e7532f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201485e0 con 0x555e74f5e390 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff82014a210 con 0x555e74f561b0 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff820148cb0 con 0x555e74f7f070 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.596+0000 7ff827fff640 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f7f070 0x555e7532f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 -- 172.21.15.29:0/4188861714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f7f070 msgr2=0x555e7532f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f7f070 0x555e7532f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 -- 172.21.15.29:0/4188861714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 msgr2=0x555e7532f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 0x555e7532f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 -- 172.21.15.29:0/4188861714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e75333100 con 0x555e74f561b0 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.597+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8280692a0 tx=0x7ff828069080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6967bbba8d9131 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.598+0000 7ff8117fa640 1 -- 172.21.15.29:0/4188861714 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82806a7d0 con 0x555e74f561b0 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.598+0000 7ff8117fa640 1 -- 172.21.15.29:0/4188861714 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82806ae20 con 0x555e74f561b0 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.598+0000 7ff8117fa640 1 -- 172.21.15.29:0/4188861714 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280818a0 con 0x555e74f561b0 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.599+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 msgr2=0x555e75327fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.599+0000 7ff83402a740 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8280692a0 tx=0x7ff828069080 comp rx=0 tx=0).stop 2024-09-21T01:52:48.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 shutdown_connections 2024-09-21T01:52:48.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f7f070 0x555e7532f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f5e390 0x555e7532f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 --2- 172.21.15.29:0/4188861714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f561b0 0x555e75327fa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 >> 172.21.15.29:0/4188861714 conn(0x555e74f73460 msgr2=0x555e75327890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 shutdown_connections 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.601+0000 7ff83402a740 1 -- 172.21.15.29:0/4188861714 wait complete. 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.602+0000 7ff83402a740 1 -- start start 2024-09-21T01:52:48.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.602+0000 7ff83402a740 1 Processor -- start 2024-09-21T01:52:48.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e7533aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff820149b40 con 0x555e74f7f070 2024-09-21T01:52:48.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff820149440 con 0x555e74f5e390 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201465c0 con 0x555e74f561b0 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36994/0 (socket says 172.21.15.29:36994) 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff826ffd640 1 -- 172.21.15.29:0/4230854744 learned_addr learned my addr 172.21.15.29:0/4230854744 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff83402a740 10 client.0 _ll_register_callbacks cb 0x7ff814035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:48.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.603+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e7533aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 -- 172.21.15.29:0/4230854744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 msgr2=0x555e7533aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e7533aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 -- 172.21.15.29:0/4230854744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 msgr2=0x555e753364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 -- 172.21.15.29:0/4230854744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e75336c10 con 0x555e74f5e390 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff826ffd640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff81402a930 tx=0x7ff814033f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01534f7403fbe72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.604+0000 7ff806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81404d070 con 0x555e74f5e390 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff8140037f0 con 0x555e74f5e390 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82806ae20 con 0x555e74f5e390 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff814033790 con 0x555e74f5e390 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff83402a740 10 client.26078 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:48.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.605+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555e753bff80 con 0x555e74f5e390 2024-09-21T01:52:48.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.606+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814062070 con 0x555e74f5e390 2024-09-21T01:52:48.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff814048040 con 0x555e74f5e390 2024-09-21T01:52:48.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 10 client.26078 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:48.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 10 client.26078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:48.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 20 client.26078 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f12da0 0x555e753c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 4 client.26078 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff82806ae20 con 0x555e74f12da0 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.607+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f12da0 0x555e753c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.608+0000 7ff8277fe640 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f12da0 0x555e753c2760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c33fe04820b26c server_cookie=a6768d6fc8a7a8ab in_seq=0 out_seq=0 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.608+0000 7ff806ffd640 10 client.26078 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff806ffd640 1 -- 172.21.15.29:0/4230854744 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff82806ae20 con 0x555e74f12da0 2024-09-21T01:52:48.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff806ffd640 10 client.26078 handle_command_reply: tid=0 2024-09-21T01:52:48.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 client.26078 shutdown 2024-09-21T01:52:48.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 msgr2=0x555e75335f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff81402a930 tx=0x7ff814033f90 comp rx=0 tx=0).stop 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 shutdown_connections 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555e74f12da0 0x555e753c2760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e74f7f070 0x555e753364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e74f5e390 0x555e75335f60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 --2- 172.21.15.29:0/4230854744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555e74f561b0 0x555e7533aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 >> 172.21.15.29:0/4230854744 conn(0x555e74f73460 msgr2=0x555e7533d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.609+0000 7ff806ffd640 0 client.26078 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.610+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 shutdown_connections 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.610+0000 7ff83402a740 1 -- 172.21.15.29:0/4230854744 wait complete. 2024-09-21T01:52:48.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.610+0000 7ff83402a740 20 client.26078 trim_cache size 0 max 16384 2024-09-21T01:52:48.613 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 msgr2=0x7ff8180b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 0x7ff8180b8c30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff814008000 tx=0x7ff814000bc0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 msgr2=0x7ff820138d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 0x7ff820138d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d680 tx=0x7ff81c00db40 comp rx=0 tx=0).stop 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 shutdown_connections 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6720 0x7ff8180b8c30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff820138970 0x7ff820138d80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a2960 0x7ff820133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 --2- 172.21.15.29:0/4111090442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82009c920 0x7ff8201331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:48.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.611+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 >> 172.21.15.29:0/4111090442 conn(0x7ff82000a6a0 msgr2=0x7ff82009f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:48.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.612+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 shutdown_connections 2024-09-21T01:52:48.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:48.612+0000 7ff82dab6640 1 -- 172.21.15.29:0/4111090442 wait complete. 2024-09-21T01:52:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:49 smithi029 ceph-mon[31228]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-09-21T01:52:50.100 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:49 smithi066 ceph-mon[35001]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-09-21T01:52:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:49 smithi190 ceph-mon[34368]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-09-21T01:52:50.683 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:52:50.684 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:52:51.146 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.749+0000 7fe992195640 1 -- 172.21.15.29:0/3285280595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c102a60 msgr2=0x7fe98c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.749+0000 7fe992195640 1 --2- 172.21.15.29:0/3285280595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c102a60 0x7fe98c102e70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe974009860 tx=0x7fe97402a360 comp rx=0 tx=0).stop 2024-09-21T01:52:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.753+0000 7fe992195640 1 -- 172.21.15.29:0/3285280595 shutdown_connections 2024-09-21T01:52:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.753+0000 7fe992195640 1 --2- 172.21.15.29:0/3285280595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c108cf0 0x7fe98c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.753+0000 7fe992195640 1 --2- 172.21.15.29:0/3285280595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.753+0000 7fe992195640 1 --2- 172.21.15.29:0/3285280595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c102a60 0x7fe98c102e70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.753+0000 7fe992195640 1 -- 172.21.15.29:0/3285280595 >> 172.21.15.29:0/3285280595 conn(0x7fe98c079730 msgr2=0x7fe98c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.754+0000 7fe992195640 1 -- 172.21.15.29:0/3285280595 shutdown_connections 2024-09-21T01:52:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.754+0000 7fe992195640 1 -- 172.21.15.29:0/3285280595 wait complete. 2024-09-21T01:52:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.755+0000 7fe992195640 1 Processor -- start 2024-09-21T01:52:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 -- start start 2024-09-21T01:52:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c102a60 0x7fe98c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c108cf0 0x7fe98c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1118a0 con 0x7fe98c102a60 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c111720 con 0x7fe98c1083a0 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe992195640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c111a20 con 0x7fe98c108cf0 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe98affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe98affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37012/0 (socket says 172.21.15.29:37012) 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.756+0000 7fe98affd640 1 -- 172.21.15.29:0/2522019667 learned_addr learned my addr 172.21.15.29:0/2522019667 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:51.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c102a60 0x7fe98c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c108cf0 0x7fe98c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 -- 172.21.15.29:0/2522019667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c108cf0 msgr2=0x7fe98c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c108cf0 0x7fe98c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 -- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c102a60 msgr2=0x7fe98c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c102a60 0x7fe98c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 -- 172.21.15.29:0/2522019667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c198e30 con 0x7fe98c1083a0 2024-09-21T01:52:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.757+0000 7fe98affd640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9c0 tx=0x7fe97c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.759+0000 7fe988ff9640 1 -- 172.21.15.29:0/2522019667 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00cd80 con 0x7fe98c1083a0 2024-09-21T01:52:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.759+0000 7fe988ff9640 1 -- 172.21.15.29:0/2522019667 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe97c0054d0 con 0x7fe98c1083a0 2024-09-21T01:52:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.759+0000 7fe988ff9640 1 -- 172.21.15.29:0/2522019667 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c010430 con 0x7fe98c1083a0 2024-09-21T01:52:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.761+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c19daf0 con 0x7fe98c1083a0 2024-09-21T01:52:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.761+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c19dfb0 con 0x7fe98c1083a0 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.768+0000 7fe998709740 1 Processor -- start 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.768+0000 7fe988ff9640 1 -- 172.21.15.29:0/2522019667 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe97c005000 con 0x7fe98c1083a0 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.769+0000 7fe998709740 1 -- start start 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.770+0000 7fe988ff9640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 0x7fe9780b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 0x7fe9780b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f67f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f67f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5f5e60 con 0x55d81f2b7670 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5e5290 con 0x55d81f2d8350 2024-09-21T01:52:51.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.771+0000 7fe998709740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5e90a0 con 0x55d81f2af490 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe988ff9640 1 -- 172.21.15.29:0/2522019667 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe97c0e52b0 con 0x7fe98c1083a0 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 0x7fe9780b8ba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe984005c40 tx=0x7fe984005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41474/0 (socket says 172.21.15.29:41474) 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 -- 172.21.15.29:0/2606602410 learned_addr learned my addr 172.21.15.29:0/2606602410 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f67f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 -- 172.21.15.29:0/2606602410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 msgr2=0x55d81f67f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f67f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 -- 172.21.15.29:0/2606602410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 msgr2=0x55d81f67f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f67f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 -- 172.21.15.29:0/2606602410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81f343930 con 0x55d81f2b7670 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f67f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.772+0000 7fe98bfff640 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe984009f80 tx=0x7fe984012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c1302e3d806f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.773+0000 7fe981ffb640 1 -- 172.21.15.29:0/2606602410 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984004b20 con 0x55d81f2b7670 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.773+0000 7fe981ffb640 1 -- 172.21.15.29:0/2606602410 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe984003e20 con 0x55d81f2b7670 2024-09-21T01:52:51.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.773+0000 7fe981ffb640 1 -- 172.21.15.29:0/2606602410 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840047d0 con 0x55d81f2b7670 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.774+0000 7fe998709740 1 -- 172.21.15.29:0/2606602410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 msgr2=0x55d81f67feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.774+0000 7fe998709740 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe984009f80 tx=0x7fe984012000 comp rx=0 tx=0).stop 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.775+0000 7fe998709740 1 -- 172.21.15.29:0/2606602410 shutdown_connections 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.775+0000 7fe998709740 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2b7670 0x55d81f67feb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.775+0000 7fe998709740 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f67f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.775+0000 7fe998709740 1 --2- 172.21.15.29:0/2606602410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f67f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.775+0000 7fe998709740 1 -- 172.21.15.29:0/2606602410 >> 172.21.15.29:0/2606602410 conn(0x55d81f2cc740 msgr2=0x55d81f67d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.777+0000 7fe998709740 1 -- 172.21.15.29:0/2606602410 shutdown_connections 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.777+0000 7fe998709740 1 -- 172.21.15.29:0/2606602410 wait complete. 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.778+0000 7fe998709740 1 -- start start 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.778+0000 7fe998709740 1 Processor -- start 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f68b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f696f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe98b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe98b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37032/0 (socket says 172.21.15.29:37032) 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d81f446890 con 0x55d81f2d8350 2024-09-21T01:52:51.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d81f57fd40 con 0x55d81f2af490 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d81f68dc10 con 0x55d81f2b7670 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe98b7fe640 1 -- 172.21.15.29:0/2858707590 learned_addr learned my addr 172.21.15.29:0/2858707590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.779+0000 7fe998709740 10 client.0 _ll_register_callbacks cb 0x55d81f5e3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98affd640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f68b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 -- 172.21.15.29:0/2858707590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 msgr2=0x55d81f68b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f68b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 -- 172.21.15.29:0/2858707590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 msgr2=0x55d81f696f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f696f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 -- 172.21.15.29:0/2858707590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81f697570 con 0x55d81f2af490 2024-09-21T01:52:51.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.780+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe974004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d8cadc0079f220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.781+0000 7fe98affd640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f68b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.781+0000 7fe96affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.781+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974037040 con 0x55d81f2af490 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.781+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe974030450 con 0x55d81f2af490 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.782+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe984003e20 con 0x55d81f2af490 2024-09-21T01:52:51.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.782+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe974030650 con 0x55d81f2af490 2024-09-21T01:52:51.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 10 client.26087 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:51.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d81f699dd0 con 0x55d81f2af490 2024-09-21T01:52:51.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9740307f0 con 0x55d81f2af490 2024-09-21T01:52:51.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fe97403d680 con 0x55d81f2af490 2024-09-21T01:52:51.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 10 client.26087 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:51.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:51.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 20 client.26087 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:51.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f296cd0 0x55d81f69d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 4 client.26087 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:51.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.786+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe984003e20 con 0x55d81f296cd0 2024-09-21T01:52:51.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.787+0000 7fe98affd640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f296cd0 0x55d81f69d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.787+0000 7fe96affd640 10 client.26087 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:51.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.789+0000 7fe98affd640 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f296cd0 0x55d81f69d3a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62c5956ba8bce25 server_cookie=acdfc9417caf6ff3 in_seq=0 out_seq=0 2024-09-21T01:52:51.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.790+0000 7fe96affd640 1 -- 172.21.15.29:0/2858707590 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe984003e20 con 0x55d81f296cd0 2024-09-21T01:52:51.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.791+0000 7fe96affd640 10 client.26087 handle_command_reply: tid=0 2024-09-21T01:52:51.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.791+0000 7fe998709740 1 client.26087 shutdown 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 msgr2=0x55d81f68b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe974004270 comp rx=0 tx=0).stop 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 shutdown_connections 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f296cd0 0x55d81f69d3a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f696f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f68b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 --2- 172.21.15.29:0/2858707590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2af490 0x55d81f68b0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 >> 172.21.15.29:0/2858707590 conn(0x55d81f2cc740 msgr2=0x55d81f682610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe96affd640 0 client.26087 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:51.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 shutdown_connections 2024-09-21T01:52:51.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.793+0000 7fe998709740 1 -- 172.21.15.29:0/2858707590 wait complete. 2024-09-21T01:52:51.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.794+0000 7fe998709740 20 client.26087 trim_cache size 0 max 16384 2024-09-21T01:52:51.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.827+0000 7fe998709740 1 Processor -- start 2024-09-21T01:52:51.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.827+0000 7fe998709740 1 -- start start 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2af490 0x55d81f696e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f684510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c074b20 con 0x55d81f2af490 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1c8710 con 0x55d81f2d8350 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe998709740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c0751f0 con 0x55d81f2b7670 2024-09-21T01:52:51.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe98bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe98bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40538/0 (socket says 172.21.15.29:40538) 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.828+0000 7fe98bfff640 1 -- 172.21.15.29:0/722673315 learned_addr learned my addr 172.21.15.29:0/722673315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98affd640 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f684510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2af490 0x55d81f696e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 -- 172.21.15.29:0/722673315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 msgr2=0x55d81f684510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f684510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 -- 172.21.15.29:0/722673315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2af490 msgr2=0x55d81f696e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2af490 0x55d81f696e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 -- 172.21.15.29:0/722673315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81f686730 con 0x55d81f2d8350 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe98bfff640 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9840095e0 tx=0x7fe98400ddc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff804bbe81f5dda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe9817fa640 1 -- 172.21.15.29:0/722673315 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984003d80 con 0x55d81f2d8350 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe9817fa640 1 -- 172.21.15.29:0/722673315 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe984005570 con 0x55d81f2d8350 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.829+0000 7fe9817fa640 1 -- 172.21.15.29:0/722673315 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984023910 con 0x55d81f2d8350 2024-09-21T01:52:51.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 -- 172.21.15.29:0/722673315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 msgr2=0x55d81f684a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9840095e0 tx=0x7fe98400ddc0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 -- 172.21.15.29:0/722673315 shutdown_connections 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2d8350 0x55d81f684a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2b7670 0x55d81f684510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 --2- 172.21.15.29:0/722673315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2af490 0x55d81f696e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 -- 172.21.15.29:0/722673315 >> 172.21.15.29:0/722673315 conn(0x55d81f2cc740 msgr2=0x55d81f689720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 -- 172.21.15.29:0/722673315 shutdown_connections 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.831+0000 7fe998709740 1 -- 172.21.15.29:0/722673315 wait complete. 2024-09-21T01:52:51.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.832+0000 7fe998709740 1 -- start start 2024-09-21T01:52:51.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.832+0000 7fe998709740 1 Processor -- start 2024-09-21T01:52:51.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.833+0000 7fe998709740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.834+0000 7fe998709740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 0x55d81f6933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.834+0000 7fe98b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.834+0000 7fe98b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41884/0 (socket says 172.21.15.29:41884) 2024-09-21T01:52:51.836 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:51 smithi029 ceph-mon[31228]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-21T01:52:51.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe98b7fe640 1 -- 172.21.15.29:0/3560678784 learned_addr learned my addr 172.21.15.29:0/3560678784 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:51.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe98affd640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 0x55d81f6933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f68de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1c8040 con 0x55d81f2d8350 2024-09-21T01:52:51.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1c7940 con 0x55d81f2b7670 2024-09-21T01:52:51.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.835+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c199390 con 0x55d81f2af490 2024-09-21T01:52:51.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe998709740 10 client.0 _ll_register_callbacks cb 0x7fe97c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:51.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe98bfff640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f68de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe98bfff640 1 -- 172.21.15.29:0/3560678784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 msgr2=0x55d81f692e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe98bfff640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe98bfff640 1 -- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 msgr2=0x55d81f6933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:51.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.836+0000 7fe98bfff640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 0x55d81f6933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.837+0000 7fe98bfff640 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81f6939f0 con 0x55d81f2d8350 2024-09-21T01:52:51.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.838+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:51.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.838+0000 7fe98affd640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 0x55d81f6933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:51.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.838+0000 7fe98bfff640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f68de20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe98402e040 tx=0x7fe984005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4a0934d01e900e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:51.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.838+0000 7fe96affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:52:51.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.839+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840052c0 con 0x55d81f2d8350 2024-09-21T01:52:51.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.839+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe984023550 con 0x55d81f2d8350 2024-09-21T01:52:51.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.839+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe984005570 con 0x55d81f2d8350 2024-09-21T01:52:51.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.840+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe984023750 con 0x55d81f2d8350 2024-09-21T01:52:51.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.840+0000 7fe998709740 10 client.16362 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:51.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.840+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d81f7081a0 con 0x55d81f2d8350 2024-09-21T01:52:51.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.840+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe984005570 con 0x55d81f2d8350 2024-09-21T01:52:51.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.841+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fe98400b040 con 0x55d81f2d8350 2024-09-21T01:52:51.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.841+0000 7fe998709740 10 client.16362 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.841+0000 7fe998709740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.842+0000 7fe998709740 20 client.16362 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.842+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f26c080 0x55d81f70a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.842+0000 7fe998709740 4 client.16362 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.842+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d81f5f7db0 con 0x55d81f26c080 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.842+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f26c080 0x55d81f70a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.843+0000 7fe98b7fe640 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f26c080 0x55d81f70a5f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7618152c75cda5be server_cookie=983ab63d8c3c8df8 in_seq=0 out_seq=0 2024-09-21T01:52:51.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.843+0000 7fe96affd640 10 client.16362 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:51.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.844+0000 7fe96affd640 1 -- 172.21.15.29:0/3560678784 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d81f5f7db0 con 0x55d81f26c080 2024-09-21T01:52:51.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.844+0000 7fe96affd640 10 client.16362 handle_command_reply: tid=0 2024-09-21T01:52:51.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.844+0000 7fe998709740 1 client.16362 shutdown 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 msgr2=0x55d81f68de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f68de20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe98402e040 tx=0x7fe984005a40 comp rx=0 tx=0).stop 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 shutdown_connections 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d81f26c080 0x55d81f70a5f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe96affd640 0 client.16362 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d81f2d8350 0x55d81f68de20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d81f2b7670 0x55d81f6933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 --2- 172.21.15.29:0/3560678784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81f2af490 0x55d81f692e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.845+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 >> 172.21.15.29:0/3560678784 conn(0x55d81f2cc740 msgr2=0x55d81f67cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.846+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 shutdown_connections 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.846+0000 7fe998709740 1 -- 172.21.15.29:0/3560678784 wait complete. 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.846+0000 7fe998709740 20 client.16362 trim_cache size 0 max 16384 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:52:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d", 2024-09-21T01:52:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:52:51.852 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:52:51.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.848+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 msgr2=0x7fe9780b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.848+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 0x7fe9780b8ba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe984005c40 tx=0x7fe984005bd0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.848+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 msgr2=0x7fe98c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.848+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9c0 tx=0x7fe97c00ee80 comp rx=0 tx=0).stop 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 shutdown_connections 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9780b6690 0x7fe9780b8ba0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c108cf0 0x7fe98c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe98c1083a0 0x7fe98c198090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 --2- 172.21.15.29:0/2522019667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c102a60 0x7fe98c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 >> 172.21.15.29:0/2522019667 conn(0x7fe98c079730 msgr2=0x7fe98c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 shutdown_connections 2024-09-21T01:52:51.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:51.849+0000 7fe992195640 1 -- 172.21.15.29:0/2522019667 wait complete. 2024-09-21T01:52:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:51 smithi190 ceph-mon[34368]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-21T01:52:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:51 smithi066 ceph-mon[35001]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-21T01:52:52.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:52:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 255.0 s, 4339.8 tps, lat 23.137 ms stddev 31.022 2024-09-21T01:52:53.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:52:53.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:52 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.170 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:52 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:52:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:52 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.878+0000 7f59a159a640 1 -- 172.21.15.29:0/2981333848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980318a0 con 0x7f59a41083a0 2024-09-21T01:52:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.878+0000 7f59a8dc0640 1 -- 172.21.15.29:0/2981333848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 msgr2=0x7f59a41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.878+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/2981333848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a41087b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5998009900 tx=0x7f599802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 -- 172.21.15.29:0/2981333848 shutdown_connections 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/2981333848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4108cf0 0x7f59a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/2981333848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a41087b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/2981333848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4102a60 0x7f59a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 -- 172.21.15.29:0/2981333848 >> 172.21.15.29:0/2981333848 conn(0x7f59a4079730 msgr2=0x7f59a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 -- 172.21.15.29:0/2981333848 shutdown_connections 2024-09-21T01:52:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.879+0000 7f59a8dc0640 1 -- 172.21.15.29:0/2981333848 wait complete. 2024-09-21T01:52:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.880+0000 7f59a8dc0640 1 Processor -- start 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 -- start start 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 0x7f59a41119c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4113ab0 con 0x7f59a41083a0 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4113930 con 0x7f59a4108cf0 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a8dc0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4113c30 con 0x7f59a4102a60 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40562/0 (socket says 172.21.15.29:40562) 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a2d9d640 1 -- 172.21.15.29:0/4159284030 learned_addr learned my addr 172.21.15.29:0/4159284030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.881+0000 7f59a259c640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 0x7f59a41119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 -- 172.21.15.29:0/4159284030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 msgr2=0x7f59a41104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 -- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 msgr2=0x7f59a410ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a410ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 -- 172.21.15.29:0/4159284030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a4110d10 con 0x7f59a4102a60 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59a259c640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 0x7f59a41119c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f599400e9f0 tx=0x7f599400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59937fe640 1 -- 172.21.15.29:0/4159284030 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400cd80 con 0x7f59a4102a60 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.882+0000 7f59937fe640 1 -- 172.21.15.29:0/4159284030 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59940035f0 con 0x7f59a4102a60 2024-09-21T01:52:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.883+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41ace40 con 0x7f59a4102a60 2024-09-21T01:52:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.883+0000 7f59937fe640 1 -- 172.21.15.29:0/4159284030 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994010670 con 0x7f59a4102a60 2024-09-21T01:52:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.883+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41ad2f0 con 0x7f59a4102a60 2024-09-21T01:52:53.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.886+0000 7f59937fe640 1 -- 172.21.15.29:0/4159284030 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f59940108d0 con 0x7f59a4102a60 2024-09-21T01:52:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.889+0000 7f59937fe640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 0x7f598c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.889+0000 7f59937fe640 1 -- 172.21.15.29:0/4159284030 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f59940e83e0 con 0x7f59a4102a60 2024-09-21T01:52:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.890+0000 7f59af334740 1 Processor -- start 2024-09-21T01:52:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.890+0000 7f59af334740 1 -- start start 2024-09-21T01:52:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.890+0000 7f59af334740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.890+0000 7f59af334740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59af334740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41912/0 (socket says 172.21.15.29:41912) 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a259c640 1 -- 172.21.15.29:0/3344277561 learned_addr learned my addr 172.21.15.29:0/3344277561 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564de3da8e50 con 0x564de3a6a390 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564de3d98280 con 0x564de3a621b0 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564de3d9c090 con 0x564de3a8b070 2024-09-21T01:52:53.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 0x7f598c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3344277561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 msgr2=0x564de3e32270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3344277561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 msgr2=0x564de3e32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.891+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3344277561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de3af6650 con 0x564de3a621b0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.892+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e327b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5998006010 tx=0x7f5998032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ded52aed820d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.892+0000 7f5990ff9640 1 -- 172.21.15.29:0/3344277561 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998035070 con 0x564de3a621b0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.892+0000 7f5990ff9640 1 -- 172.21.15.29:0/3344277561 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5998031820 con 0x564de3a621b0 2024-09-21T01:52:53.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.892+0000 7f5990ff9640 1 -- 172.21.15.29:0/3344277561 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980039f0 con 0x564de3a621b0 2024-09-21T01:52:53.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 msgr2=0x564de3e327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e327b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5998006010 tx=0x7f5998032f90 comp rx=0 tx=0).stop 2024-09-21T01:52:53.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 0x7f598c0b8aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5998009850 tx=0x7f5998003b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 shutdown_connections 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e327b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 --2- 172.21.15.29:0/3344277561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 >> 172.21.15.29:0/3344277561 conn(0x564de3a7f460 msgr2=0x564de3e30160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 shutdown_connections 2024-09-21T01:52:53.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.894+0000 7f59af334740 1 -- 172.21.15.29:0/3344277561 wait complete. 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 -- start start 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 Processor -- start 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a621b0 0x564de3e4e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a8b070 0x564de3e5ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564de3bf9700 con 0x564de3a6a390 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564de3d33110 con 0x564de3a8b070 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.895+0000 7f59af334740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564de3e519b0 con 0x564de3a621b0 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59af334740 10 client.0 _ll_register_callbacks cb 0x564de3d4c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59a1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59a1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34806/0 (socket says 172.21.15.29:34806) 2024-09-21T01:52:53.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3077283225 learned_addr learned my addr 172.21.15.29:0/3077283225 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:53.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59a259c640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a621b0 0x564de3e4e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.896+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a8b070 0x564de3e5ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3077283225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a621b0 msgr2=0x564de3e4e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a621b0 0x564de3e4e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a8b070 msgr2=0x564de3e5ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a8b070 0x564de3e5ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 -- 172.21.15.29:0/3077283225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de3d60c40 con 0x564de3a6a390 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d605bb86cca1b60c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.897+0000 7f5989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:52:53.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.899+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998035030 con 0x564de3a6a390 2024-09-21T01:52:53.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.899+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f599804e070 con 0x564de3a6a390 2024-09-21T01:52:53.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.900+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5998031820 con 0x564de3a6a390 2024-09-21T01:52:53.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.901+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f59980039f0 con 0x564de3a6a390 2024-09-21T01:52:53.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.901+0000 7f59af334740 10 client.16371 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:53.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.901+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564de3e5d6c0 con 0x564de3a6a390 2024-09-21T01:52:53.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5998031820 con 0x564de3a6a390 2024-09-21T01:52:53.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f5998067020 con 0x564de3a6a390 2024-09-21T01:52:53.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 10 client.16371 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:53.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:53.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 20 client.16371 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a499f0 0x564de3e60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 4 client.16371 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.902+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de3daada0 con 0x564de3a499f0 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.903+0000 7f59a259c640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a499f0 0x564de3e60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.903+0000 7f5989ffb640 10 client.16371 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:53.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.904+0000 7f59a259c640 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a499f0 0x564de3e60a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2668b3b0ae6343d server_cookie=97b5a3474d113054 in_seq=0 out_seq=0 2024-09-21T01:52:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.905+0000 7f5989ffb640 1 -- 172.21.15.29:0/3077283225 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564de3daada0 con 0x564de3a499f0 2024-09-21T01:52:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.905+0000 7f5989ffb640 10 client.16371 handle_command_reply: tid=0 2024-09-21T01:52:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.905+0000 7f59af334740 1 client.16371 shutdown 2024-09-21T01:52:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 msgr2=0x564de3e4ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998005780 comp rx=0 tx=0).stop 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 shutdown_connections 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a499f0 0x564de3e60a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a8b070 0x564de3e5ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a6a390 0x564de3e4ee50 secure :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998005780 comp rx=0 tx=0).stop 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 --2- 172.21.15.29:0/3077283225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a621b0 0x564de3e4e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 >> 172.21.15.29:0/3077283225 conn(0x564de3a7f460 msgr2=0x564de3e332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.906+0000 7f5989ffb640 0 client.16371 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:53.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.914+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 shutdown_connections 2024-09-21T01:52:53.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.915+0000 7f59af334740 1 -- 172.21.15.29:0/3077283225 wait complete. 2024-09-21T01:52:53.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.916+0000 7f59af334740 20 client.16371 trim_cache size 0 max 16384 2024-09-21T01:52:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.952+0000 7f59af334740 1 Processor -- start 2024-09-21T01:52:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.953+0000 7f59af334740 1 -- start start 2024-09-21T01:52:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.953+0000 7f59af334740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.953+0000 7f59af334740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.953+0000 7f59a259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34810/0 (socket says 172.21.15.29:34810) 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 -- 172.21.15.29:0/906793604 learned_addr learned my addr 172.21.15.29:0/906793604 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59af334740 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4075280 con 0x564de3a7f460 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41aee80 con 0x564de3a621b0 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ad920 con 0x564de3a8b070 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 -- 172.21.15.29:0/906793604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 msgr2=0x564de3e32cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 -- 172.21.15.29:0/906793604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 msgr2=0x564de3e32790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e32790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 -- 172.21.15.29:0/906793604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de3e30510 con 0x564de3a7f460 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.954+0000 7f59a259c640 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f59940ae510 tx=0x7f59940b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a6b63bfda4bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.955+0000 7f598bfff640 1 -- 172.21.15.29:0/906793604 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940c9070 con 0x564de3a7f460 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.955+0000 7f598bfff640 1 -- 172.21.15.29:0/906793604 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59940b7fa0 con 0x564de3a7f460 2024-09-21T01:52:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.955+0000 7f598bfff640 1 -- 172.21.15.29:0/906793604 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940aa7f0 con 0x564de3a7f460 2024-09-21T01:52:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.956+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 msgr2=0x564de3e32250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.956+0000 7f59af334740 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f59940ae510 tx=0x7f59940b1040 comp rx=0 tx=0).stop 2024-09-21T01:52:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 shutdown_connections 2024-09-21T01:52:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e32cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 --2- 172.21.15.29:0/906793604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e32250 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 >> 172.21.15.29:0/906793604 conn(0x564de3e2f120 msgr2=0x564de3e4d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 shutdown_connections 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.957+0000 7f59af334740 1 -- 172.21.15.29:0/906793604 wait complete. 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.958+0000 7f59af334740 1 -- start start 2024-09-21T01:52:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.958+0000 7f59af334740 1 Processor -- start 2024-09-21T01:52:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59a259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e62fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59a259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e63fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40622/0 (socket says 172.21.15.29:40622) 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ae7b0 con 0x564de3a7f460 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ae0b0 con 0x564de3a621b0 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4073260 con 0x564de3a8b070 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59af334740 10 client.0 _ll_register_callbacks cb 0x7f5998009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59a259c640 1 -- 172.21.15.29:0/3252079387 learned_addr learned my addr 172.21.15.29:0/3252079387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.959+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e62fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 -- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 msgr2=0x564de3e63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 -- 172.21.15.29:0/3252079387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 msgr2=0x564de3e62a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e62a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 -- 172.21.15.29:0/3252079387 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de3e634e0 con 0x564de3a8b070 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a1d9b640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f59a2d9d640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e62fa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f599c0049d0 tx=0x7f599c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a971cfd83b6fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.960+0000 7f5989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c021060 con 0x564de3a8b070 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f599c00e7c0 con 0x564de3a8b070 2024-09-21T01:52:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59940b7fa0 con 0x564de3a8b070 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f599c00e9c0 con 0x564de3a8b070 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f59af334740 10 client.26149 fetch_fsmap learned FSMap version 38 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.961+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564de3ecbf30 con 0x564de3a8b070 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59940b7fa0 con 0x564de3a8b070 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f599c01b070 con 0x564de3a8b070 2024-09-21T01:52:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 10 client.26149 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 10 client.26149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 20 client.26149 populate_metadata read hostname 'smithi029' 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a1eda0 0x564de3ece750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 4 client.26149 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564de3da6850 con 0x564de3a1eda0 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.963+0000 7f59a259c640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a1eda0 0x564de3ece750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:52:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.964+0000 7f5989ffb640 10 client.26149 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.964+0000 7f59a259c640 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a1eda0 0x564de3ece750 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dd0720c48c53b4 server_cookie=a025f4d9b746c4d5 in_seq=0 out_seq=0 2024-09-21T01:52:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.965+0000 7f5989ffb640 1 -- 172.21.15.29:0/3252079387 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x564de3da6850 con 0x564de3a1eda0 2024-09-21T01:52:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.965+0000 7f5989ffb640 10 client.26149 handle_command_reply: tid=0 2024-09-21T01:52:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.965+0000 7f59af334740 1 client.26149 shutdown 2024-09-21T01:52:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.965+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 msgr2=0x564de3e62fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.965+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e62fa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f599c0049d0 tx=0x7f599c010800 comp rx=0 tx=0).stop 2024-09-21T01:52:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 shutdown_connections 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564de3a1eda0 0x564de3ece750 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564de3a8b070 0x564de3e62fa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de3a7f460 0x564de3e62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 --2- 172.21.15.29:0/3252079387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564de3a621b0 0x564de3e63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 >> 172.21.15.29:0/3252079387 conn(0x564de3e2f120 msgr2=0x564de3e32250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f5989ffb640 0 client.26149 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 shutdown_connections 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 1 -- 172.21.15.29:0/3252079387 wait complete. 2024-09-21T01:52:53.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.966+0000 7f59af334740 20 client.26149 trim_cache size 0 max 16384 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d", 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:52:53.969 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.970 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:52:53.972 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:52:53.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 msgr2=0x7f598c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 0x7f598c0b8aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5998009850 tx=0x7f5998003b90 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 msgr2=0x7f59a41119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 0x7f59a41119c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f599400e9f0 tx=0x7f599400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 shutdown_connections 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f598c0b6590 0x7f598c0b8aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a4108cf0 0x7f59a41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59a41083a0 0x7f59a410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 --2- 172.21.15.29:0/4159284030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59a4102a60 0x7f59a41119c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 >> 172.21.15.29:0/4159284030 conn(0x7f59a4079730 msgr2=0x7f59a4104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 shutdown_connections 2024-09-21T01:52:53.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:52:53.967+0000 7f59a8dc0640 1 -- 172.21.15.29:0/4159284030 wait complete. 2024-09-21T01:52:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:53 smithi029 ceph-mon[31228]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-09-21T01:52:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:53 smithi190 ceph-mon[34368]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-09-21T01:52:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:52:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:53 smithi066 ceph-mon[35001]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-09-21T01:52:55.227 INFO:tasks.cephfs.filesystem:scrub status for tag:6fc36252-80c6-4cdc-b5bd-92ba0ba9585d - {'path': '/', 'tag': '6fc36252-80c6-4cdc-b5bd-92ba0ba9585d', 'options': 'recursive,force'} 2024-09-21T01:52:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:55 smithi029 ceph-mon[31228]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-09-21T01:52:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:55 smithi190 ceph-mon[34368]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-09-21T01:52:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:55 smithi066 ceph-mon[35001]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-09-21T01:52:57.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 260.0 s, 4258.8 tps, lat 23.434 ms stddev 27.230 2024-09-21T01:52:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:57 smithi029 ceph-mon[31228]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-09-21T01:52:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:57 smithi190 ceph-mon[34368]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-09-21T01:52:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:57 smithi066 ceph-mon[35001]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-09-21T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:52:59 smithi190 ceph-mon[34368]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T01:53:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:52:59 smithi066 ceph-mon[35001]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T01:53:00.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:52:59 smithi029 ceph-mon[31228]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T01:53:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 -- 172.21.15.29:0/1049493639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176806c0d0 msgr2=0x7f1768104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 --2- 172.21.15.29:0/1049493639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176806c0d0 0x7f1768104100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1758009900 tx=0x7f175802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 -- 172.21.15.29:0/1049493639 shutdown_connections 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 --2- 172.21.15.29:0/1049493639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768107200 0x7f1768107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 --2- 172.21.15.29:0/1049493639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1768104640 0x7f1768106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.504+0000 7f176d48b640 1 --2- 172.21.15.29:0/1049493639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176806c0d0 0x7f1768104100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.505+0000 7f176d48b640 1 -- 172.21.15.29:0/1049493639 >> 172.21.15.29:0/1049493639 conn(0x7f17680fdf30 msgr2=0x7f17681003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.505+0000 7f176d48b640 1 -- 172.21.15.29:0/1049493639 shutdown_connections 2024-09-21T01:53:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.505+0000 7f176d48b640 1 -- 172.21.15.29:0/1049493639 wait complete. 2024-09-21T01:53:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.505+0000 7f176d48b640 1 Processor -- start 2024-09-21T01:53:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 -- start start 2024-09-21T01:53:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 0x7f17681a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1768107200 0x7f17681a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1768115950 con 0x7f176806c0d0 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17681157d0 con 0x7f1768104640 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f176d48b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1768115ad0 con 0x7f1768107200 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f17667fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.506+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 0x7f17681a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 0x7f17681a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34832/0 (socket says 172.21.15.29:34832) 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f1766ffd640 1 -- 172.21.15.29:0/3564303979 learned_addr learned my addr 172.21.15.29:0/3564303979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40640/0 (socket says 172.21.15.29:40640) 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17677fe640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1768107200 0x7f17681a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 -- 172.21.15.29:0/3564303979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1768107200 msgr2=0x7f17681a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1768107200 0x7f17681a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 -- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 msgr2=0x7f17681a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 0x7f17681a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 -- 172.21.15.29:0/3564303979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17681a2010 con 0x7f1768104640 2024-09-21T01:53:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.507+0000 7f17667fc640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1760005fd0 tx=0x7f176000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.508+0000 7f174ffff640 1 -- 172.21.15.29:0/3564303979 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760010430 con 0x7f1768104640 2024-09-21T01:53:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.508+0000 7f174ffff640 1 -- 172.21.15.29:0/3564303979 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1760010a80 con 0x7f1768104640 2024-09-21T01:53:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.508+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17681a2300 con 0x7f1768104640 2024-09-21T01:53:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.508+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a89f0 con 0x7f1768104640 2024-09-21T01:53:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.508+0000 7f174ffff640 1 -- 172.21.15.29:0/3564303979 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760017ac0 con 0x7f1768104640 2024-09-21T01:53:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.511+0000 7f174ffff640 1 -- 172.21.15.29:0/3564303979 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1760017c60 con 0x7f1768104640 2024-09-21T01:53:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 Processor -- start 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 -- start start 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c97ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c980010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557a3c8f16f0 con 0x557a3c5d71b0 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557a3c8e0b20 con 0x557a3c5ae2f0 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17743f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a3c8e4930 con 0x557a3c5b64d0 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40650/0 (socket says 172.21.15.29:40650) 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f1766ffd640 1 -- 172.21.15.29:0/284682287 learned_addr learned my addr 172.21.15.29:0/284682287 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.512+0000 7f17677fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c97ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.513+0000 7f17667fc640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c980010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.513+0000 7f17677fe640 1 -- 172.21.15.29:0/284682287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 msgr2=0x557a3c980010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17677fe640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c980010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17677fe640 1 -- 172.21.15.29:0/284682287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 msgr2=0x557a3c97f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17677fe640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17677fe640 1 -- 172.21.15.29:0/284682287 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3c8a6530 con 0x557a3c5d71b0 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f1766ffd640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17667fc640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c980010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.514+0000 7f17677fe640 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c97ee50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f175c00aab0 tx=0x7f175c00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c690471f3a4cff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.515+0000 7f174d7fa640 1 -- 172.21.15.29:0/284682287 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c01d070 con 0x557a3c5d71b0 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.515+0000 7f174d7fa640 1 -- 172.21.15.29:0/284682287 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f175c0188f0 con 0x557a3c5d71b0 2024-09-21T01:53:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.515+0000 7f174d7fa640 1 -- 172.21.15.29:0/284682287 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c010a20 con 0x557a3c5d71b0 2024-09-21T01:53:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f174ffff640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 0x7f17500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 0x7f17500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 -- 172.21.15.29:0/284682287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 msgr2=0x557a3c97ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c97ee50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f175c00aab0 tx=0x7f175c00af70 comp rx=0 tx=0).stop 2024-09-21T01:53:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 -- 172.21.15.29:0/284682287 shutdown_connections 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c980010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 --2- 172.21.15.29:0/284682287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c97ee50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f17743f5740 1 -- 172.21.15.29:0/284682287 >> 172.21.15.29:0/284682287 conn(0x557a3c5cb300 msgr2=0x557a3c97a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.516+0000 7f174ffff640 1 -- 172.21.15.29:0/3564303979 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f17600e7a80 con 0x7f1768104640 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.517+0000 7f17743f5740 1 -- 172.21.15.29:0/284682287 shutdown_connections 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.517+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 0x7f17500b8aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1758005f70 tx=0x7f17580314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.517+0000 7f17743f5740 1 -- 172.21.15.29:0/284682287 wait complete. 2024-09-21T01:53:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.517+0000 7f17743f5740 1 -- start start 2024-09-21T01:53:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 Processor -- start 2024-09-21T01:53:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c999610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c999cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557a3c818ab0 con 0x557a3c5b64d0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557a3c886730 con 0x557a3c5d71b0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a3c980b30 con 0x557a3c5ae2f0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17743f5740 10 client.0 _ll_register_callbacks cb 0x557a3c8da6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17667fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c999cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17667fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c999cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34842/0 (socket says 172.21.15.29:34842) 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c999610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17667fc640 1 -- 172.21.15.29:0/1365371606 learned_addr learned my addr 172.21.15.29:0/1365371606 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.518+0000 7f17677fe640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.519+0000 7f17677fe640 1 -- 172.21.15.29:0/1365371606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 msgr2=0x557a3c999610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.519+0000 7f17677fe640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c999610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.519+0000 7f17677fe640 1 -- 172.21.15.29:0/1365371606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 msgr2=0x557a3c999cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.519+0000 7f17677fe640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c999cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.519+0000 7f17677fe640 1 -- 172.21.15.29:0/1365371606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3c642790 con 0x557a3c5d71b0 2024-09-21T01:53:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f17677fe640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f175c01a000 tx=0x7f175c00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=943fc0bdfa115488 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f174aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c999610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c027020 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175c0188f0 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.520+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f175c005040 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f175c01d040 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 10 client.26117 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a3c6bb620 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f175c0107a0 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f175c0046c0 con 0x557a3c5d71b0 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 10 client.26117 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 20 client.26117 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c595d80 0x557a3c9a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 4 client.26117 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.521+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f175c0188f0 con 0x557a3c595d80 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.522+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c595d80 0x557a3c9a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.523+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c595d80 0x557a3c9a69d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99dd2d04a821b4c server_cookie=ff088d2a18b96586 in_seq=0 out_seq=0 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.523+0000 7f174aff5640 10 client.26117 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f174aff5640 1 -- 172.21.15.29:0/1365371606 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f175c0188f0 con 0x557a3c595d80 2024-09-21T01:53:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f174aff5640 10 client.26117 handle_command_reply: tid=0 2024-09-21T01:53:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f17743f5740 1 client.26117 shutdown 2024-09-21T01:53:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 msgr2=0x557a3c99a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f175c01a000 tx=0x7f175c00a710 comp rx=0 tx=0).stop 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 shutdown_connections 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.524+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c595d80 0x557a3c9a69d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.525+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99a840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.525+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c999cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.525+0000 7f17743f5740 1 --2- 172.21.15.29:0/1365371606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c999610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.525+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 >> 172.21.15.29:0/1365371606 conn(0x557a3c5cb300 msgr2=0x557a3c978ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.525+0000 7f174aff5640 0 client.26117 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.526+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 shutdown_connections 2024-09-21T01:53:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.526+0000 7f17743f5740 1 -- 172.21.15.29:0/1365371606 wait complete. 2024-09-21T01:53:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.526+0000 7f17743f5740 20 client.26117 trim_cache size 0 max 16384 2024-09-21T01:53:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.552+0000 7f17743f5740 1 Processor -- start 2024-09-21T01:53:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.552+0000 7f17743f5740 1 -- start start 2024-09-21T01:53:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c9755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c998830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17667fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c9755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17667fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c9755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41994/0 (socket says 172.21.15.29:41994) 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17667fc640 1 -- 172.21.15.29:0/3723625049 learned_addr learned my addr 172.21.15.29:0/3723625049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f175400e3e0 con 0x557a3c5d71b0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1754010010 con 0x557a3c5ae2f0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f175400eab0 con 0x557a3c5b64d0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.553+0000 7f17677fe640 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c998830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 -- 172.21.15.29:0/3723625049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 msgr2=0x557a3c9755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c9755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 -- 172.21.15.29:0/3723625049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 msgr2=0x557a3c998830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c998830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 -- 172.21.15.29:0/3723625049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3c595fb0 con 0x557a3c5ae2f0 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f1766ffd640 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97fc90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1758031440 tx=0x7f175803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ca2d9769c61066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f174cff9640 1 -- 172.21.15.29:0/3723625049 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758035070 con 0x557a3c5ae2f0 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f174cff9640 1 -- 172.21.15.29:0/3723625049 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1758003ca0 con 0x557a3c5ae2f0 2024-09-21T01:53:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.554+0000 7f174cff9640 1 -- 172.21.15.29:0/3723625049 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758046ba0 con 0x557a3c5ae2f0 2024-09-21T01:53:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.555+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 msgr2=0x557a3c97fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.555+0000 7f17743f5740 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97fc90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1758031440 tx=0x7f175803f000 comp rx=0 tx=0).stop 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 shutdown_connections 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5d71b0 0x557a3c998830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5b64d0 0x557a3c9755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 --2- 172.21.15.29:0/3723625049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5ae2f0 0x557a3c97fc90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 >> 172.21.15.29:0/3723625049 conn(0x557a3c5cb300 msgr2=0x557a3c998060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 shutdown_connections 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.556+0000 7f17743f5740 1 -- 172.21.15.29:0/3723625049 wait complete. 2024-09-21T01:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.557+0000 7f17743f5740 1 -- start start 2024-09-21T01:53:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.557+0000 7f17743f5740 1 Processor -- start 2024-09-21T01:53:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c99ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c99d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f175400f940 con 0x557a3c5b64d0 2024-09-21T01:53:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40684/0 (socket says 172.21.15.29:40684) 2024-09-21T01:53:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17677fe640 1 -- 172.21.15.29:0/1227250072 learned_addr learned my addr 172.21.15.29:0/1227250072 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f175400f240 con 0x557a3c5d71b0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.558+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f175400c3c0 con 0x557a3c5ae2f0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17743f5740 10 client.0 _ll_register_callbacks cb 0x7f175c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c99ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17667fc640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c99d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 -- 172.21.15.29:0/1227250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 msgr2=0x557a3c99ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c99ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 -- 172.21.15.29:0/1227250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 msgr2=0x557a3c99d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c99d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3c99e3b0 con 0x557a3c5d71b0 2024-09-21T01:53:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17667fc640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c99d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.559+0000 7f17677fe640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f175c004b10 tx=0x7f175c00aa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7348ea5c23029117 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.560+0000 7f174aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.563+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c018b60 con 0x557a3c5d71b0 2024-09-21T01:53:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.563+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1758003ca0 con 0x557a3c5d71b0 2024-09-21T01:53:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.563+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f175c018d00 con 0x557a3c5d71b0 2024-09-21T01:53:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.563+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f175c004d00 con 0x557a3c5d71b0 2024-09-21T01:53:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.564+0000 7f17743f5740 10 client.26123 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.564+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a3c9a16c0 con 0x557a3c5d71b0 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1758003ca0 con 0x557a3c5d71b0 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f175c019320 con 0x557a3c5d71b0 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f17743f5740 10 client.26123 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f17743f5740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f17743f5740 20 client.26123 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.569+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c56b1c0 0x557a3ca13750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.570+0000 7f17743f5740 4 client.26123 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:53:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.570+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a3c8f3640 con 0x557a3c56b1c0 2024-09-21T01:53:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.570+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c56b1c0 0x557a3ca13750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.571+0000 7f1766ffd640 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c56b1c0 0x557a3ca13750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7183bc0f3986580 server_cookie=db204e2c5f5acf9f in_seq=0 out_seq=0 2024-09-21T01:53:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.572+0000 7f174aff5640 10 client.26123 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.623+0000 7f174aff5640 1 -- 172.21.15.29:0/1227250072 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a3c8f3640 con 0x557a3c56b1c0 2024-09-21T01:53:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.623+0000 7f174aff5640 10 client.26123 handle_command_reply: tid=0 2024-09-21T01:53:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.623+0000 7f17743f5740 1 client.26123 shutdown 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 msgr2=0x557a3c99dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f175c004b10 tx=0x7f175c00aa70 comp rx=0 tx=0).stop 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 shutdown_connections 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557a3c56b1c0 0x557a3ca13750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a3c5d71b0 0x557a3c99dc50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557a3c5b64d0 0x557a3c99d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 --2- 172.21.15.29:0/1227250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a3c5ae2f0 0x557a3c99ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 >> 172.21.15.29:0/1227250072 conn(0x557a3c5cb300 msgr2=0x557a3c9755c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f174aff5640 0 client.26123 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 shutdown_connections 2024-09-21T01:53:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 1 -- 172.21.15.29:0/1227250072 wait complete. 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.624+0000 7f17743f5740 20 client.26123 trim_cache size 0 max 16384 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:53:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 msgr2=0x7f17500b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 0x7f17500b8aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1758005f70 tx=0x7f17580314a0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 msgr2=0x7f17681a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1760005fd0 tx=0x7f176000ef90 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 shutdown_connections 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f17500b6590 0x7f17500b8aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1768107200 0x7f17681a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1768104640 0x7f17681a0c40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 --2- 172.21.15.29:0/3564303979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f176806c0d0 0x7f17681a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 >> 172.21.15.29:0/3564303979 conn(0x7f17680fdf30 msgr2=0x7f17681055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 shutdown_connections 2024-09-21T01:53:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:01.626+0000 7f176d48b640 1 -- 172.21.15.29:0/3564303979 wait complete. 2024-09-21T01:53:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:01 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:01 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:01 smithi190 ceph-mon[34368]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-09-21T01:53:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:01 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:01 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:01 smithi066 ceph-mon[35001]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-09-21T01:53:02.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:01 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:02.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:01 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:02.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:01 smithi029 ceph-mon[31228]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-09-21T01:53:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 265.0 s, 4095.0 tps, lat 24.495 ms stddev 27.354 2024-09-21T01:53:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:03 smithi190 ceph-mon[34368]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.5 MiB/s wr, 855 op/s 2024-09-21T01:53:04.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:03 smithi066 ceph-mon[35001]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.5 MiB/s wr, 855 op/s 2024-09-21T01:53:04.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:03 smithi029 ceph-mon[31228]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.5 MiB/s wr, 855 op/s 2024-09-21T01:53:06.146 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:05 smithi029 ceph-mon[31228]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-09-21T01:53:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:05 smithi190 ceph-mon[34368]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-09-21T01:53:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:05 smithi066 ceph-mon[35001]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-09-21T01:53:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 270.0 s, 2853.0 tps, lat 34.804 ms stddev 39.365 2024-09-21T01:53:08.144 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:07 smithi029 ceph-mon[31228]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T01:53:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:07 smithi190 ceph-mon[34368]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T01:53:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:07 smithi066 ceph-mon[35001]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T01:53:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:09 smithi190 ceph-mon[34368]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 674 op/s 2024-09-21T01:53:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:09 smithi066 ceph-mon[35001]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 674 op/s 2024-09-21T01:53:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:09 smithi029 ceph-mon[31228]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 674 op/s 2024-09-21T01:53:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:11 smithi190 ceph-mon[34368]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 999 op/s 2024-09-21T01:53:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:11 smithi066 ceph-mon[35001]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 999 op/s 2024-09-21T01:53:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:11 smithi029 ceph-mon[31228]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 999 op/s 2024-09-21T01:53:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 275.0 s, 2958.4 tps, lat 33.790 ms stddev 39.104 2024-09-21T01:53:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:13 smithi190 ceph-mon[34368]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 834 op/s 2024-09-21T01:53:14.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:13 smithi066 ceph-mon[35001]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 834 op/s 2024-09-21T01:53:14.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:13 smithi029 ceph-mon[31228]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 834 op/s 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.784+0000 7fb4afbda640 1 -- 172.21.15.29:0/3954801275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a8103fc0 msgr2=0x7fb4a810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.784+0000 7fb4afbda640 1 --2- 172.21.15.29:0/3954801275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a8103fc0 0x7fb4a810a960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009910 tx=0x7fb4a402a3e0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 -- 172.21.15.29:0/3954801275 shutdown_connections 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 --2- 172.21.15.29:0/3954801275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a8103fc0 0x7fb4a810a960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 --2- 172.21.15.29:0/3954801275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103620 0x7fb4a8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 --2- 172.21.15.29:0/3954801275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a81025a0 0x7fb4a8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 -- 172.21.15.29:0/3954801275 >> 172.21.15.29:0/3954801275 conn(0x7fb4a80fdee0 msgr2=0x7fb4a8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 -- 172.21.15.29:0/3954801275 shutdown_connections 2024-09-21T01:53:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.785+0000 7fb4afbda640 1 -- 172.21.15.29:0/3954801275 wait complete. 2024-09-21T01:53:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.786+0000 7fb4afbda640 1 Processor -- start 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.786+0000 7fb4afbda640 1 -- start start 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4afbda640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4afbda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a8103620 0x7fb4a819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4afbda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 0x7fb4a819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4afbda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e240 con 0x7fb4a8103620 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4afbda640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e0c0 con 0x7fb4a81025a0 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4ae177640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 0x7fb4a819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40928/0 (socket says 172.21.15.29:40928) 2024-09-21T01:53:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.788+0000 7fb4ad976640 1 -- 172.21.15.29:0/2747970282 learned_addr learned my addr 172.21.15.29:0/2747970282 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.787+0000 7fb4ae177640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 0x7fb4a819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34482/0 (socket says 172.21.15.29:34482) 2024-09-21T01:53:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.788+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e3c0 con 0x7fb4a8103fc0 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 -- 172.21.15.29:0/2747970282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 msgr2=0x7fb4a819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 0x7fb4a819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 -- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a8103620 msgr2=0x7fb4a819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a8103620 0x7fb4a819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 -- 172.21.15.29:0/2747970282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a819b940 con 0x7fb4a81025a0 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4a0 tx=0x7fb49800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.789+0000 7fb49effd640 1 -- 172.21.15.29:0/2747970282 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49801d070 con 0x7fb4a81025a0 2024-09-21T01:53:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.790+0000 7fb49effd640 1 -- 172.21.15.29:0/2747970282 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb498018440 con 0x7fb4a81025a0 2024-09-21T01:53:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.790+0000 7fb49effd640 1 -- 172.21.15.29:0/2747970282 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498016580 con 0x7fb4a81025a0 2024-09-21T01:53:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.790+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a819bc30 con 0x7fb4a81025a0 2024-09-21T01:53:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.790+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a8104930 con 0x7fb4a81025a0 2024-09-21T01:53:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 Processor -- start 2024-09-21T01:53:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 -- start start 2024-09-21T01:53:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e154ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e154d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5593e14bf6f0 con 0x5593e11844d0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5593e14aeb20 con 0x5593e11a51b0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.794+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5593e14b2930 con 0x5593e117c2f0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb49effd640 1 -- 172.21.15.29:0/2747970282 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb498020070 con 0x7fb4a81025a0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42490/0 (socket says 172.21.15.29:42490) 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 -- 172.21.15.29:0/2785503010 learned_addr learned my addr 172.21.15.29:0/2785503010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ae177640 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e154ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 -- 172.21.15.29:0/2785503010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 msgr2=0x5593e154d4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:53:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e154d4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 -- 172.21.15.29:0/2785503010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 msgr2=0x5593e154ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e154ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.795+0000 7fb4ad175640 1 -- 172.21.15.29:0/2785503010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e1474530 con 0x5593e11844d0 2024-09-21T01:53:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.796+0000 7fb4ad175640 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0007d80 tx=0x7fb4a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae43460acc63b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.796+0000 7fb493fff640 1 -- 172.21.15.29:0/2785503010 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0014070 con 0x5593e11844d0 2024-09-21T01:53:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.797+0000 7fb493fff640 1 -- 172.21.15.29:0/2785503010 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a000fb00 con 0x5593e11844d0 2024-09-21T01:53:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.797+0000 7fb49effd640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 0x7fb4940b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.797+0000 7fb49effd640 1 -- 172.21.15.29:0/2747970282 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb4980ea190 con 0x7fb4a81025a0 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.797+0000 7fb4ae177640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 0x7fb4940b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.797+0000 7fb493fff640 1 -- 172.21.15.29:0/2785503010 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0014070 con 0x5593e11844d0 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2785503010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 msgr2=0x5593e154e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0007d80 tx=0x7fb4a000c6a0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4ae177640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 0x7fb4940b8b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4000e60 tx=0x7fb4a4031500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2785503010 shutdown_connections 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e154e010 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e154d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2785503010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e154ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.798+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2785503010 >> 172.21.15.29:0/2785503010 conn(0x5593e1199300 msgr2=0x5593e15488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.799+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2785503010 shutdown_connections 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.799+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2785503010 wait complete. 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.799+0000 7fb4b6c20740 1 -- start start 2024-09-21T01:53:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.800+0000 7fb4b6c20740 1 Processor -- start 2024-09-21T01:53:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.800+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 0x5593e15565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11844d0 0x5593e1556cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e1557820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5593e13e6ab0 con 0x5593e117c2f0 2024-09-21T01:53:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5593e1454730 con 0x5593e11a51b0 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5593e154eb80 con 0x5593e11844d0 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4b6c20740 10 client.0 _ll_register_callbacks cb 0x5593e14a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 0x5593e15565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e1557820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 0x5593e15565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42506/0 (socket says 172.21.15.29:42506) 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ad976640 1 -- 172.21.15.29:0/3940791880 learned_addr learned my addr 172.21.15.29:0/3940791880 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ad175640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11844d0 0x5593e1556cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 -- 172.21.15.29:0/3940791880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11844d0 msgr2=0x5593e1556cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11844d0 0x5593e1556cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 -- 172.21.15.29:0/3940791880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 msgr2=0x5593e15565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 0x5593e15565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.801+0000 7fb4ae177640 1 -- 172.21.15.29:0/3940791880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e1210790 con 0x5593e11a51b0 2024-09-21T01:53:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.802+0000 7fb4ae177640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e1557820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402a900 tx=0x7fb4a402a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7912503b5e93d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.802+0000 7fb4917fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.802+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a404f070 con 0x5593e11a51b0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.803+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb4a4005450 con 0x5593e11a51b0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.803+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a000fb00 con 0x5593e11a51b0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.804+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4a4036040 con 0x5593e11a51b0 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.804+0000 7fb4b6c20740 10 client.26135 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.804+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5593e1289620 con 0x5593e11a51b0 2024-09-21T01:53:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.804+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a000fb00 con 0x5593e11a51b0 2024-09-21T01:53:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb4a4004820 con 0x5593e11a51b0 2024-09-21T01:53:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 10 client.26135 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 20 client.26135 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e1163d80 0x5593e1563d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 4 client.26135 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.805+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593e14c1640 con 0x5593e1163d80 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.806+0000 7fb4ad976640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e1163d80 0x5593e1563d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.806+0000 7fb4917fa640 10 client.26135 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.807+0000 7fb4ad976640 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e1163d80 0x5593e1563d60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e994a6fc166efa93 server_cookie=cde0b4abff6ba163 in_seq=0 out_seq=0 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.808+0000 7fb4917fa640 1 -- 172.21.15.29:0/3940791880 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593e14c1640 con 0x5593e1163d80 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.808+0000 7fb4917fa640 10 client.26135 handle_command_reply: tid=0 2024-09-21T01:53:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.808+0000 7fb4b6c20740 1 client.26135 shutdown 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 msgr2=0x5593e1557820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e1557820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402a900 tx=0x7fb4a402a3e0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 shutdown_connections 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e1163d80 0x5593e1563d60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11a51b0 0x5593e1557820 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11844d0 0x5593e1556cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/3940791880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e117c2f0 0x5593e15565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 >> 172.21.15.29:0/3940791880 conn(0x5593e1199300 msgr2=0x5593e1546ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4917fa640 0 client.26135 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 shutdown_connections 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.809+0000 7fb4b6c20740 1 -- 172.21.15.29:0/3940791880 wait complete. 2024-09-21T01:53:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.810+0000 7fb4b6c20740 20 client.26135 trim_cache size 0 max 16384 2024-09-21T01:53:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.840+0000 7fb4b6c20740 1 Processor -- start 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.840+0000 7fb4b6c20740 1 -- start start 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e15490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 0x5593e15495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81a8ce0 con 0x5593e11844d0 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aa910 con 0x5593e117c2f0 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81a93b0 con 0x5593e11a51b0 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40956/0 (socket says 172.21.15.29:40956) 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 -- 172.21.15.29:0/2461291454 learned_addr learned my addr 172.21.15.29:0/2461291454 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad175640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e15490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ae177640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 0x5593e15495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 -- 172.21.15.29:0/2461291454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 msgr2=0x5593e15495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 0x5593e15495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 -- 172.21.15.29:0/2461291454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 msgr2=0x5593e15490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e15490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ad976640 1 -- 172.21.15.29:0/2461291454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e154e470 con 0x5593e117c2f0 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.841+0000 7fb4ae177640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 0x5593e15495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.842+0000 7fb4ad976640 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4980ac9b0 tx=0x7fb498012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf46175c2839e7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.842+0000 7fb4937fe640 1 -- 172.21.15.29:0/2461291454 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980b9d50 con 0x5593e117c2f0 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.842+0000 7fb4937fe640 1 -- 172.21.15.29:0/2461291454 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4980cb070 con 0x5593e117c2f0 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.842+0000 7fb4937fe640 1 -- 172.21.15.29:0/2461291454 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980af640 con 0x5593e117c2f0 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2461291454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 msgr2=0x5593e1555810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4980ac9b0 tx=0x7fb498012040 comp rx=0 tx=0).stop 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2461291454 shutdown_connections 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e11a51b0 0x5593e15495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11844d0 0x5593e15490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/2461291454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e117c2f0 0x5593e1555810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2461291454 >> 172.21.15.29:0/2461291454 conn(0x5593e1199300 msgr2=0x5593e154dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2461291454 shutdown_connections 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- 172.21.15.29:0/2461291454 wait complete. 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 -- start start 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 Processor -- start 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.843+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11844d0 0x5593e155ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11a51b0 0x5593e155d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aa240 con 0x5593e11a51b0 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81a9b40 con 0x5593e11844d0 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81a6cc0 con 0x5593e117c2f0 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4b6c20740 10 client.0 _ll_register_callbacks cb 0x7fb4a4042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34508/0 (socket says 172.21.15.29:34508) 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 -- 172.21.15.29:0/1678198291 learned_addr learned my addr 172.21.15.29:0/1678198291 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ae177640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11a51b0 0x5593e155d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad175640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11844d0 0x5593e155ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 -- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11844d0 msgr2=0x5593e155ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11844d0 0x5593e155ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 -- 172.21.15.29:0/1678198291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11a51b0 msgr2=0x5593e155d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11a51b0 0x5593e155d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 -- 172.21.15.29:0/1678198291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e155db00 con 0x5593e117c2f0 2024-09-21T01:53:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4ad976640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4980e7ca0 tx=0x7fb4980186e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14897048442fe52d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4917fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.844+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498010d10 con 0x5593e117c2f0 2024-09-21T01:53:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb4980af640 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4980cb070 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4980af840 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4b6c20740 10 client.26173 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5593e15ce710 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.845+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4980cb070 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb4980af0b0 con 0x5593e117c2f0 2024-09-21T01:53:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 10 client.26173 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 10 client.26173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 20 client.26173 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e11391c0 0x5593e15d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 4 client.26173 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593e14bd0f0 con 0x5593e11391c0 2024-09-21T01:53:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4ad175640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e11391c0 0x5593e15d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4ad175640 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e11391c0 0x5593e15d1000 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbcfa2ea0a93074 server_cookie=e2f70acd05f3c134 in_seq=0 out_seq=0 2024-09-21T01:53:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.846+0000 7fb4917fa640 10 client.26173 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.911+0000 7fb4917fa640 1 -- 172.21.15.29:0/1678198291 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593e14bd0f0 con 0x5593e11391c0 2024-09-21T01:53:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.911+0000 7fb4917fa640 10 client.26173 handle_command_reply: tid=0 2024-09-21T01:53:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.911+0000 7fb4b6c20740 1 client.26173 shutdown 2024-09-21T01:53:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 msgr2=0x5593e155e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4980e7ca0 tx=0x7fb4980186e0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 shutdown_connections 2024-09-21T01:53:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5593e11391c0 0x5593e15d1000 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5593e11a51b0 0x5593e155d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593e11844d0 0x5593e155ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 --2- 172.21.15.29:0/1678198291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593e117c2f0 0x5593e155e1f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 >> 172.21.15.29:0/1678198291 conn(0x5593e1199300 msgr2=0x5593e154dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4917fa640 0 client.26173 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 shutdown_connections 2024-09-21T01:53:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 1 -- 172.21.15.29:0/1678198291 wait complete. 2024-09-21T01:53:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.912+0000 7fb4b6c20740 20 client.26173 trim_cache size 0 max 16384 2024-09-21T01:53:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:53:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:53:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:53:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:53:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 msgr2=0x7fb4940b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 0x7fb4940b8b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4000e60 tx=0x7fb4a4031500 comp rx=0 tx=0).stop 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 msgr2=0x7fb4a8199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4a0 tx=0x7fb49800b960 comp rx=0 tx=0).stop 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 shutdown_connections 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb4940b6660 0x7fb4940b8b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103fc0 0x7fb4a819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a8103620 0x7fb4a819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 --2- 172.21.15.29:0/2747970282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4a81025a0 0x7fb4a8199f90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 >> 172.21.15.29:0/2747970282 conn(0x7fb4a80fdee0 msgr2=0x7fb4a80ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 shutdown_connections 2024-09-21T01:53:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:14.913+0000 7fb4afbda640 1 -- 172.21.15.29:0/2747970282 wait complete. 2024-09-21T01:53:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:15 smithi190 ceph-mon[34368]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 849 op/s 2024-09-21T01:53:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:15 smithi066 ceph-mon[35001]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 849 op/s 2024-09-21T01:53:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:15 smithi029 ceph-mon[31228]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 849 op/s 2024-09-21T01:53:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 280.0 s, 2873.2 tps, lat 35.044 ms stddev 41.965 2024-09-21T01:53:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:17 smithi190 ceph-mon[34368]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 984 op/s 2024-09-21T01:53:18.246 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:17 smithi029 ceph-mon[31228]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 984 op/s 2024-09-21T01:53:18.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:17 smithi066 ceph-mon[35001]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 984 op/s 2024-09-21T01:53:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:19 smithi190 ceph-mon[34368]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 763 op/s 2024-09-21T01:53:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:19 smithi066 ceph-mon[35001]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 763 op/s 2024-09-21T01:53:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:19 smithi029 ceph-mon[31228]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 763 op/s 2024-09-21T01:53:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:21 smithi190 ceph-mon[34368]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 979 op/s 2024-09-21T01:53:22.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:21 smithi066 ceph-mon[35001]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 979 op/s 2024-09-21T01:53:22.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:21 smithi029 ceph-mon[31228]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 979 op/s 2024-09-21T01:53:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 285.0 s, 2610.8 tps, lat 38.230 ms stddev 43.606 2024-09-21T01:53:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:23 smithi190 ceph-mon[34368]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 744 op/s 2024-09-21T01:53:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:23 smithi066 ceph-mon[35001]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 744 op/s 2024-09-21T01:53:24.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:23 smithi029 ceph-mon[31228]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 744 op/s 2024-09-21T01:53:25.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:53:25.637 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:26.090 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:25 smithi029 ceph-mon[31228]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 747 op/s 2024-09-21T01:53:26.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.223+0000 7f9fc6cea640 1 -- 172.21.15.29:0/1165184788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0103bb0 msgr2=0x7f9fc0103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.224+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/1165184788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0103bb0 0x7f9fc0103fc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009900 tx=0x7f9fb402a410 comp rx=0 tx=0).stop 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.224+0000 7f9fc6cea640 1 -- 172.21.15.29:0/1165184788 shutdown_connections 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.224+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/1165184788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0109e40 0x7f9fc0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.225+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/1165184788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.225+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/1165184788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0103bb0 0x7f9fc0103fc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.225+0000 7f9fc6cea640 1 -- 172.21.15.29:0/1165184788 >> 172.21.15.29:0/1165184788 conn(0x7f9fc00ff600 msgr2=0x7f9fc0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.225+0000 7f9fc6cea640 1 -- 172.21.15.29:0/1165184788 shutdown_connections 2024-09-21T01:53:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.225+0000 7f9fc6cea640 1 -- 172.21.15.29:0/1165184788 wait complete. 2024-09-21T01:53:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.226+0000 7f9fc6cea640 1 Processor -- start 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.226+0000 7f9fc6cea640 1 -- start start 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 0x7f9fc019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 0x7f9fc01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0113b00 con 0x7f9fc0109e40 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0113980 con 0x7f9fc0103bb0 2024-09-21T01:53:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc6cea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0113c80 con 0x7f9fc01094f0 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 0x7f9fc01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc4a86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 0x7f9fc019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 0x7f9fc01a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37070/0 (socket says 172.21.15.29:37070) 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc4a86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 0x7f9fc019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40802/0 (socket says 172.21.15.29:40802) 2024-09-21T01:53:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fc5287640 1 -- 172.21.15.29:0/817849047 learned_addr learned my addr 172.21.15.29:0/817849047 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:25 smithi190 ceph-mon[34368]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 747 op/s 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 -- 172.21.15.29:0/817849047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 msgr2=0x7f9fc019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 0x7f9fc019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 -- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 msgr2=0x7f9fc01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 0x7f9fc01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.227+0000 7f9fbffff640 1 -- 172.21.15.29:0/817849047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc01a2660 con 0x7f9fc01094f0 2024-09-21T01:53:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fbffff640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc019cc90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9fc006cbc0 tx=0x7f9fac00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fbdffb640 1 -- 172.21.15.29:0/817849047 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00cc40 con 0x7f9fc01094f0 2024-09-21T01:53:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc01a2950 con 0x7f9fc01094f0 2024-09-21T01:53:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fbdffb640 1 -- 172.21.15.29:0/817849047 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fac00cde0 con 0x7f9fc01094f0 2024-09-21T01:53:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fbdffb640 1 -- 172.21.15.29:0/817849047 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004a00 con 0x7f9fc01094f0 2024-09-21T01:53:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.228+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a90b0 con 0x7f9fc01094f0 2024-09-21T01:53:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.231+0000 7f9fcd25e740 1 Processor -- start 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 -- start start 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fbdffb640 1 -- 172.21.15.29:0/817849047 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9fac004ba0 con 0x7f9fc01094f0 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560da863ae50 con 0x560da82fc390 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560da862a280 con 0x560da82f41b0 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560da862e090 con 0x560da831d070 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fc4a86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40720/0 (socket says 172.21.15.29:40720) 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.232+0000 7f9fc5287640 1 -- 172.21.15.29:0/3812359385 learned_addr learned my addr 172.21.15.29:0/3812359385 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3812359385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 msgr2=0x560da86cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3812359385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 msgr2=0x560da86cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3812359385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da8388650 con 0x560da82f41b0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86ccfc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009230 tx=0x7f9fb4032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f81de68505a8826 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9fbdffb640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 0x7f9fb00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9faaffd640 1 -- 172.21.15.29:0/3812359385 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4040070 con 0x560da82f41b0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.233+0000 7f9faaffd640 1 -- 172.21.15.29:0/3812359385 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb4004030 con 0x560da82f41b0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fc5287640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 0x7f9fb00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fbdffb640 1 -- 172.21.15.29:0/817849047 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9fac0e7240 con 0x7f9fc01094f0 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9faaffd640 1 -- 172.21.15.29:0/3812359385 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4035040 con 0x560da82f41b0 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fc5287640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 0x7f9fb00b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3812359385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 msgr2=0x560da86ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86ccfc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009230 tx=0x7f9fb4032b90 comp rx=0 tx=0).stop 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3812359385 shutdown_connections 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86ccfc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3812359385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3812359385 >> 172.21.15.29:0/3812359385 conn(0x560da8311460 msgr2=0x560da86ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.234+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3812359385 shutdown_connections 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3812359385 wait complete. 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 -- start start 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 Processor -- start 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.235+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560da848b700 con 0x560da82fc390 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560da85c5110 con 0x560da82f41b0 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560da86db250 con 0x560da831d070 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fcd25e740 10 client.0 _ll_register_callbacks cb 0x560da85de990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40832/0 (socket says 172.21.15.29:40832) 2024-09-21T01:53:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3482206113 learned_addr learned my addr 172.21.15.29:0/3482206113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc5287640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3482206113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 msgr2=0x560da86d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3482206113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 msgr2=0x560da86d85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86d85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.236+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3482206113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da85f2c40 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.237+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4032fb0 tx=0x7f9fb402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433c68b3d90b69f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.238+0000 7f9f9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.238+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4052060 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.238+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9fb40314a0 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.238+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4004030 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.238+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9fb4035070 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.239+0000 7f9fcd25e740 10 client.26153 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.239+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560da86e6ed0 con 0x560da82f41b0 2024-09-21T01:53:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.239+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb4004030 con 0x560da82f41b0 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.239+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f9fb4004c90 con 0x560da82f41b0 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 10 client.26153 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 20 client.26153 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da82db9f0 0x560da86ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 4 client.26153 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da863cda0 con 0x560da82db9f0 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fbffff640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da82db9f0 0x560da86ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9fbffff640 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da82db9f0 0x560da86ea230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403487577071ec9a server_cookie=4f65de19141a5644 in_seq=0 out_seq=0 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.240+0000 7f9f9ffff640 10 client.26153 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.241+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3482206113 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560da863cda0 con 0x560da82db9f0 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.241+0000 7f9f9ffff640 10 client.26153 handle_command_reply: tid=0 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.241+0000 7f9fcd25e740 1 client.26153 shutdown 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.241+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 msgr2=0x560da86d8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.241+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4032fb0 tx=0x7f9fb402a410 comp rx=0 tx=0).stop 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 shutdown_connections 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da82db9f0 0x560da86ea230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da831d070 0x560da86d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9f9ffff640 0 client.26153 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82fc390 0x560da86d85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3482206113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d8060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 >> 172.21.15.29:0/3482206113 conn(0x560da8311460 msgr2=0x560da86cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 shutdown_connections 2024-09-21T01:53:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.242+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3482206113 wait complete. 2024-09-21T01:53:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.243+0000 7f9fcd25e740 20 client.26153 trim_cache size 0 max 16384 2024-09-21T01:53:26.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.266+0000 7f9fcd25e740 1 Processor -- start 2024-09-21T01:53:26.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.267+0000 7f9fcd25e740 1 -- start start 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.269+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 0x560da86d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.269+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.269+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da86d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ad1b0 con 0x560da82db9f0 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01aede0 con 0x560da82f41b0 2024-09-21T01:53:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ad880 con 0x560da82fc390 2024-09-21T01:53:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da86d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fc5287640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da86d5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40730/0 (socket says 172.21.15.29:40730) 2024-09-21T01:53:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.270+0000 7f9fc5287640 1 -- 172.21.15.29:0/133870591 learned_addr learned my addr 172.21.15.29:0/133870591 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 0x560da86d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 -- 172.21.15.29:0/133870591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 msgr2=0x560da86d5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da86d5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 -- 172.21.15.29:0/133870591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 msgr2=0x560da86d9c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:53:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d9c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.271+0000 7f9fc4a86640 1 -- 172.21.15.29:0/133870591 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da86d6fc0 con 0x560da82db9f0 2024-09-21T01:53:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.273+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 0x560da86d2900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a930 tx=0x7f9fb4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269eab415b806eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.274+0000 7f9faa7fc640 1 -- 172.21.15.29:0/133870591 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb405c070 con 0x560da82db9f0 2024-09-21T01:53:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.274+0000 7f9faa7fc640 1 -- 172.21.15.29:0/133870591 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb4031cc0 con 0x560da82db9f0 2024-09-21T01:53:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.274+0000 7f9faa7fc640 1 -- 172.21.15.29:0/133870591 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4040040 con 0x560da82db9f0 2024-09-21T01:53:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.276+0000 7f9fcd25e740 1 -- 172.21.15.29:0/133870591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 msgr2=0x560da86d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.276+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 0x560da86d2900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a930 tx=0x7f9fb4004600 comp rx=0 tx=0).stop 2024-09-21T01:53:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.284+0000 7f9fcd25e740 1 -- 172.21.15.29:0/133870591 shutdown_connections 2024-09-21T01:53:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.284+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da86d5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.284+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82f41b0 0x560da86d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.284+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/133870591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82db9f0 0x560da86d2900 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.284+0000 7f9fcd25e740 1 -- 172.21.15.29:0/133870591 >> 172.21.15.29:0/133870591 conn(0x560da8311460 msgr2=0x560da86d9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.287+0000 7f9fcd25e740 1 -- 172.21.15.29:0/133870591 shutdown_connections 2024-09-21T01:53:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.288+0000 7f9fcd25e740 1 -- 172.21.15.29:0/133870591 wait complete. 2024-09-21T01:53:26.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.289+0000 7f9fcd25e740 1 -- start start 2024-09-21T01:53:26.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.289+0000 7f9fcd25e740 1 Processor -- start 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 0x560da86eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 0x560da86eba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da87539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ae710 con 0x560da82f41b0 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ae010 con 0x560da82db9f0 2024-09-21T01:53:26.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ab230 con 0x560da82fc390 2024-09-21T01:53:26.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.293+0000 7f9fcd25e740 10 client.0 _ll_register_callbacks cb 0x7f9fb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:26.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.295+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 0x560da86eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.295+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 0x560da86eba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37114/0 (socket says 172.21.15.29:37114) 2024-09-21T01:53:26.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.295+0000 7f9fbffff640 1 -- 172.21.15.29:0/3245626442 learned_addr learned my addr 172.21.15.29:0/3245626442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:26.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.295+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 0x560da86eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.297+0000 7f9fc5287640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da87539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:25 smithi066 ceph-mon[35001]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 747 op/s 2024-09-21T01:53:26.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.300+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3245626442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 msgr2=0x560da87539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.300+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da87539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.300+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3245626442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 msgr2=0x560da86eba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:26.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.300+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 0x560da86eba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.300+0000 7f9fc4a86640 1 -- 172.21.15.29:0/3245626442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da86ec280 con 0x560da82db9f0 2024-09-21T01:53:26.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.302+0000 7f9fc4a86640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 0x560da86eb530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a930 tx=0x7f9fb4005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdccc672c6c1582f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:26.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.303+0000 7f9f9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:26.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.304+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4064020 con 0x560da82db9f0 2024-09-21T01:53:26.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.304+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4031cc0 con 0x560da82db9f0 2024-09-21T01:53:26.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.304+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9fb40314a0 con 0x560da82db9f0 2024-09-21T01:53:26.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.305+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9fb4040070 con 0x560da82db9f0 2024-09-21T01:53:26.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.307+0000 7f9fcd25e740 10 client.26156 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:26.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.307+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560da8765ed0 con 0x560da82db9f0 2024-09-21T01:53:26.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb4031cc0 con 0x560da82db9f0 2024-09-21T01:53:26.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f9fb40048a0 con 0x560da82db9f0 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 10 client.26156 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 10 client.26156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 20 client.26156 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da831d070 0x560da87683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 4 client.26156 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:53:26.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.308+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560da8638850 con 0x560da831d070 2024-09-21T01:53:26.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.310+0000 7f9fbffff640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da831d070 0x560da87683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:26.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.311+0000 7f9fbffff640 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da831d070 0x560da87683c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a707554b08dccf0 server_cookie=7cf80b840d99ea3c in_seq=0 out_seq=0 2024-09-21T01:53:26.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.311+0000 7f9f9ffff640 10 client.26156 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.314+0000 7f9f9ffff640 1 -- 172.21.15.29:0/3245626442 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x560da8638850 con 0x560da831d070 2024-09-21T01:53:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.314+0000 7f9f9ffff640 10 client.26156 handle_command_reply: tid=0 2024-09-21T01:53:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.314+0000 7f9fcd25e740 1 client.26156 shutdown 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.315+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 msgr2=0x560da86eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.315+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 0x560da86eb530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a930 tx=0x7f9fb4005ed0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 shutdown_connections 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560da831d070 0x560da87683c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560da82fc390 0x560da87539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560da82f41b0 0x560da86eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 --2- 172.21.15.29:0/3245626442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560da82db9f0 0x560da86eb530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 >> 172.21.15.29:0/3245626442 conn(0x560da8311460 msgr2=0x560da86ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9f9ffff640 0 client.26156 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 shutdown_connections 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.316+0000 7f9fcd25e740 1 -- 172.21.15.29:0/3245626442 wait complete. 2024-09-21T01:53:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.317+0000 7f9fcd25e740 20 client.26156 trim_cache size 0 max 16384 2024-09-21T01:53:26.318 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:53:26.318 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:53:26.318 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:53:26.318 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.320 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:53:26.321 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:53:26.322 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 msgr2=0x7f9fb00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 0x7f9fb00b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8009500 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 msgr2=0x7f9fc019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc019cc90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9fc006cbc0 tx=0x7f9fac00ee50 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 shutdown_connections 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9fb00b6510 0x7f9fb00b8a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc0109e40 0x7f9fc01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc01094f0 0x7f9fc019cc90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 --2- 172.21.15.29:0/817849047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc0103bb0 0x7f9fc019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.318+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 >> 172.21.15.29:0/817849047 conn(0x7f9fc00ff600 msgr2=0x7f9fc010e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.319+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 shutdown_connections 2024-09-21T01:53:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:26.319+0000 7f9fc6cea640 1 -- 172.21.15.29:0/817849047 wait complete. 2024-09-21T01:53:27.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:53:27.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:53:27.718 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 290.0 s, 2627.2 tps, lat 37.979 ms stddev 44.520 2024-09-21T01:53:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 -- 172.21.15.29:0/2636655060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c094c60 msgr2=0x7f376c0950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 --2- 172.21.15.29:0/2636655060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c094c60 0x7f376c0950d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f377000dc90 tx=0x7f377002e390 comp rx=0 tx=0).stop 2024-09-21T01:53:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 -- 172.21.15.29:0/2636655060 shutdown_connections 2024-09-21T01:53:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 --2- 172.21.15.29:0/2636655060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c096680 0x7f376c09d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 --2- 172.21.15.29:0/2636655060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c0960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 --2- 172.21.15.29:0/2636655060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c094c60 0x7f376c0950d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.070+0000 7f377af10640 1 -- 172.21.15.29:0/2636655060 >> 172.21.15.29:0/2636655060 conn(0x7f376c0905a0 msgr2=0x7f376c092a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- 172.21.15.29:0/2636655060 shutdown_connections 2024-09-21T01:53:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- 172.21.15.29:0/2636655060 wait complete. 2024-09-21T01:53:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 Processor -- start 2024-09-21T01:53:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- start start 2024-09-21T01:53:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 0x7f376c12a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c12aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 0x7f376c12b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 0x7f376c12a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 0x7f376c12a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40856/0 (socket says 172.21.15.29:40856) 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377970d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c12aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377970d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c12aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40760/0 (socket says 172.21.15.29:40760) 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f3779f0e640 1 -- 172.21.15.29:0/716204528 learned_addr learned my addr 172.21.15.29:0/716204528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 0x7f376c12b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c09f880 con 0x7f376c096680 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f376c09f700 con 0x7f376c094c60 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f376c09fa00 con 0x7f376c095ce0 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 -- 172.21.15.29:0/716204528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 msgr2=0x7f376c12aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c12aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 -- 172.21.15.29:0/716204528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 msgr2=0x7f376c12a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 0x7f376c12a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c12be00 con 0x7f376c096680 2024-09-21T01:53:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.084+0000 7f377a70f640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 0x7f376c12b670 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3774069690 tx=0x7f377407c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.090+0000 7f376affd640 1 -- 172.21.15.29:0/716204528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377407e680 con 0x7f376c096680 2024-09-21T01:53:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.090+0000 7f376affd640 1 -- 172.21.15.29:0/716204528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377407ecd0 con 0x7f376c096680 2024-09-21T01:53:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.090+0000 7f376affd640 1 -- 172.21.15.29:0/716204528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377407fe00 con 0x7f376c096680 2024-09-21T01:53:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.090+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c12c0f0 con 0x7f376c096680 2024-09-21T01:53:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.090+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c132690 con 0x7f376c096680 2024-09-21T01:53:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f376affd640 1 -- 172.21.15.29:0/716204528 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3774086030 con 0x7f376c096680 2024-09-21T01:53:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 Processor -- start 2024-09-21T01:53:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- start start 2024-09-21T01:53:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f376affd640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 0x7f37600b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38135850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f376affd640 1 -- 172.21.15.29:0/716204528 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f37741542c0 con 0x7f376c096680 2024-09-21T01:53:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 0x7f37600b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be38136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38135850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40770/0 (socket says 172.21.15.29:40770) 2024-09-21T01:53:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f3779f0e640 1 -- 172.21.15.29:0/353943530 learned_addr learned my addr 172.21.15.29:0/353943530 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55be3809f6f0 con 0x55be37d644d0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55be3808eb20 con 0x55be37d5c2f0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55be38092930 con 0x55be37d851b0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377a70f640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be38136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 -- 172.21.15.29:0/353943530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 msgr2=0x55be38135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 -- 172.21.15.29:0/353943530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 msgr2=0x55be38136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be38136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 -- 172.21.15.29:0/353943530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be38054530 con 0x55be37d5c2f0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377a70f640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be38136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38135ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3764018640 tx=0x7f3764019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cd8a68e73e1351 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f377970d640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 0x7f37600b8c30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3764016560 tx=0x7f3764009d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f375ffff640 1 -- 172.21.15.29:0/353943530 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764026070 con 0x55be37d5c2f0 2024-09-21T01:53:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f375ffff640 1 -- 172.21.15.29:0/353943530 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764010040 con 0x55be37d5c2f0 2024-09-21T01:53:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f375ffff640 1 -- 172.21.15.29:0/353943530 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640218c0 con 0x55be37d5c2f0 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 msgr2=0x55be38135ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38135ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3764018640 tx=0x7f3764019000 comp rx=0 tx=0).stop 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 shutdown_connections 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be38136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38135ec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 --2- 172.21.15.29:0/353943530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 >> 172.21.15.29:0/353943530 conn(0x55be37d79300 msgr2=0x55be381312e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 shutdown_connections 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.095+0000 7f37831c0740 1 -- 172.21.15.29:0/353943530 wait complete. 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- start start 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 Processor -- start 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d644d0 0x55be3813f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d851b0 0x55be381401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55be37fc6ab0 con 0x55be37d851b0 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55be38034730 con 0x55be37d5c2f0 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55be381374d0 con 0x55be37d644d0 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 10 client.0 _ll_register_callbacks cb 0x55be380886b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40876/0 (socket says 172.21.15.29:40876) 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 -- 172.21.15.29:0/3201609244 learned_addr learned my addr 172.21.15.29:0/3201609244 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f377a70f640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d851b0 0x55be381401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 -- 172.21.15.29:0/3201609244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d644d0 msgr2=0x55be3813f680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d644d0 0x55be3813f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 -- 172.21.15.29:0/3201609244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d851b0 msgr2=0x55be381401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d851b0 0x55be381401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 -- 172.21.15.29:0/3201609244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be37df0790 con 0x55be37d5c2f0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3770004900 tx=0x7f3770004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66864d8adc81faf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770044020 con 0x55be37d5c2f0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764010040 con 0x55be37d5c2f0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3770004f40 con 0x55be37d5c2f0 2024-09-21T01:53:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3770005140 con 0x55be37d5c2f0 2024-09-21T01:53:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 10 client.26165 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55be37e69620 con 0x55be37d5c2f0 2024-09-21T01:53:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764010040 con 0x55be37d5c2f0 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f3770036400 con 0x55be37d5c2f0 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 10 client.26165 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 20 client.26165 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d43d80 0x55be3814c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 4 client.26165 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be380a1640 con 0x55be37d43d80 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f377970d640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d43d80 0x55be3814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.100+0000 7f375d7fa640 10 client.26165 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.104+0000 7f377970d640 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d43d80 0x55be3814c810 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329a47d5135b79c6 server_cookie=599eabc919c72746 in_seq=0 out_seq=0 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f375d7fa640 1 -- 172.21.15.29:0/3201609244 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be380a1640 con 0x55be37d43d80 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f375d7fa640 10 client.26165 handle_command_reply: tid=0 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 client.26165 shutdown 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 msgr2=0x55be3813efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3770004900 tx=0x7f3770004930 comp rx=0 tx=0).stop 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 shutdown_connections 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d43d80 0x55be3814c810 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d851b0 0x55be381401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d644d0 0x55be3813f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 --2- 172.21.15.29:0/3201609244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be3813efc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f375d7fa640 0 client.26165 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.105+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 >> 172.21.15.29:0/3201609244 conn(0x55be37d79300 msgr2=0x55be38127150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.108+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 shutdown_connections 2024-09-21T01:53:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.108+0000 7f37831c0740 1 -- 172.21.15.29:0/3201609244 wait complete. 2024-09-21T01:53:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.108+0000 7f37831c0740 20 client.26165 trim_cache size 0 max 16384 2024-09-21T01:53:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.158+0000 7f37831c0740 1 Processor -- start 2024-09-21T01:53:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.158+0000 7f37831c0740 1 -- start start 2024-09-21T01:53:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.159+0000 7f37831c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38130c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.159+0000 7f37831c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be381470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.159+0000 7f37831c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38131160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.159+0000 7f37831c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c13b220 con 0x55be37d644d0 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f37831c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f376c13ce50 con 0x55be37d5c2f0 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f37831c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f376c13b8f0 con 0x55be37d851b0 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38130c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38130c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40882/0 (socket says 172.21.15.29:40882) 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f3779f0e640 1 -- 172.21.15.29:0/2593376539 learned_addr learned my addr 172.21.15.29:0/2593376539 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 -- 172.21.15.29:0/2593376539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 msgr2=0x55be38130c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38130c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 -- 172.21.15.29:0/2593376539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 msgr2=0x55be381470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be381470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.160+0000 7f377a70f640 1 -- 172.21.15.29:0/2593376539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3813d180 con 0x55be37d851b0 2024-09-21T01:53:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.161+0000 7f377a70f640 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38131160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f377402ebd0 tx=0x7f37740694a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cde3ac6f0532e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.165+0000 7f375f7fe640 1 -- 172.21.15.29:0/2593376539 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774105070 con 0x55be37d851b0 2024-09-21T01:53:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.165+0000 7f375f7fe640 1 -- 172.21.15.29:0/2593376539 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377411a030 con 0x55be37d851b0 2024-09-21T01:53:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.165+0000 7f375f7fe640 1 -- 172.21.15.29:0/2593376539 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774121200 con 0x55be37d851b0 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 -- 172.21.15.29:0/2593376539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 msgr2=0x55be38131160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38131160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f377402ebd0 tx=0x7f37740694a0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 -- 172.21.15.29:0/2593376539 shutdown_connections 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38131160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d644d0 0x55be381470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 --2- 172.21.15.29:0/2593376539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d5c2f0 0x55be38130c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 -- 172.21.15.29:0/2593376539 >> 172.21.15.29:0/2593376539 conn(0x55be37d79300 msgr2=0x55be38140820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 -- 172.21.15.29:0/2593376539 shutdown_connections 2024-09-21T01:53:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.176+0000 7f37831c0740 1 -- 172.21.15.29:0/2593376539 wait complete. 2024-09-21T01:53:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.177+0000 7f37831c0740 1 -- start start 2024-09-21T01:53:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.177+0000 7f37831c0740 1 Processor -- start 2024-09-21T01:53:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.178+0000 7f37831c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.178+0000 7f37831c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d644d0 0x55be38144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.178+0000 7f37831c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38145470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.179+0000 7f37831c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c13c780 con 0x55be37d5c2f0 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.179+0000 7f37831c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f376c13c080 con 0x55be37d644d0 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.179+0000 7f37831c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f376c139200 con 0x55be37d851b0 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.179+0000 7f37831c0740 10 client.0 _ll_register_callbacks cb 0x7f377000da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.180+0000 7f377970d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d644d0 0x55be38144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37168/0 (socket says 172.21.15.29:37168) 2024-09-21T01:53:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 -- 172.21.15.29:0/3796927051 learned_addr learned my addr 172.21.15.29:0/3796927051 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 -- 172.21.15.29:0/3796927051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 msgr2=0x55be38145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 -- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d644d0 msgr2=0x55be38144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d644d0 0x55be38144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.181+0000 7f3779f0e640 1 -- 172.21.15.29:0/3796927051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be38145bd0 con 0x55be37d5c2f0 2024-09-21T01:53:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.182+0000 7f3779f0e640 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3770036a10 tx=0x7f3770036cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3946b597be01a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.183+0000 7f375d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:28.184 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:27 smithi029 ceph-mon[31228]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 848 op/s 2024-09-21T01:53:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.187+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770044070 con 0x55be37d5c2f0 2024-09-21T01:53:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.187+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3770004f40 con 0x55be37d5c2f0 2024-09-21T01:53:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.187+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377411a030 con 0x55be37d5c2f0 2024-09-21T01:53:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.190+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f37700054f0 con 0x55be37d5c2f0 2024-09-21T01:53:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.190+0000 7f37831c0740 10 client.16434 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.190+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55be38146f30 con 0x55be37d5c2f0 2024-09-21T01:53:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.191+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3770059070 con 0x55be37d5c2f0 2024-09-21T01:53:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.192+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f3770054470 con 0x55be37d5c2f0 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 10 client.16434 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 20 client.16434 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d191c0 0x55be381b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 4 client.16434 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:53:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.193+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f377411a030 con 0x55be37d191c0 2024-09-21T01:53:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.194+0000 7f377970d640 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d191c0 0x55be381b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.194+0000 7f377970d640 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d191c0 0x55be381b9440 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40ff2e175e9c8b server_cookie=f51c742fe9b1f36f in_seq=0 out_seq=0 2024-09-21T01:53:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.194+0000 7f375d7fa640 10 client.16434 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:27 smithi190 ceph-mon[34368]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 848 op/s 2024-09-21T01:53:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f375d7fa640 1 -- 172.21.15.29:0/3796927051 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f377411a030 con 0x55be37d191c0 2024-09-21T01:53:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f375d7fa640 10 client.16434 handle_command_reply: tid=0 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 client.16434 shutdown 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 msgr2=0x55be38144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3770036a10 tx=0x7f3770036cc0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 shutdown_connections 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55be37d191c0 0x55be381b9440 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.260+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55be37d851b0 0x55be38145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be37d644d0 0x55be38144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 1 --2- 172.21.15.29:0/3796927051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be37d5c2f0 0x55be38144240 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 >> 172.21.15.29:0/3796927051 conn(0x55be37d79300 msgr2=0x55be3813a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f375d7fa640 0 client.16434 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 shutdown_connections 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 1 -- 172.21.15.29:0/3796927051 wait complete. 2024-09-21T01:53:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.261+0000 7f37831c0740 20 client.16434 trim_cache size 0 max 16384 2024-09-21T01:53:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:53:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:53:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:53:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.262+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 msgr2=0x7f37600b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.262+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 0x7f37600b8c30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3764016560 tx=0x7f3764009d30 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.262+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 msgr2=0x7f376c12b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.262+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 0x7f376c12b670 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3774069690 tx=0x7f377407c990 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 shutdown_connections 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37600b6720 0x7f37600b8c30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c096680 0x7f376c12b670 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c095ce0 0x7f376c12aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 --2- 172.21.15.29:0/716204528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f376c094c60 0x7f376c12a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.263+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 >> 172.21.15.29:0/716204528 conn(0x7f376c0905a0 msgr2=0x7f376c099000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.264+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 shutdown_connections 2024-09-21T01:53:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:28.264+0000 7f377af10640 1 -- 172.21.15.29:0/716204528 wait complete. 2024-09-21T01:53:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:27 smithi066 ceph-mon[35001]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 848 op/s 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.352+0000 7f20998a6640 1 -- 172.21.15.29:0/3860395216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094102a60 msgr2=0x7f2094102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.352+0000 7f20998a6640 1 --2- 172.21.15.29:0/3860395216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094102a60 0x7f2094102e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2084009900 tx=0x7f208402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 -- 172.21.15.29:0/3860395216 shutdown_connections 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 --2- 172.21.15.29:0/3860395216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094108cf0 0x7f209406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 --2- 172.21.15.29:0/3860395216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f20941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 --2- 172.21.15.29:0/3860395216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094102a60 0x7f2094102e70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 -- 172.21.15.29:0/3860395216 >> 172.21.15.29:0/3860395216 conn(0x7f2094079730 msgr2=0x7f2094079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 -- 172.21.15.29:0/3860395216 shutdown_connections 2024-09-21T01:53:28.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.353+0000 7f20998a6640 1 -- 172.21.15.29:0/3860395216 wait complete. 2024-09-21T01:53:28.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 Processor -- start 2024-09-21T01:53:28.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 -- start start 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 0x7f2094112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f2094113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094108cf0 0x7f2094113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2094115cc0 con 0x7f2094102a60 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2094115b40 con 0x7f20941083a0 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20998a6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2094115e40 con 0x7f2094108cf0 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f2092ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 0x7f2094112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f2092ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 0x7f2094112af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37190/0 (socket says 172.21.15.29:37190) 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f2092ffd640 1 -- 172.21.15.29:0/1495953504 learned_addr learned my addr 172.21.15.29:0/1495953504 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20927fc640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f2094113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.354+0000 7f20937fe640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094108cf0 0x7f2094113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.355+0000 7f20927fc640 1 -- 172.21.15.29:0/1495953504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094108cf0 msgr2=0x7f2094113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.355+0000 7f20927fc640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094108cf0 0x7f2094113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.355+0000 7f20927fc640 1 -- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 msgr2=0x7f2094112af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.355+0000 7f20927fc640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 0x7f2094112af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.355+0000 7f20927fc640 1 -- 172.21.15.29:0/1495953504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20941acb40 con 0x7f20941083a0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.356+0000 7f20927fc640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f2094113030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f208800d6a0 tx=0x7f208800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.356+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f20941083a0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.357+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088009820 con 0x7f20941083a0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.357+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088004c40 con 0x7f20941083a0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.357+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20941ace30 con 0x7f20941083a0 2024-09-21T01:53:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.357+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941ad200 con 0x7f20941083a0 2024-09-21T01:53:28.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.358+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20941087b0 con 0x7f20941083a0 2024-09-21T01:53:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.360+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2088005220 con 0x7f20941083a0 2024-09-21T01:53:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.361+0000 7f20988a4640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 0x7f207c0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.362+0000 7f2092ffd640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 0x7f207c0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.362+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f20880e6ae0 con 0x7f20941083a0 2024-09-21T01:53:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.364+0000 7f2092ffd640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 0x7f207c0b8b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2084005d90 tx=0x7f2084005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:28.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.366+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20880ad380 con 0x7f20941083a0 2024-09-21T01:53:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.755+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20941ad660 con 0x7f20941083a0 2024-09-21T01:53:28.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.757+0000 7f20988a4640 1 -- 172.21.15.29:0/1495953504 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11978 (secure 0 0 0) 0x7f20880b1760 con 0x7f20941083a0 2024-09-21T01:53:28.758 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:53:28.758 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":38,"btime":"2024-09-21T01:52:45:801163+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:52:45.234926+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:53:28.758 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 38 2024-09-21T01:53:28.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.759+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 msgr2=0x7f207c0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 0x7f207c0b8b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2084005d90 tx=0x7f2084005a40 comp rx=0 tx=0).stop 2024-09-21T01:53:28.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 msgr2=0x7f2094113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:28.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f2094113030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f208800d6a0 tx=0x7f208800db60 comp rx=0 tx=0).stop 2024-09-21T01:53:28.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 shutdown_connections 2024-09-21T01:53:28.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f207c0b6660 0x7f207c0b8b70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2094108cf0 0x7f2094113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20941083a0 0x7f2094113030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 --2- 172.21.15.29:0/1495953504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2094102a60 0x7f2094112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:28.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.760+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 >> 172.21.15.29:0/1495953504 conn(0x7f2094079730 msgr2=0x7f2094106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:28.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.762+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 shutdown_connections 2024-09-21T01:53:28.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:28.762+0000 7f20998a6640 1 -- 172.21.15.29:0/1495953504 wait complete. 2024-09-21T01:53:29.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:28 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1495953504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:53:29.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:28 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1495953504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:53:29.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:28 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1495953504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:53:29.487 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:53:29.890 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:30.178 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:29 smithi029 ceph-mon[31228]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-09-21T01:53:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:29 smithi190 ceph-mon[34368]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-09-21T01:53:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:29 smithi066 ceph-mon[35001]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-09-21T01:53:31.015 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.580+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/2453616933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 msgr2=0x7fa8f011f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.580+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/2453616933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 0x7fa8f011f140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009ca0 tx=0x7fa8e802a8f0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.583+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/2453616933 shutdown_connections 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.583+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/2453616933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 0x7fa8f011f140 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.583+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/2453616933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 0x7fa8f010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.583+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/2453616933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.583+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/2453616933 >> 172.21.15.29:0/2453616933 conn(0x7fa8f0070ec0 msgr2=0x7fa8f00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.585+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/2453616933 shutdown_connections 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.585+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/2453616933 wait complete. 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.586+0000 7fa8f7f1b640 1 Processor -- start 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.590+0000 7fa8f7f1b640 1 -- start start 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.590+0000 7fa8f7f1b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 0x7fa8f011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.590+0000 7fa8f7f1b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.590+0000 7fa8f7f1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 0x7fa8f0113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f7f1b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0122990 con 0x7fa8f0111f40 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f7f1b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0122810 con 0x7fa8f011cc30 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f7f1b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0122b10 con 0x7fa8f010a9e0 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f54b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f54b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37220/0 (socket says 172.21.15.29:37220) 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f54b6640 1 -- 172.21.15.29:0/1865249382 learned_addr learned my addr 172.21.15.29:0/1865249382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.591+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 0x7fa8f011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 -- 172.21.15.29:0/1865249382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 msgr2=0x7fa8f011a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 0x7fa8f011a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 -- 172.21.15.29:0/1865249382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 msgr2=0x7fa8f0113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 0x7fa8f0113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 -- 172.21.15.29:0/1865249382 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f0119190 con 0x7fa8f0111f40 2024-09-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.592+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0006fd0 tx=0x7fa8e000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.595+0000 7fa8e6ffd640 1 -- 172.21.15.29:0/1865249382 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0015020 con 0x7fa8f0111f40 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.595+0000 7fa8e6ffd640 1 -- 172.21.15.29:0/1865249382 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e0011ce0 con 0x7fa8f0111f40 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.595+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f0119480 con 0x7fa8f0111f40 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.596+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f01199c0 con 0x7fa8f0111f40 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.603+0000 7fa8fe48f740 1 Processor -- start 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.603+0000 7fa8fe48f740 1 -- start start 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f204390 0x557e1f5cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557e1f542ea0 con 0x557e1f1fc1b0 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557e1f5322d0 con 0x557e1f204390 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557e1f5360e0 con 0x557e1f225070 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8f64b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8f64b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40850/0 (socket says 172.21.15.29:40850) 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8f64b8640 1 -- 172.21.15.29:0/1715007939 learned_addr learned my addr 172.21.15.29:0/1715007939 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8e6ffd640 1 -- 172.21.15.29:0/1865249382 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0010d20 con 0x7fa8f0111f40 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.604+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f204390 0x557e1f5cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8e6ffd640 1 -- 172.21.15.29:0/1865249382 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa8e0018020 con 0x7fa8f0111f40 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 -- 172.21.15.29:0/1715007939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f204390 msgr2=0x557e1f5cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f204390 0x557e1f5cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 -- 172.21.15.29:0/1715007939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 msgr2=0x557e1f5ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 -- 172.21.15.29:0/1715007939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1f290650 con 0x557e1f225070 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.605+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005c40 tx=0x7fa8e8031bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df614c0fdc0fa23f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.606+0000 7fa8dbfff640 1 -- 172.21.15.29:0/1715007939 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8031680 con 0x557e1f225070 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.606+0000 7fa8dbfff640 1 -- 172.21.15.29:0/1715007939 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e8036070 con 0x557e1f225070 2024-09-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.609+0000 7fa8fe48f740 1 -- 172.21.15.29:0/1715007939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 msgr2=0x557e1f5cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.609+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005c40 tx=0x7fa8e8031bd0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.609+0000 7fa8fe48f740 1 -- 172.21.15.29:0/1715007939 shutdown_connections 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.610+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f204390 0x557e1f5cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.610+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.610+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/1715007939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f225070 0x557e1f5cc6a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.610+0000 7fa8fe48f740 1 -- 172.21.15.29:0/1715007939 >> 172.21.15.29:0/1715007939 conn(0x557e1f219460 msgr2=0x557e1f5ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.611+0000 7fa8fe48f740 1 -- 172.21.15.29:0/1715007939 shutdown_connections 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.611+0000 7fa8fe48f740 1 -- 172.21.15.29:0/1715007939 wait complete. 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.612+0000 7fa8fe48f740 1 -- start start 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.612+0000 7fa8e6ffd640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 0x7fa8dc0b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.612+0000 7fa8fe48f740 1 Processor -- start 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1e39f0 0x557e1f5d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557e1f393700 con 0x557e1f1e39f0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557e1f4cd160 con 0x557e1f1fc1b0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557e1f5daec0 con 0x557e1f204390 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.613+0000 7fa8fe48f740 10 client.0 _ll_register_callbacks cb 0x557e1f4e69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.614+0000 7fa8f5cb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.614+0000 7fa8f5cb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40936/0 (socket says 172.21.15.29:40936) 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.614+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/330536360 learned_addr learned my addr 172.21.15.29:0/330536360 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.615+0000 7fa8e6ffd640 1 -- 172.21.15.29:0/1865249382 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa8e0011910 con 0x7fa8f0111f40 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.615+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.617+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/330536360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 msgr2=0x557e1f5d79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.617+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5d79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.617+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/330536360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1e39f0 msgr2=0x557e1f5d80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.617+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1e39f0 0x557e1f5d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.617+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/330536360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1f4fac90 con 0x557e1f1fc1b0 2024-09-21T01:53:31.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.618+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 0x7fa8dc0b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.618+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e450 tx=0x7fa8ec00e850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbeefcd99e7c79a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.619+0000 7fa8d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017070 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e1f544df0 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa8ec013680 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec023050 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8fe48f740 10 client.26183 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.621+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557e1f5e6970 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.623+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e1f544df0 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.623+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa8ec012900 con 0x557e1f1fc1b0 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 10 client.26183 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 10 client.26183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 20 client.26183 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f1b8da0 0x557e1f5ea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 4 client.26183 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.625+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e1f5408a0 con 0x557e1f1b8da0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.629+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f1b8da0 0x557e1f5ea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.631+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 0x7fa8dc0b8940 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009a60 tx=0x7fa8e80093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.635+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f1b8da0 0x557e1f5ea200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50e39e1b5bb52005 server_cookie=dddda8ed8f8812de in_seq=0 out_seq=0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.635+0000 7fa8d97fa640 10 client.26183 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.639+0000 7fa8d97fa640 1 -- 172.21.15.29:0/330536360 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e1f5408a0 con 0x557e1f1b8da0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.639+0000 7fa8d97fa640 10 client.26183 handle_command_reply: tid=0 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 client.26183 shutdown 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 msgr2=0x557e1f5da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e450 tx=0x7fa8ec00e850 comp rx=0 tx=0).stop 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 shutdown_connections 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f1b8da0 0x557e1f5ea200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1e39f0 0x557e1f5d80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5d79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/330536360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1fc1b0 0x557e1f5da490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 >> 172.21.15.29:0/330536360 conn(0x557e1f219460 msgr2=0x557e1f5cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.640+0000 7fa8d97fa640 0 client.26183 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.641+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 shutdown_connections 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.641+0000 7fa8fe48f740 1 -- 172.21.15.29:0/330536360 wait complete. 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.641+0000 7fa8fe48f740 20 client.26183 trim_cache size 0 max 16384 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.687+0000 7fa8fe48f740 1 Processor -- start 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.687+0000 7fa8fe48f740 1 -- start start 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c69f0 con 0x557e1f1fc1b0 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c8620 con 0x557e1f1e39f0 2024-09-21T01:53:31.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.688+0000 7fa8f5cb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.689+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c70c0 con 0x557e1f204390 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.691+0000 7fa8f64b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.691+0000 7fa8f64b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40868/0 (socket says 172.21.15.29:40868) 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.691+0000 7fa8f64b8640 1 -- 172.21.15.29:0/561226740 learned_addr learned my addr 172.21.15.29:0/561226740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.692+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/561226740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 msgr2=0x557e1f5ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.692+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.692+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/561226740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 msgr2=0x557e1f5c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.692+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.692+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/561226740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1f5cf500 con 0x557e1f1e39f0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.697+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e4100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec012e70 tx=0x7fa8ec01b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec72a6870477595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.700+0000 7fa8db7fe640 1 -- 172.21.15.29:0/561226740 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017070 con 0x557e1f1e39f0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.700+0000 7fa8db7fe640 1 -- 172.21.15.29:0/561226740 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec020070 con 0x557e1f1e39f0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.700+0000 7fa8db7fe640 1 -- 172.21.15.29:0/561226740 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec005e20 con 0x557e1f1e39f0 2024-09-21T01:53:31.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 -- 172.21.15.29:0/561226740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 msgr2=0x557e1f5e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e4100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec012e70 tx=0x7fa8ec01b5c0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 -- 172.21.15.29:0/561226740 shutdown_connections 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f204390 0x557e1f5ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f1fc1b0 0x557e1f5c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/561226740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e4100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.702+0000 7fa8fe48f740 1 -- 172.21.15.29:0/561226740 >> 172.21.15.29:0/561226740 conn(0x557e1f219460 msgr2=0x557e1f5d2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.704+0000 7fa8fe48f740 1 -- 172.21.15.29:0/561226740 shutdown_connections 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.705+0000 7fa8fe48f740 1 -- 172.21.15.29:0/561226740 wait complete. 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.705+0000 7fa8fe48f740 1 -- start start 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.706+0000 7fa8fe48f740 1 Processor -- start 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.707+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.713+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f1fc1b0 0x557e1f5dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.713+0000 7fa8fe48f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f204390 0x557e1f5dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.713+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c7f50 con 0x557e1f204390 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.713+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c7850 con 0x557e1f1e39f0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.714+0000 7fa8fe48f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01c49d0 con 0x557e1f1fc1b0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.714+0000 7fa8fe48f740 10 client.0 _ll_register_callbacks cb 0x7fa8ec0093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.715+0000 7fa8f5cb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.716+0000 7fa8f5cb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40952/0 (socket says 172.21.15.29:40952) 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.716+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/2413113713 learned_addr learned my addr 172.21.15.29:0/2413113713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f1fc1b0 0x557e1f5dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f64b8640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f204390 0x557e1f5dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/2413113713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f1fc1b0 msgr2=0x557e1f5dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f1fc1b0 0x557e1f5dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/2413113713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f204390 msgr2=0x557e1f5dc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f204390 0x557e1f5dc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.717+0000 7fa8f5cb7640 1 -- 172.21.15.29:0/2413113713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1f5dc8f0 con 0x557e1f1e39f0 2024-09-21T01:53:31.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.719+0000 7fa8f5cb7640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec015040 tx=0x7fa8ec009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07bc18857d18d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.719+0000 7fa8d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017040 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa8ec01b830 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec020070 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec01be50 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8fe48f740 10 client.26189 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.720+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557e1f654a50 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec009490 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa8ec02d020 con 0x557e1f1e39f0 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8fe48f740 10 client.26189 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8fe48f740 10 client.26189 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8fe48f740 20 client.26189 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.721+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f655060 0x557e1f657480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:31.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.722+0000 7fa8fe48f740 4 client.26189 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.722+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa8ec020070 con 0x557e1f655060 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.725+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f655060 0x557e1f657480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.736+0000 7fa8f54b6640 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f655060 0x557e1f657480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1362223e51e4371a server_cookie=8bc1172fb4b05d20 in_seq=0 out_seq=0 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.736+0000 7fa8d97fa640 10 client.26189 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.737+0000 7fa8d97fa640 1 -- 172.21.15.29:0/2413113713 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa8ec020070 con 0x557e1f655060 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.737+0000 7fa8d97fa640 10 client.26189 handle_command_reply: tid=0 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.737+0000 7fa8fe48f740 1 client.26189 shutdown 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.737+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 msgr2=0x557e1f5e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.737+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec015040 tx=0x7fa8ec009dc0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 shutdown_connections 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x557e1f655060 0x557e1f657480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557e1f204390 0x557e1f5dc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e1f1fc1b0 0x557e1f5dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 --2- 172.21.15.29:0/2413113713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e1f1e39f0 0x557e1f5e0860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 >> 172.21.15.29:0/2413113713 conn(0x557e1f219460 msgr2=0x557e1f5d2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.738+0000 7fa8d97fa640 0 client.26189 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.740+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 shutdown_connections 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.740+0000 7fa8fe48f740 1 -- 172.21.15.29:0/2413113713 wait complete. 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.745+0000 7fa8fe48f740 20 client.26189 trim_cache size 0 max 16384 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 msgr2=0x7fa8dc0b8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 0x7fa8dc0b8940 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009a60 tx=0x7fa8e80093e0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 msgr2=0x7fa8f01137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0006fd0 tx=0x7fa8e000b820 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 shutdown_connections 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa8dc0b6430 0x7fa8dc0b8940 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8f011cc30 0x7fa8f0113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8f0111f40 0x7fa8f01137f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 --2- 172.21.15.29:0/1865249382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f010a9e0 0x7fa8f011a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:31.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.749+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 >> 172.21.15.29:0/1865249382 conn(0x7fa8f0070ec0 msgr2=0x7fa8f011e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:31.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.750+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 shutdown_connections 2024-09-21T01:53:31.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:30.750+0000 7fa8f7f1b640 1 -- 172.21.15.29:0/1865249382 wait complete. 2024-09-21T01:53:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:31 smithi029 ceph-mon[31228]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 895 op/s 2024-09-21T01:53:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:31 smithi190 ceph-mon[34368]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 895 op/s 2024-09-21T01:53:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:31 smithi066 ceph-mon[35001]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 895 op/s 2024-09-21T01:53:32.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:53:32.680 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:32.969 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 295.0 s, 1189.9 tps, lat 40.192 ms stddev 84.051 2024-09-21T01:53:33.075 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:53:33.075 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:53:33.075 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:53:33.076 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:32 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:53:33.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- 172.21.15.29:0/623322974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac108d20 msgr2=0x7ff1ac10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 --2- 172.21.15.29:0/623322974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac108d20 0x7ff1ac10f550 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009ca0 tx=0x7ff1a402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- 172.21.15.29:0/623322974 shutdown_connections 2024-09-21T01:53:33.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 --2- 172.21.15.29:0/623322974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac108d20 0x7ff1ac10f550 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 --2- 172.21.15.29:0/623322974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 --2- 172.21.15.29:0/623322974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac102a90 0x7ff1ac102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- 172.21.15.29:0/623322974 >> 172.21.15.29:0/623322974 conn(0x7ff1ac079750 msgr2=0x7ff1ac079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- 172.21.15.29:0/623322974 shutdown_connections 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- 172.21.15.29:0/623322974 wait complete. 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 Processor -- start 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.171+0000 7ff1b274a640 1 -- start start 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 0x7ff1ac10fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 0x7ff1ac110860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac113aa0 con 0x7ff1ac1083d0 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac113920 con 0x7ff1ac108d20 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b274a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac113c20 con 0x7ff1ac102a90 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48398/0 (socket says 172.21.15.29:48398) 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2291803982 learned_addr learned my addr 172.21.15.29:0/2291803982 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 0x7ff1ac10fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 0x7ff1ac110860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2291803982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 msgr2=0x7ff1ac10fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 0x7ff1ac10fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2291803982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 msgr2=0x7ff1ac110860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 0x7ff1ac110860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2291803982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac1a8710 con 0x7ff1ac1083d0 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 0x7ff1ac110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.172+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 0x7ff1ac10fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000e9c0 tx=0x7ff1a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1a97fa640 1 -- 172.21.15.29:0/2291803982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000cd80 con 0x7ff1ac1083d0 2024-09-21T01:53:33.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1a97fa640 1 -- 172.21.15.29:0/2291803982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a00054d0 con 0x7ff1ac1083d0 2024-09-21T01:53:33.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac1a8a00 con 0x7ff1ac1083d0 2024-09-21T01:53:33.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1a97fa640 1 -- 172.21.15.29:0/2291803982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0010430 con 0x7ff1ac1083d0 2024-09-21T01:53:33.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.173+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a8f40 con 0x7ff1ac1083d0 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.176+0000 7ff1b8cbe740 1 Processor -- start 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.176+0000 7ff1b8cbe740 1 -- start start 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5936070 0x5637b5cdd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1a97fa640 1 -- 172.21.15.29:0/2291803982 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff1a0004fd0 con 0x7ff1ac1083d0 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5cde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5637b5c53ea0 con 0x5637b590d1b0 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5637b5c432d0 con 0x5637b5915390 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5637b5c470e0 con 0x5637b5936070 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48410/0 (socket says 172.21.15.29:48410) 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 -- 172.21.15.29:0/536519104 learned_addr learned my addr 172.21.15.29:0/536519104 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5936070 0x5637b5cdd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5cde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 -- 172.21.15.29:0/536519104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5936070 msgr2=0x5637b5cdd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5936070 0x5637b5cdd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 -- 172.21.15.29:0/536519104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 msgr2=0x5637b5cde120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5cde120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.177+0000 7ff1abfff640 1 -- 172.21.15.29:0/536519104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b59a1650 con 0x5637b590d1b0 2024-09-21T01:53:33.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1abfff640 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009cf0 tx=0x7ff19c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ad2fdce591036a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1a97fa640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 0x7ff1940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1927fc640 1 -- 172.21.15.29:0/536519104 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c012450 con 0x5637b590d1b0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 0x7ff1940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1a97fa640 1 -- 172.21.15.29:0/2291803982 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff1a00e7870 con 0x7ff1ac1083d0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1927fc640 1 -- 172.21.15.29:0/536519104 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c012aa0 con 0x5637b590d1b0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.178+0000 7ff1927fc640 1 -- 172.21.15.29:0/536519104 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00fb80 con 0x5637b590d1b0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/536519104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 msgr2=0x5637b5cddbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 0x7ff1940b8a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004af0 tx=0x7ff1a4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009cf0 tx=0x7ff19c0117b0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/536519104 shutdown_connections 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5cde120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5cddbe0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/536519104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5936070 0x5637b5cdd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/536519104 >> 172.21.15.29:0/536519104 conn(0x5637b592a460 msgr2=0x5637b5cdb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/536519104 shutdown_connections 2024-09-21T01:53:33.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/536519104 wait complete. 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.179+0000 7ff1b8cbe740 1 -- start start 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 Processor -- start 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5ce8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5637b5aa4700 con 0x5637b5936070 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5637b5bde160 con 0x5637b5915390 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5637b5cebe80 con 0x5637b590d1b0 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b8cbe740 10 client.0 _ll_register_callbacks cb 0x5637b5bf79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b0ce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1b0ce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48414/0 (socket says 172.21.15.29:48414) 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48042/0 (socket says 172.21.15.29:48042) 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/3572734039 learned_addr learned my addr 172.21.15.29:0/3572734039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:33.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.180+0000 7ff1abfff640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5ce8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 msgr2=0x5637b5ce8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5ce8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/3572734039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 msgr2=0x5637b5ce9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/3572734039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b5c0bc90 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00a9fe0 tx=0x7ff1a0005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a023da67a322c9df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff18b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00eae60 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff1a00b20e0 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c012aa0 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff1a00ad7e0 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1b8cbe740 10 client.26198 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5637b5cf7be0 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.181+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c012aa0 con 0x5637b5915390 2024-09-21T01:53:33.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff1a00ac6e0 con 0x5637b5915390 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 10 client.26198 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 10 client.26198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 20 client.26198 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58f49f0 0x5637b5cfb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 4 client.26198 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b5c55df0 con 0x5637b58f49f0 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1abfff640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58f49f0 0x5637b5cfb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff18b7fe640 10 client.26198 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:53:33.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.182+0000 7ff1abfff640 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58f49f0 0x5637b5cfb0d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5864a4710d3ebcee server_cookie=620ac4f6d9e8bf90 in_seq=0 out_seq=0 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff18b7fe640 1 -- 172.21.15.29:0/3572734039 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637b5c55df0 con 0x5637b58f49f0 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff18b7fe640 10 client.26198 handle_command_reply: tid=0 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 client.26198 shutdown 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 msgr2=0x5637b5ce9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00a9fe0 tx=0x7ff1a0005680 comp rx=0 tx=0).stop 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 shutdown_connections 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce9240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58f49f0 0x5637b5cfb0d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/3572734039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5ce8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff18b7fe640 0 client.26198 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.183+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 >> 172.21.15.29:0/3572734039 conn(0x5637b592a460 msgr2=0x5637b5ce0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.184+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 shutdown_connections 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.184+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/3572734039 wait complete. 2024-09-21T01:53:33.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.184+0000 7ff1b8cbe740 20 client.26198 trim_cache size 0 max 16384 2024-09-21T01:53:33.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 Processor -- start 2024-09-21T01:53:33.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 -- start start 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5cdede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b15c0 con 0x5637b5936070 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b31f0 con 0x5637b5915390 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b1c90 con 0x5637b590d1b0 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b0ce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b0ce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48416/0 (socket says 172.21.15.29:48416) 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1b0ce7640 1 -- 172.21.15.29:0/2656610341 learned_addr learned my addr 172.21.15.29:0/2656610341 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5cdede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.197+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 -- 172.21.15.29:0/2656610341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 msgr2=0x5637b5cdede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5cdede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 -- 172.21.15.29:0/2656610341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 msgr2=0x5637b5ce5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 -- 172.21.15.29:0/2656610341 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b5ce0500 con 0x5637b5936070 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff1b0ce7640 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4032e80 tx=0x7ff1a4043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d379d7b07cd4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff191ffb640 1 -- 172.21.15.29:0/2656610341 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4032eb0 con 0x5637b5936070 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff191ffb640 1 -- 172.21.15.29:0/2656610341 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a4036050 con 0x5637b5936070 2024-09-21T01:53:33.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.198+0000 7ff191ffb640 1 -- 172.21.15.29:0/2656610341 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a402f9e0 con 0x5637b5936070 2024-09-21T01:53:33.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2656610341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 msgr2=0x5637b5ce6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4032e80 tx=0x7ff1a4043000 comp rx=0 tx=0).stop 2024-09-21T01:53:33.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2656610341 shutdown_connections 2024-09-21T01:53:33.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b5936070 0x5637b5ce6430 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5915390 0x5637b5ce5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2656610341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b590d1b0 0x5637b5cdede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2656610341 >> 172.21.15.29:0/2656610341 conn(0x5637b592a460 msgr2=0x5637b5cde6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2656610341 shutdown_connections 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2656610341 wait complete. 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.199+0000 7ff1b8cbe740 1 -- start start 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 Processor -- start 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 0x5637b5ced7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5936070 0x5637b5cedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b2b20 con 0x5637b590d1b0 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b2420 con 0x5637b5936070 2024-09-21T01:53:33.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1af5a0 con 0x5637b5915390 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b8cbe740 10 client.0 _ll_register_callbacks cb 0x7ff1a00ed240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1b0ce7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5936070 0x5637b5cedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48418/0 (socket says 172.21.15.29:48418) 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1abfff640 1 -- 172.21.15.29:0/2318181930 learned_addr learned my addr 172.21.15.29:0/2318181930 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 0x5637b5ced7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2318181930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5936070 msgr2=0x5637b5cedd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5936070 0x5637b5cedd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2318181930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 msgr2=0x5637b5ced280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1ab7fe640 1 -- 172.21.15.29:0/2318181930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b5d64c10 con 0x5637b5915390 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.200+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1ab7fe640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 0x5637b5ced7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00058c0 tx=0x7ff1a00b7500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4ebc927942df21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:33.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00a9180 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff1a00c3bc0 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a4036050 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff1a00ad180 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 10 client.26233 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5637b5d65e90 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a4036050 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff1a00b3680 con 0x5637b5915390 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 10 client.26233 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 10 client.26233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 20 client.26233 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58c9da0 0x5637b5d683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 4 client.26233 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:53:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.201+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637b5c518a0 con 0x5637b58c9da0 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58c9da0 0x5637b5d683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff18b7fe640 10 client.26233 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1abfff640 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58c9da0 0x5637b5d683d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfd4a2fb62e134df server_cookie=f8c069625d8faf5 in_seq=0 out_seq=0 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff18b7fe640 1 -- 172.21.15.29:0/2318181930 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637b5c518a0 con 0x5637b58c9da0 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff18b7fe640 10 client.26233 handle_command_reply: tid=0 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1b8cbe740 1 client.26233 shutdown 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 msgr2=0x5637b5ced7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 0x5637b5ced7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00058c0 tx=0x7ff1a00b7500 comp rx=0 tx=0).stop 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 shutdown_connections 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.202+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637b5936070 0x5637b5cedd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5637b58c9da0 0x5637b5d683d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637b5915390 0x5637b5ced7c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 --2- 172.21.15.29:0/2318181930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637b590d1b0 0x5637b5ced280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff18b7fe640 0 client.26233 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 >> 172.21.15.29:0/2318181930 conn(0x5637b592a460 msgr2=0x5637b5cde6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 shutdown_connections 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 1 -- 172.21.15.29:0/2318181930 wait complete. 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b8cbe740 20 client.26233 trim_cache size 0 max 16384 2024-09-21T01:53:33.208 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 msgr2=0x7ff1940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 0x7ff1940b8a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004af0 tx=0x7ff1a4005d80 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 msgr2=0x7ff1ac110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000e9c0 tx=0x7ff1a000ee80 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 shutdown_connections 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff1940b6510 0x7ff1940b8a20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1ac108d20 0x7ff1ac110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1ac1083d0 0x7ff1ac110320 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 --2- 172.21.15.29:0/2291803982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac102a90 0x7ff1ac10fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.203+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 >> 172.21.15.29:0/2291803982 conn(0x7ff1ac079750 msgr2=0x7ff1ac1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:33.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.204+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 shutdown_connections 2024-09-21T01:53:33.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:33.204+0000 7ff1b274a640 1 -- 172.21.15.29:0/2291803982 wait complete. 2024-09-21T01:53:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:53:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:53:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:53:33.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:32 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:53:33.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:53:33.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:53:33.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:53:33.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:32 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:53:33.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:53:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:33 smithi029 ceph-mon[31228]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 793 op/s 2024-09-21T01:53:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:33 smithi190 ceph-mon[34368]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 793 op/s 2024-09-21T01:53:34.257 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:33 smithi066 ceph-mon[35001]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 793 op/s 2024-09-21T01:53:34.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 -- 172.21.15.29:0/2357912326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 msgr2=0x7f04f8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 --2- 172.21.15.29:0/2357912326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f8110620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a410 comp rx=0 tx=0).stop 2024-09-21T01:53:34.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 -- 172.21.15.29:0/2357912326 shutdown_connections 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 --2- 172.21.15.29:0/2357912326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f8110620 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 --2- 172.21.15.29:0/2357912326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 0x7f04f81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 --2- 172.21.15.29:0/2357912326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 -- 172.21.15.29:0/2357912326 >> 172.21.15.29:0/2357912326 conn(0x7f04f807b850 msgr2=0x7f04f807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 -- 172.21.15.29:0/2357912326 shutdown_connections 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.738+0000 7f04fe196640 1 -- 172.21.15.29:0/2357912326 wait complete. 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 Processor -- start 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 -- start start 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 0x7f04f819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8113ab0 con 0x7f04f8109df0 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8113930 con 0x7f04f81094a0 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04fe196640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8113c30 con 0x7f04f8103b60 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57396/0 (socket says 172.21.15.29:57396) 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48444/0 (socket says 172.21.15.29:48444) 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.739+0000 7f04f77fe640 1 -- 172.21.15.29:0/609154315 learned_addr learned my addr 172.21.15.29:0/609154315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 0x7f04f819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 -- 172.21.15.29:0/609154315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 msgr2=0x7f04f819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 0x7f04f819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 -- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 msgr2=0x7f04f819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 -- 172.21.15.29:0/609154315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81a03f0 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f77fe640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04e800eb90 tx=0x7f04e800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f4ff9640 1 -- 172.21.15.29:0/609154315 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8004d20 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f4ff9640 1 -- 172.21.15.29:0/609154315 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e800ee90 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f81a06e0 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04f4ff9640 1 -- 172.21.15.29:0/609154315 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8010670 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.740+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81a6e40 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 Processor -- start 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 -- start start 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f4ff9640 1 -- 172.21.15.29:0/609154315 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f04e8005020 con 0x7f04f8103b60 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c70f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0c7065ea0 con 0x55e0c6d48070 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e0c70552d0 con 0x55e0c6d1f1b0 2024-09-21T01:53:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f050470a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0c70590e0 con 0x55e0c6d27390 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48446/0 (socket says 172.21.15.29:48446) 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f7fff640 1 -- 172.21.15.29:0/3615447615 learned_addr learned my addr 172.21.15.29:0/3615447615 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f77fe640 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.743+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c70f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 -- 172.21.15.29:0/3615447615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 msgr2=0x55e0c70f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c70f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 -- 172.21.15.29:0/3615447615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 msgr2=0x55e0c70efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 -- 172.21.15.29:0/3615447615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c6db3650 con 0x55e0c6d48070 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f7fff640 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f0032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba4a7487f3334ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f4ff9640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 0x7f04e00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04ddffb640 1 -- 172.21.15.29:0/3615447615 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0040070 con 0x55e0c6d48070 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04ddffb640 1 -- 172.21.15.29:0/3615447615 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f0004030 con 0x55e0c6d48070 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04ddffb640 1 -- 172.21.15.29:0/3615447615 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0035040 con 0x55e0c6d48070 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f77fe640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 0x7f04e00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.744+0000 7f04f4ff9640 1 -- 172.21.15.29:0/609154315 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f04e80e78f0 con 0x7f04f8103b60 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 -- 172.21.15.29:0/3615447615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 msgr2=0x55e0c70ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f0032b20 comp rx=0 tx=0).stop 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f04f77fe640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 0x7f04e00b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55e0c70ee320 tx=0x7f04e8095040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 -- 172.21.15.29:0/3615447615 shutdown_connections 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c70f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 --2- 172.21.15.29:0/3615447615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d48070 0x55e0c70ef6a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 -- 172.21.15.29:0/3615447615 >> 172.21.15.29:0/3615447615 conn(0x55e0c6d3c460 msgr2=0x55e0c70ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 -- 172.21.15.29:0/3615447615 shutdown_connections 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.745+0000 7f050470a740 1 -- 172.21.15.29:0/3615447615 wait complete. 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 -- start start 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 Processor -- start 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d48070 0x55e0c7107140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0c6eb6700 con 0x55e0c6d27390 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e0c6ff0160 con 0x55e0c6d1f1b0 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0c70fde20 con 0x55e0c6d48070 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f050470a740 10 client.0 _ll_register_callbacks cb 0x55e0c70099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48458/0 (socket says 172.21.15.29:48458) 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48094/0 (socket says 172.21.15.29:48094) 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.746+0000 7f04f6ffd640 1 -- 172.21.15.29:0/4276770317 learned_addr learned my addr 172.21.15.29:0/4276770317 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f7fff640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d48070 0x55e0c7107140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 -- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d48070 msgr2=0x55e0c7107140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d48070 0x55e0c7107140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 -- 172.21.15.29:0/4276770317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 msgr2=0x55e0c70fae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 -- 172.21.15.29:0/4276770317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c701dc90 con 0x55e0c6d27390 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0126e0 tx=0x7f04ec012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d29274c7d01e12d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012190 con 0x55e0c6d27390 2024-09-21T01:53:34.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f04ec00f920 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f0004030 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f04ec00fb20 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f050470a740 10 client.16476 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e0c7109c20 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.747+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f0004030 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f04ec01c420 con 0x55e0c6d27390 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 10 client.16476 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 10 client.16476 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 20 client.16476 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6d069f0 0x55e0c710d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 4 client.16476 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c7067df0 con 0x55e0c6d069f0 2024-09-21T01:53:34.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f04f7fff640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6d069f0 0x55e0c710d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f04f7fff640 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6d069f0 0x55e0c710d1e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1df7ea7086d790a server_cookie=8b929237d93346e6 in_seq=0 out_seq=0 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.748+0000 7f04d6ffd640 10 client.16476 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f04d6ffd640 1 -- 172.21.15.29:0/4276770317 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0c7067df0 con 0x55e0c6d069f0 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f04d6ffd640 10 client.16476 handle_command_reply: tid=0 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 client.16476 shutdown 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 msgr2=0x55e0c70fb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0126e0 tx=0x7f04ec012ba0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 shutdown_connections 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6d069f0 0x55e0c710d1e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d48070 0x55e0c7107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fb340 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 --2- 172.21.15.29:0/4276770317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f04d6ffd640 0 client.16476 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:53:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.749+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 >> 172.21.15.29:0/4276770317 conn(0x55e0c6d3c460 msgr2=0x55e0c70f2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.750+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 shutdown_connections 2024-09-21T01:53:34.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.750+0000 7f050470a740 1 -- 172.21.15.29:0/4276770317 wait complete. 2024-09-21T01:53:34.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.750+0000 7f050470a740 20 client.16476 trim_cache size 0 max 16384 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 Processor -- start 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 -- start start 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c70f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81af350 con 0x55e0c6d27390 2024-09-21T01:53:34.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81b0f80 con 0x55e0c6d48070 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f050470a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81afa20 con 0x55e0c6d1f1b0 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48474/0 (socket says 172.21.15.29:48474) 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 -- 172.21.15.29:0/2373155320 learned_addr learned my addr 172.21.15.29:0/2373155320 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f7fff640 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f7fff640 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c70f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 -- 172.21.15.29:0/2373155320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d1f1b0 msgr2=0x55e0c70fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 -- 172.21.15.29:0/2373155320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 msgr2=0x55e0c70f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c70f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.764+0000 7f04f6ffd640 1 -- 172.21.15.29:0/2373155320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c70fc0c0 con 0x55e0c6d27390 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.765+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0059f0 tx=0x7f04ec00f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd93a6605ad54fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.765+0000 7f04dd7fa640 1 -- 172.21.15.29:0/2373155320 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012210 con 0x55e0c6d27390 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.765+0000 7f04dd7fa640 1 -- 172.21.15.29:0/2373155320 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ec022030 con 0x55e0c6d27390 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.765+0000 7f04dd7fa640 1 -- 172.21.15.29:0/2373155320 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0050e0 con 0x55e0c6d27390 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- 172.21.15.29:0/2373155320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 msgr2=0x55e0c70fca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0059f0 tx=0x7f04ec00f9a0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- 172.21.15.29:0/2373155320 shutdown_connections 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c70f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d27390 0x55e0c70fca30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 --2- 172.21.15.29:0/2373155320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d1f1b0 0x55e0c70fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- 172.21.15.29:0/2373155320 >> 172.21.15.29:0/2373155320 conn(0x55e0c6d3c460 msgr2=0x55e0c70fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- 172.21.15.29:0/2373155320 shutdown_connections 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- 172.21.15.29:0/2373155320 wait complete. 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.766+0000 7f050470a740 1 -- start start 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 Processor -- start 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 0x55e0c71037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c7100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c71011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81b08b0 con 0x55e0c6d1f1b0 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81b01b0 con 0x55e0c6d48070 2024-09-21T01:53:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81ad330 con 0x55e0c6d27390 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f050470a740 10 client.0 _ll_register_callbacks cb 0x7f04ec009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 0x55e0c71037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 0x55e0c71037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48480/0 (socket says 172.21.15.29:48480) 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f7fff640 1 -- 172.21.15.29:0/1307780335 learned_addr learned my addr 172.21.15.29:0/1307780335 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c7100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f7fff640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c71011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 -- 172.21.15.29:0/1307780335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 msgr2=0x55e0c71011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c71011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 -- 172.21.15.29:0/1307780335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 msgr2=0x55e0c71037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:34.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 0x55e0c71037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f6ffd640 1 -- 172.21.15.29:0/1307780335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c71018d0 con 0x55e0c6d27390 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.767+0000 7f04f7fff640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c71011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04f6ffd640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c7100ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04ec01c810 tx=0x7f04ec012210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf6bc6d21bedf84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec035060 con 0x55e0c6d27390 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f04ec02c040 con 0x55e0c6d27390 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ec022030 con 0x55e0c6d27390 2024-09-21T01:53:34.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f04ec01ca00 con 0x55e0c6d27390 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 10 client.26251 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e0c7177a40 con 0x55e0c6d27390 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ec022030 con 0x55e0c6d27390 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f04ec0035f0 con 0x55e0c6d27390 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 10 client.26251 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 10 client.26251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 20 client.26251 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6cdbda0 0x55e0c717a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 4 client.26251 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:53:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.768+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0c70638a0 con 0x55e0c6cdbda0 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04f7fff640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6cdbda0 0x55e0c717a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04f7fff640 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6cdbda0 0x55e0c717a1f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1a8c95359d8232 server_cookie=8b7ca41f1a05a895 in_seq=0 out_seq=0 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04d6ffd640 10 client.26251 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04d6ffd640 1 -- 172.21.15.29:0/1307780335 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0c70638a0 con 0x55e0c6cdbda0 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04d6ffd640 10 client.26251 handle_command_reply: tid=0 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 client.26251 shutdown 2024-09-21T01:53:34.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 msgr2=0x55e0c7100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c7100ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04ec01c810 tx=0x7f04ec012210 comp rx=0 tx=0).stop 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 shutdown_connections 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e0c6cdbda0 0x55e0c717a1f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0c6d48070 0x55e0c71011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0c6d27390 0x55e0c7100ca0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f04d6ffd640 0 client.26251 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.769+0000 7f050470a740 1 --2- 172.21.15.29:0/1307780335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c6d1f1b0 0x55e0c71037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 >> 172.21.15.29:0/1307780335 conn(0x55e0c6d3c460 msgr2=0x55e0c70fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 shutdown_connections 2024-09-21T01:53:34.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f050470a740 1 -- 172.21.15.29:0/1307780335 wait complete. 2024-09-21T01:53:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f050470a740 20 client.26251 trim_cache size 0 max 16384 2024-09-21T01:53:34.786 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 msgr2=0x7f04e00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 0x7f04e00b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55e0c70ee320 tx=0x7f04e8095040 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 msgr2=0x7f04f819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04e800eb90 tx=0x7f04e800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 shutdown_connections 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f04e00b6510 0x7f04e00b8a20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8109df0 0x7f04f819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f81094a0 0x7f04f819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 --2- 172.21.15.29:0/609154315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f8103b60 0x7f04f819a480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 >> 172.21.15.29:0/609154315 conn(0x7f04f807b850 msgr2=0x7f04f8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 shutdown_connections 2024-09-21T01:53:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:34.770+0000 7f04fe196640 1 -- 172.21.15.29:0/609154315 wait complete. 2024-09-21T01:53:35.697 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:53:35.961 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:35 smithi190 ceph-mon[34368]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 892 op/s 2024-09-21T01:53:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:35 smithi066 ceph-mon[35001]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 892 op/s 2024-09-21T01:53:36.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:35 smithi029 ceph-mon[31228]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 892 op/s 2024-09-21T01:53:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 -- 172.21.15.29:0/1290782655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff980109df0 msgr2=0x7ff980110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 --2- 172.21.15.29:0/1290782655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff980109df0 0x7ff980110620 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff978009ca0 tx=0x7ff97802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 -- 172.21.15.29:0/1290782655 shutdown_connections 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 --2- 172.21.15.29:0/1290782655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff980109df0 0x7ff980110620 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 --2- 172.21.15.29:0/1290782655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9801094a0 0x7ff9801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 --2- 172.21.15.29:0/1290782655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980103b60 0x7ff980103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 -- 172.21.15.29:0/1290782655 >> 172.21.15.29:0/1290782655 conn(0x7ff98007b850 msgr2=0x7ff98007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 -- 172.21.15.29:0/1290782655 shutdown_connections 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.853+0000 7ff987ece640 1 -- 172.21.15.29:0/1290782655 wait complete. 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 Processor -- start 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 -- start start 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff980103b60 0x7ff98019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980109df0 0x7ff98019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113ab0 con 0x7ff9801094a0 2024-09-21T01:53:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113930 con 0x7ff980103b60 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff987ece640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113c30 con 0x7ff980109df0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48494/0 (socket says 172.21.15.29:48494) 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 -- 172.21.15.29:0/2163142437 learned_addr learned my addr 172.21.15.29:0/2163142437 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff98646b640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980109df0 0x7ff98019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985c6a640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff980103b60 0x7ff98019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 -- 172.21.15.29:0/2163142437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980109df0 msgr2=0x7ff98019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980109df0 0x7ff98019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 -- 172.21.15.29:0/2163142437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff980103b60 msgr2=0x7ff98019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.854+0000 7ff985469640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff980103b60 0x7ff98019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff985469640 1 -- 172.21.15.29:0/2163142437 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801a0390 con 0x7ff9801094a0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff985469640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff97000e9f0 tx=0x7ff97000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff976ffd640 1 -- 172.21.15.29:0/2163142437 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ccc0 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff976ffd640 1 -- 172.21.15.29:0/2163142437 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9700108e0 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801a0680 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff976ffd640 1 -- 172.21.15.29:0/2163142437 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970018860 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.855+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9801a6de0 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 Processor -- start 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff976ffd640 1 -- 172.21.15.29:0/2163142437 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff970010430 con 0x7ff9801094a0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 -- start start 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b811e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558b8115fea0 con 0x558b80e191b0 2024-09-21T01:53:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558b8114f2d0 con 0x558b80e42070 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.858+0000 7ff98e442740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b811530e0 con 0x558b80e21390 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811e9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48504/0 (socket says 172.21.15.29:48504) 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b811e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff985c6a640 1 -- 172.21.15.29:0/3102031794 learned_addr learned my addr 172.21.15.29:0/3102031794 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff985469640 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 -- 172.21.15.29:0/3102031794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 msgr2=0x558b811ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 -- 172.21.15.29:0/3102031794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 msgr2=0x558b811e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 -- 172.21.15.29:0/3102031794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b80ead650 con 0x558b80e42070 2024-09-21T01:53:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff98646b640 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b811e96a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff978005e30 tx=0x7ff97802f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12e8f0deaacfb84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff976ffd640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 0x7ff96c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff985c6a640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 0x7ff96c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff96bfff640 1 -- 172.21.15.29:0/3102031794 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978031cf0 con 0x558b80e42070 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff96bfff640 1 -- 172.21.15.29:0/3102031794 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff978036030 con 0x558b80e42070 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff96bfff640 1 -- 172.21.15.29:0/3102031794 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978004240 con 0x558b80e42070 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.859+0000 7ff976ffd640 1 -- 172.21.15.29:0/2163142437 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff9700e7550 con 0x7ff9801094a0 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 -- 172.21.15.29:0/3102031794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 msgr2=0x558b811e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff985c6a640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 0x7ff96c0b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x558b811e8320 tx=0x7ff97c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b811e96a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff978005e30 tx=0x7ff97802f4c0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 -- 172.21.15.29:0/3102031794 shutdown_connections 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 --2- 172.21.15.29:0/3102031794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b811e96a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 -- 172.21.15.29:0/3102031794 >> 172.21.15.29:0/3102031794 conn(0x558b80e36460 msgr2=0x558b811e7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 -- 172.21.15.29:0/3102031794 shutdown_connections 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.860+0000 7ff98e442740 1 -- 172.21.15.29:0/3102031794 wait complete. 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 -- start start 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 Processor -- start 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b81200fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558b80fb0700 con 0x558b80e191b0 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558b810ea160 con 0x558b80e42070 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b811f7ca0 con 0x558b80e21390 2024-09-21T01:53:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.861+0000 7ff98e442740 10 client.0 _ll_register_callbacks cb 0x558b811039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48508/0 (socket says 172.21.15.29:48508) 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 -- 172.21.15.29:0/4027325736 learned_addr learned my addr 172.21.15.29:0/4027325736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff98646b640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b81200fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985469640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 -- 172.21.15.29:0/4027325736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 msgr2=0x558b811f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 -- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 msgr2=0x558b81200fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b81200fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 -- 172.21.15.29:0/4027325736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b81117c90 con 0x558b80e191b0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff985c6a640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003d30 tx=0x7ff97c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1a9ae212f62e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff9697fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:36.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c02c060 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff978036030 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff97c017030 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.862+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff97c027410 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 10 client.16494 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558b81203a00 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff978036030 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff97c01c510 con 0x558b80e191b0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 10 client.16494 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 10 client.16494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 20 client.16494 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e009f0 0x558b81206f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 4 client.16494 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b81161df0 con 0x558b80e009f0 2024-09-21T01:53:36.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.863+0000 7ff985469640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e009f0 0x558b81206f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.864+0000 7ff9697fa640 10 client.16494 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.864+0000 7ff985469640 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e009f0 0x558b81206f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=868a91de4cb519b8 server_cookie=eb9094dd93fc244b in_seq=0 out_seq=0 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.864+0000 7ff9697fa640 1 -- 172.21.15.29:0/4027325736 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b81161df0 con 0x558b80e009f0 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.864+0000 7ff9697fa640 10 client.16494 handle_command_reply: tid=0 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.864+0000 7ff98e442740 1 client.16494 shutdown 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 msgr2=0x558b811f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003d30 tx=0x7ff97c004730 comp rx=0 tx=0).stop 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 shutdown_connections 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e009f0 0x558b81206f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e42070 0x558b81200fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b811f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff9697fa640 0 client.16494 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 --2- 172.21.15.29:0/4027325736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e191b0 0x558b811f4e00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 >> 172.21.15.29:0/4027325736 conn(0x558b80e36460 msgr2=0x558b811ec830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 shutdown_connections 2024-09-21T01:53:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 1 -- 172.21.15.29:0/4027325736 wait complete. 2024-09-21T01:53:36.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.865+0000 7ff98e442740 20 client.16494 trim_cache size 0 max 16384 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 Processor -- start 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 -- start start 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e191b0 0x558b81200ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e21390 0x558b811f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801af2e0 con 0x558b80e21390 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b0f10 con 0x558b80e009f0 2024-09-21T01:53:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.878+0000 7ff98e442740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801af9b0 con 0x558b80e191b0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff98646b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e21390 0x558b811f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48160/0 (socket says 172.21.15.29:48160) 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 -- 172.21.15.29:0/1714294290 learned_addr learned my addr 172.21.15.29:0/1714294290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985469640 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e191b0 0x558b81200ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 -- 172.21.15.29:0/1714294290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e191b0 msgr2=0x558b81200ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e191b0 0x558b81200ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 -- 172.21.15.29:0/1714294290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e21390 msgr2=0x558b811f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e21390 0x558b811f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 -- 172.21.15.29:0/1714294290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b811f5c50 con 0x558b80e009f0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff985c6a640 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003a30 tx=0x7ff97c01ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9990d54f81dea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff96b7fe640 1 -- 172.21.15.29:0/1714294290 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c037070 con 0x558b80e009f0 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff96b7fe640 1 -- 172.21.15.29:0/1714294290 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97c01c5f0 con 0x558b80e009f0 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.879+0000 7ff96b7fe640 1 -- 172.21.15.29:0/1714294290 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0336b0 con 0x558b80e009f0 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 -- 172.21.15.29:0/1714294290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 msgr2=0x558b811f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003a30 tx=0x7ff97c01ccb0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 -- 172.21.15.29:0/1714294290 shutdown_connections 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e21390 0x558b811f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e191b0 0x558b81200ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 --2- 172.21.15.29:0/1714294290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e009f0 0x558b811f52c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 -- 172.21.15.29:0/1714294290 >> 172.21.15.29:0/1714294290 conn(0x558b80e36460 msgr2=0x558b811f4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 -- 172.21.15.29:0/1714294290 shutdown_connections 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.880+0000 7ff98e442740 1 -- 172.21.15.29:0/1714294290 wait complete. 2024-09-21T01:53:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 -- start start 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 Processor -- start 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e191b0 0x558b811f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b8126a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b0840 con 0x558b80e009f0 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b0140 con 0x558b80e191b0 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801ad2c0 con 0x558b80e21390 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff98e442740 10 client.0 _ll_register_callbacks cb 0x7ff97c0042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.881+0000 7ff985c6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48540/0 (socket says 172.21.15.29:48540) 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 -- 172.21.15.29:0/3766860732 learned_addr learned my addr 172.21.15.29:0/3766860732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985469640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e191b0 0x558b811f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff98646b640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b8126a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 -- 172.21.15.29:0/3766860732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 msgr2=0x558b8126a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b8126a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 -- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e191b0 msgr2=0x558b811f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e191b0 0x558b811f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 -- 172.21.15.29:0/3766860732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8126af40 con 0x558b80e009f0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff985c6a640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff97c01c8b0 tx=0x7ff97c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd63994eb6408d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff9697fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c040020 con 0x558b80e009f0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff97c004730 con 0x558b80e009f0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff97c01c5f0 con 0x558b80e009f0 2024-09-21T01:53:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.882+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff97c037070 con 0x558b80e009f0 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 10 client.16500 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558b81272270 con 0x558b80e009f0 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97c01c5f0 con 0x558b80e009f0 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff97c0275b0 con 0x558b80e009f0 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 10 client.16500 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 10 client.16500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 20 client.16500 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e42070 0x558b81274970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 4 client.16500 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b8115d8a0 con 0x558b80e42070 2024-09-21T01:53:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.883+0000 7ff985469640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e42070 0x558b81274970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff985469640 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e42070 0x558b81274970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37ab03a99a4f87e5 server_cookie=11da2ee44f10cf84 in_seq=0 out_seq=0 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff9697fa640 10 client.16500 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff9697fa640 1 -- 172.21.15.29:0/3766860732 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b8115d8a0 con 0x558b80e42070 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff9697fa640 10 client.16500 handle_command_reply: tid=0 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 client.16500 shutdown 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 msgr2=0x558b811f9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff97c01c8b0 tx=0x7ff97c016000 comp rx=0 tx=0).stop 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 shutdown_connections 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x558b80e42070 0x558b81274970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b80e21390 0x558b8126a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b80e191b0 0x558b811f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 --2- 172.21.15.29:0/3766860732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558b80e009f0 0x558b811f9830 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff9697fa640 0 client.16500 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 >> 172.21.15.29:0/3766860732 conn(0x558b80e36460 msgr2=0x558b811ea6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 shutdown_connections 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 1 -- 172.21.15.29:0/3766860732 wait complete. 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.884+0000 7ff98e442740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T01:53:36.896 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 msgr2=0x7ff96c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 0x7ff96c0b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x558b811e8320 tx=0x7ff97c00b390 comp rx=0 tx=0).stop 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 msgr2=0x7ff98019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff97000e9f0 tx=0x7ff97000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 shutdown_connections 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff96c0b6510 0x7ff96c0b8a20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff980109df0 0x7ff98019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9801094a0 0x7ff98019a9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 --2- 172.21.15.29:0/2163142437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff980103b60 0x7ff98019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:36.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 >> 172.21.15.29:0/2163142437 conn(0x7ff98007b850 msgr2=0x7ff9801074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:36.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 shutdown_connections 2024-09-21T01:53:36.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:36.885+0000 7ff987ece640 1 -- 172.21.15.29:0/2163142437 wait complete. 2024-09-21T01:53:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 300.0 s, 321.7 tps, lat 472.892 ms stddev 1536.236 2024-09-21T01:53:38.213 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:37 smithi029 ceph-mon[31228]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2024-09-21T01:53:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:37 smithi190 ceph-mon[34368]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2024-09-21T01:53:38.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:53:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:37 smithi066 ceph-mon[35001]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2024-09-21T01:53:38.682 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:39.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:38 smithi029 ceph-mon[31228]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 974 op/s 2024-09-21T01:53:39.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:38 smithi029 ceph-mon[31228]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.279+0000 7fcae776f640 1 -- 172.21.15.29:0/1400029327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 msgr2=0x7fcae010eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.279+0000 7fcae776f640 1 --2- 172.21.15.29:0/1400029327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae010eca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 -- 172.21.15.29:0/1400029327 shutdown_connections 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 --2- 172.21.15.29:0/1400029327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae010eca0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 --2- 172.21.15.29:0/1400029327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae0104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 --2- 172.21.15.29:0/1400029327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 -- 172.21.15.29:0/1400029327 >> 172.21.15.29:0/1400029327 conn(0x7fcae0077800 msgr2=0x7fcae0077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.280+0000 7fcae776f640 1 -- 172.21.15.29:0/1400029327 shutdown_connections 2024-09-21T01:53:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.281+0000 7fcae776f640 1 -- 172.21.15.29:0/1400029327 wait complete. 2024-09-21T01:53:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.281+0000 7fcae776f640 1 Processor -- start 2024-09-21T01:53:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 -- start start 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae5d0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae5d0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48558/0 (socket says 172.21.15.29:48558) 2024-09-21T01:53:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae01a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48196/0 (socket says 172.21.15.29:48196) 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae550b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae550b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57488/0 (socket says 172.21.15.29:57488) 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae5d0c640 1 -- 172.21.15.29:0/1385567340 learned_addr learned my addr 172.21.15.29:0/1385567340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0113ac0 con 0x7fcae0105230 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0113940 con 0x7fcae01048e0 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.282+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0113c40 con 0x7fcae0102430 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 -- 172.21.15.29:0/1385567340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 msgr2=0x7fcae01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 -- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 msgr2=0x7fcae01ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae01a2580 con 0x7fcae0102430 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae550b640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcad67fc640 1 -- 172.21.15.29:0/1385567340 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000ccc0 con 0x7fcae0102430 2024-09-21T01:53:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae01ace20 con 0x7fcae0102430 2024-09-21T01:53:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcad67fc640 1 -- 172.21.15.29:0/1385567340 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad0005020 con 0x7fcae0102430 2024-09-21T01:53:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.283+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae01ad2d0 con 0x7fcae0102430 2024-09-21T01:53:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.284+0000 7fcad67fc640 1 -- 172.21.15.29:0/1385567340 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000ccc0 con 0x7fcae0102430 2024-09-21T01:53:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.290+0000 7fcaedce3740 1 Processor -- start 2024-09-21T01:53:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.291+0000 7fcaedce3740 1 -- start start 2024-09-21T01:53:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.291+0000 7fcad67fc640 1 -- 172.21.15.29:0/1385567340 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcad00047c0 con 0x7fcae0102430 2024-09-21T01:53:39.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.291+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d9582f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9582f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcae5d0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d9582f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcae5d0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d9582f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48560/0 (socket says 172.21.15.29:48560) 2024-09-21T01:53:39.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcae5d0c640 1 -- 172.21.15.29:0/3984983379 learned_addr learned my addr 172.21.15.29:0/3984983379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:39.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcae550b640 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9582f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcaedce3740 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d9582f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d958269ea0 con 0x55d957f4c070 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d9582592d0 con 0x55d957f2b390 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d95825d0e0 con 0x55d957f231b0 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d9582f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.292+0000 7fcad67fc640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 0x7fcacc0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 0x7fcacc0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 -- 172.21.15.29:0/3984983379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 msgr2=0x55d9582f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d9582f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 -- 172.21.15.29:0/3984983379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 msgr2=0x55d9582f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d9582f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 -- 172.21.15.29:0/3984983379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d957fb7650 con 0x55d957f231b0 2024-09-21T01:53:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcad67fc640 1 -- 172.21.15.29:0/1385567340 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcad00e6fb0 con 0x7fcae0102430 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae550b640 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9582f3800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcad00e4a30 tx=0x7fcad00a19a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503cf5cfe369d707 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 0x7fcacc0b8ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009a60 tx=0x7fcad802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcacb7fe640 1 -- 172.21.15.29:0/3984983379 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad001e300 con 0x55d957f231b0 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcacb7fe640 1 -- 172.21.15.29:0/3984983379 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad0017e70 con 0x55d957f231b0 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.293+0000 7fcacb7fe640 1 -- 172.21.15.29:0/3984983379 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000f3c0 con 0x55d957f231b0 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.294+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 msgr2=0x55d9582f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.294+0000 7fcaedce3740 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9582f3800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcad00e4a30 tx=0x7fcad00a19a0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.295+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 shutdown_connections 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.295+0000 7fcaedce3740 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d9582f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.295+0000 7fcaedce3740 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9582f3800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.295+0000 7fcaedce3740 1 --2- 172.21.15.29:0/3984983379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d9582f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.295+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 >> 172.21.15.29:0/3984983379 conn(0x55d957f40460 msgr2=0x55d9582f11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.296+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 shutdown_connections 2024-09-21T01:53:39.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.296+0000 7fcaedce3740 1 -- 172.21.15.29:0/3984983379 wait complete. 2024-09-21T01:53:39.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.297+0000 7fcaedce3740 1 -- start start 2024-09-21T01:53:39.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.297+0000 7fcaedce3740 1 Processor -- start 2024-09-21T01:53:39.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f2b390 0x55d9582fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 0x55d9582ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9580ba700 con 0x55d957f231b0 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48562/0 (socket says 172.21.15.29:48562) 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 -- 172.21.15.29:0/4265556725 learned_addr learned my addr 172.21.15.29:0/4265556725 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d9581f4160 con 0x55d957f4c070 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f2b390 0x55d9582fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d958301a40 con 0x55d957f2b390 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 0x55d9582ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcaedce3740 10 client.0 _ll_register_callbacks cb 0x55d95820d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 -- 172.21.15.29:0/4265556725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f2b390 msgr2=0x55d9582fee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f2b390 0x55d9582fee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 -- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 msgr2=0x55d9582ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 0x55d9582ff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d958221c90 con 0x55d957f231b0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 0x55d9582ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcae550b640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcad00acdf0 tx=0x7fcad0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657281d36b8a71f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.298+0000 7fcac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad001c070 con 0x55d957f231b0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcad000f6a0 con 0x55d957f231b0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d95826bdf0 con 0x55d957f231b0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcad000f3c0 con 0x55d957f231b0 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcaedce3740 10 client.16512 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d95830d6c0 con 0x55d957f231b0 2024-09-21T01:53:39.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d95826bdf0 con 0x55d957f231b0 2024-09-21T01:53:39.302 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:38 smithi066 ceph-mon[35001]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 974 op/s 2024-09-21T01:53:39.302 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:38 smithi066 ceph-mon[35001]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:39.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.299+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fcad00a8a40 con 0x55d957f231b0 2024-09-21T01:53:39.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 10 client.16512 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:39.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:39.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 20 client.16512 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957f0a9f0 0x55d958310a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 4 client.16512 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9582678a0 con 0x55d957f0a9f0 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957f0a9f0 0x55d958310a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcac8ff9640 10 client.16512 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.300+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957f0a9f0 0x55d958310a20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7a91a75df9900e server_cookie=7ea6042b1021413b in_seq=0 out_seq=0 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.301+0000 7fcac8ff9640 1 -- 172.21.15.29:0/4265556725 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9582678a0 con 0x55d957f0a9f0 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.301+0000 7fcac8ff9640 10 client.16512 handle_command_reply: tid=0 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.301+0000 7fcaedce3740 1 client.16512 shutdown 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 msgr2=0x55d9582fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcad00acdf0 tx=0x7fcad0009f60 comp rx=0 tx=0).stop 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 shutdown_connections 2024-09-21T01:53:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957f0a9f0 0x55d958310a20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f4c070 0x55d9582ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f2b390 0x55d9582fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcac8ff9640 0 client.16512 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 --2- 172.21.15.29:0/4265556725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f231b0 0x55d9582fe900 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.302+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 >> 172.21.15.29:0/4265556725 conn(0x55d957f40460 msgr2=0x55d9582f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.303+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 shutdown_connections 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.303+0000 7fcaedce3740 1 -- 172.21.15.29:0/4265556725 wait complete. 2024-09-21T01:53:39.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.303+0000 7fcaedce3740 20 client.16512 trim_cache size 0 max 16384 2024-09-21T01:53:39.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 Processor -- start 2024-09-21T01:53:39.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 -- start start 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 0x55d9582f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 0x55d9582fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b5a80 con 0x55d957f2b390 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b76b0 con 0x55d957f231b0 2024-09-21T01:53:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcaedce3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b6150 con 0x55d957f4c070 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48574/0 (socket says 172.21.15.29:48574) 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcae550b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 0x55d9582f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcae4d0a640 1 -- 172.21.15.29:0/955910666 learned_addr learned my addr 172.21.15.29:0/955910666 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.322+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 0x55d9582fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 -- 172.21.15.29:0/955910666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 msgr2=0x55d9582f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 0x55d9582f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 -- 172.21.15.29:0/955910666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 msgr2=0x55d958300530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 -- 172.21.15.29:0/955910666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9582ffbc0 con 0x55d957f4c070 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae550b640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 0x55d9582f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 0x55d9582fbaf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcad8031bd0 tx=0x7fcad8005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663a4b70e176111f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcacaffd640 1 -- 172.21.15.29:0/955910666 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad80317f0 con 0x55d957f4c070 2024-09-21T01:53:39.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcacaffd640 1 -- 172.21.15.29:0/955910666 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad8036040 con 0x55d957f4c070 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.323+0000 7fcacaffd640 1 -- 172.21.15.29:0/955910666 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad80305c0 con 0x55d957f4c070 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.324+0000 7fcaedce3740 1 -- 172.21.15.29:0/955910666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 msgr2=0x55d9582fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.324+0000 7fcaedce3740 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 0x55d9582fbaf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcad8031bd0 tx=0x7fcad8005df0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 -- 172.21.15.29:0/955910666 shutdown_connections 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f4c070 0x55d9582fbaf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f2b390 0x55d958300530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 --2- 172.21.15.29:0/955910666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f231b0 0x55d9582f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 -- 172.21.15.29:0/955910666 >> 172.21.15.29:0/955910666 conn(0x55d957f40460 msgr2=0x55d9582ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 -- 172.21.15.29:0/955910666 shutdown_connections 2024-09-21T01:53:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.325+0000 7fcaedce3740 1 -- 172.21.15.29:0/955910666 wait complete. 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.326+0000 7fcaedce3740 1 -- start start 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.326+0000 7fcaedce3740 1 Processor -- start 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.326+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.326+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d958312730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcaedce3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d95837a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57514/0 (socket says 172.21.15.29:57514) 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d958312730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcaedce3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b6fe0 con 0x55d957f4c070 2024-09-21T01:53:39.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcaedce3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b68e0 con 0x55d957f2b390 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae4d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d958312730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48228/0 (socket says 172.21.15.29:48228) 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcaedce3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b3a60 con 0x55d957f231b0 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae4d0a640 1 -- 172.21.15.29:0/2660661085 learned_addr learned my addr 172.21.15.29:0/2660661085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcaedce3740 10 client.0 _ll_register_callbacks cb 0x7fcadc009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae5d0c640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d95837a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 -- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 msgr2=0x55d958312730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d958312730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 -- 172.21.15.29:0/2660661085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 msgr2=0x55d95837a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d95837a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.327+0000 7fcae550b640 1 -- 172.21.15.29:0/2660661085 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95837a760 con 0x55d957f231b0 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcae550b640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcad00a9670 tx=0x7fcad00abff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40265a282b7342 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:39.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:39.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad001c070 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcad00b2d60 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad8036040 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcad00a9740 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcaedce3740 10 client.26287 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d95837b9c0 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.328+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad8036040 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fcad00b2f60 con 0x55d957f231b0 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 10 client.26287 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 20 client.26287 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957edfda0 0x55d95837dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:39.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 4 client.26287 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d958264cd0 con 0x55d957edfda0 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957edfda0 0x55d95837dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcac8ff9640 10 client.26287 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.329+0000 7fcae4d0a640 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957edfda0 0x55d95837dfb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40750cfe877d45c8 server_cookie=21cb280cba39e76 in_seq=0 out_seq=0 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcac8ff9640 1 -- 172.21.15.29:0/2660661085 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d958264cd0 con 0x55d957edfda0 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcac8ff9640 10 client.26287 handle_command_reply: tid=0 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 client.26287 shutdown 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 msgr2=0x55d9583121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcad00a9670 tx=0x7fcad00abff0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 shutdown_connections 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d957edfda0 0x55d95837dfb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d957f4c070 0x55d95837a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d957f2b390 0x55d958312730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 --2- 172.21.15.29:0/2660661085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d957f231b0 0x55d9583121f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcac8ff9640 0 client.26287 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 >> 172.21.15.29:0/2660661085 conn(0x55d957f40460 msgr2=0x55d9582f4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.330+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 shutdown_connections 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcaedce3740 1 -- 172.21.15.29:0/2660661085 wait complete. 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcaedce3740 20 client.26287 trim_cache size 0 max 16384 2024-09-21T01:53:39.334 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 msgr2=0x7fcacc0b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 0x7fcacc0b8ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009a60 tx=0x7fcad802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 msgr2=0x7fcae01a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.331+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 shutdown_connections 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcacc0b6790 0x7fcacc0b8ca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcae0105230 0x7fcae01ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcae01048e0 0x7fcae01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 --2- 172.21.15.29:0/1385567340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcae0102430 0x7fcae01a17a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:39.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 >> 172.21.15.29:0/1385567340 conn(0x7fcae0077800 msgr2=0x7fcae0105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:39.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 shutdown_connections 2024-09-21T01:53:39.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:39.332+0000 7fcae776f640 1 -- 172.21.15.29:0/1385567340 wait complete. 2024-09-21T01:53:39.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:38 smithi190 ceph-mon[34368]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 974 op/s 2024-09-21T01:53:39.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:38 smithi190 ceph-mon[34368]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:41.159 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:53:41.160 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.421+0000 7f1db51ca640 1 -- 172.21.15.29:0/3310495781 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da802f310 con 0x7f1db010cdb0 2024-09-21T01:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 -- 172.21.15.29:0/3310495781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 msgr2=0x7f1db010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 --2- 172.21.15.29:0/3310495781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db010f290 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009ca0 tx=0x7f1da802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 -- 172.21.15.29:0/3310495781 shutdown_connections 2024-09-21T01:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 --2- 172.21.15.29:0/3310495781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db011ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 --2- 172.21.15.29:0/3310495781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db010f290 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 --2- 172.21.15.29:0/3310495781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.422+0000 7f1db8430640 1 -- 172.21.15.29:0/3310495781 >> 172.21.15.29:0/3310495781 conn(0x7f1db0070810 msgr2=0x7f1db0072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.423+0000 7f1db8430640 1 -- 172.21.15.29:0/3310495781 shutdown_connections 2024-09-21T01:53:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.423+0000 7f1db8430640 1 -- 172.21.15.29:0/3310495781 wait complete. 2024-09-21T01:53:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 Processor -- start 2024-09-21T01:53:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 -- start start 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db0118b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1db011e120 con 0x7f1db010cdb0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1db011dfa0 con 0x7f1db010f9d0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db8430640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1db011e2a0 con 0x7f1db010a3c0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db59cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db61cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db61cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57536/0 (socket says 172.21.15.29:57536) 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db59cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48602/0 (socket says 172.21.15.29:48602) 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db61cc640 1 -- 172.21.15.29:0/2383083764 learned_addr learned my addr 172.21.15.29:0/2383083764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.424+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db0118b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 -- 172.21.15.29:0/2383083764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 msgr2=0x7f1db0118b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db0118b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 -- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 msgr2=0x7f1db0117f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 -- 172.21.15.29:0/2383083764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db0119360 con 0x7f1db010a3c0 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db59cb640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db0118b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.425+0000 7f1db61cc640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d950 tx=0x7f1da400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.426+0000 7f1daf7fe640 1 -- 172.21.15.29:0/2383083764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4017030 con 0x7f1db010a3c0 2024-09-21T01:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.426+0000 7f1daf7fe640 1 -- 172.21.15.29:0/2383083764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da4004020 con 0x7f1db010a3c0 2024-09-21T01:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.426+0000 7f1daf7fe640 1 -- 172.21.15.29:0/2383083764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40047a0 con 0x7f1db010a3c0 2024-09-21T01:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.428+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db0119650 con 0x7f1db010a3c0 2024-09-21T01:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.428+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db0119d10 con 0x7f1db010a3c0 2024-09-21T01:53:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.432+0000 7f1dbf481740 1 Processor -- start 2024-09-21T01:53:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.432+0000 7f1dbf481740 1 -- start start 2024-09-21T01:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 0x5575f8332e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f83334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f8334010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5575f82a56f0 con 0x5575f7f8b1b0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5575f8294b20 con 0x5575f7f622f0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1dbf481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5575f8298930 con 0x5575f7f6a4d0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1db59cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f8334010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1db59cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f8334010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57542/0 (socket says 172.21.15.29:57542) 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1db59cb640 1 -- 172.21.15.29:0/2561869087 learned_addr learned my addr 172.21.15.29:0/2561869087 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.433+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 0x5575f8332e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1daf7fe640 1 -- 172.21.15.29:0/2383083764 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1da4009a30 con 0x7f1db010a3c0 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1db69cd640 1 -- 172.21.15.29:0/2561869087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 msgr2=0x5575f8334010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f8334010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1db69cd640 1 -- 172.21.15.29:0/2561869087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 msgr2=0x5575f83334c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f83334c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.434+0000 7f1db69cd640 1 -- 172.21.15.29:0/2561869087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f825a530 con 0x5575f7f8b1b0 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.436+0000 7f1db69cd640 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 0x5575f8332e50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009f30 tx=0x7f1da00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22066fbb2109a341 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.436+0000 7f1daf7fe640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 0x7f1d980b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.436+0000 7f1dacbfe640 1 -- 172.21.15.29:0/2561869087 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004310 con 0x5575f7f8b1b0 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.436+0000 7f1dacbfe640 1 -- 172.21.15.29:0/2561869087 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00044b0 con 0x5575f7f8b1b0 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.436+0000 7f1dacbfe640 1 -- 172.21.15.29:0/2561869087 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00035f0 con 0x5575f7f8b1b0 2024-09-21T01:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.437+0000 7f1daf7fe640 1 -- 172.21.15.29:0/2383083764 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1da40e6340 con 0x7f1db010a3c0 2024-09-21T01:53:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.437+0000 7f1db61cc640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 0x7f1d980b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1db61cc640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 0x7f1d980b8aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1da40e3e50 tx=0x7f1da40a8bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 -- 172.21.15.29:0/2561869087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 msgr2=0x5575f8332e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 0x5575f8332e50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009f30 tx=0x7f1da00118d0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 -- 172.21.15.29:0/2561869087 shutdown_connections 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f8334010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f83334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 --2- 172.21.15.29:0/2561869087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f8b1b0 0x5575f8332e50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.438+0000 7f1dbf481740 1 -- 172.21.15.29:0/2561869087 >> 172.21.15.29:0/2561869087 conn(0x5575f7f7f300 msgr2=0x5575f832e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.439+0000 7f1dbf481740 1 -- 172.21.15.29:0/2561869087 shutdown_connections 2024-09-21T01:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.439+0000 7f1dbf481740 1 -- 172.21.15.29:0/2561869087 wait complete. 2024-09-21T01:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 -- start start 2024-09-21T01:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 Processor -- start 2024-09-21T01:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f833cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f8b1b0 0x5575f833d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5575f81ccab0 con 0x5575f7f6a4d0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5575f823a730 con 0x5575f7f622f0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5575f8334b30 con 0x5575f7f8b1b0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.440+0000 7f1dbf481740 10 client.0 _ll_register_callbacks cb 0x5575f828e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48264/0 (socket says 172.21.15.29:48264) 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 -- 172.21.15.29:0/3467957085 learned_addr learned my addr 172.21.15.29:0/3467957085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db59cb640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f833cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f8b1b0 0x5575f833d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 -- 172.21.15.29:0/3467957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f8b1b0 msgr2=0x5575f833d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f8b1b0 0x5575f833d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 -- 172.21.15.29:0/3467957085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 msgr2=0x5575f833cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f833cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 -- 172.21.15.29:0/3467957085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f7ff6790 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db59cb640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f833cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1db69cd640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005bc0 tx=0x7f1da00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c3ba07c8c95975 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.441+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00041b0 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1da0025670 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da00044b0 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1da0025870 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 10 client.26264 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5575f806f620 con 0x5575f7f622f0 2024-09-21T01:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00044b0 con 0x5575f7f622f0 2024-09-21T01:53:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f1da0028b00 con 0x5575f7f622f0 2024-09-21T01:53:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 10 client.26264 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 20 client.26264 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f49d80 0x5575f8349b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.442+0000 7f1dbf481740 4 client.26264 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.443+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f82a7640 con 0x5575f7f49d80 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.443+0000 7f1db59cb640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f49d80 0x5575f8349b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.444+0000 7f1db59cb640 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f49d80 0x5575f8349b70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ac28c2a4a1045c server_cookie=e11891a8a3a813e in_seq=0 out_seq=0 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.444+0000 7f1d95ffb640 10 client.26264 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.445+0000 7f1d95ffb640 1 -- 172.21.15.29:0/3467957085 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575f82a7640 con 0x5575f7f49d80 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.445+0000 7f1d95ffb640 10 client.26264 handle_command_reply: tid=0 2024-09-21T01:53:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.445+0000 7f1dbf481740 1 client.26264 shutdown 2024-09-21T01:53:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 msgr2=0x5575f833c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005bc0 tx=0x7f1da00096a0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 shutdown_connections 2024-09-21T01:53:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f49d80 0x5575f8349b70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1d95ffb640 0 client.26264 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f8b1b0 0x5575f833d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f833cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 --2- 172.21.15.29:0/3467957085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833c690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 >> 172.21.15.29:0/3467957085 conn(0x5575f7f7f300 msgr2=0x5575f832cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 shutdown_connections 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 1 -- 172.21.15.29:0/3467957085 wait complete. 2024-09-21T01:53:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.446+0000 7f1dbf481740 20 client.26264 trim_cache size 0 max 16384 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.473+0000 7f1dbf481740 1 Processor -- start 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.473+0000 7f1dbf481740 1 -- start start 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f833ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c2650 con 0x5575f7f49d80 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c4280 con 0x5575f7f622f0 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1dbf481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c2d20 con 0x5575f7f6a4d0 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48626/0 (socket says 172.21.15.29:48626) 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db69cd640 1 -- 172.21.15.29:0/506577226 learned_addr learned my addr 172.21.15.29:0/506577226 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db69cd640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f833ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 -- 172.21.15.29:0/506577226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 msgr2=0x5575f833ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f833ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 -- 172.21.15.29:0/506577226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 msgr2=0x5575f833afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db59cb640 1 -- 172.21.15.29:0/506577226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f82a5390 con 0x5575f7f622f0 2024-09-21T01:53:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.474+0000 7f1db69cd640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.475+0000 7f1db59cb640 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833b500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5575f834a280 tx=0x7f1da8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f8ed81297ef388 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.475+0000 7f1d97fff640 1 -- 172.21.15.29:0/506577226 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8054070 con 0x5575f7f622f0 2024-09-21T01:53:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.475+0000 7f1d97fff640 1 -- 172.21.15.29:0/506577226 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da8036040 con 0x5575f7f622f0 2024-09-21T01:53:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.475+0000 7f1d97fff640 1 -- 172.21.15.29:0/506577226 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da802f660 con 0x5575f7f622f0 2024-09-21T01:53:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- 172.21.15.29:0/506577226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 msgr2=0x5575f833b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833b500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5575f834a280 tx=0x7f1da8005fd0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- 172.21.15.29:0/506577226 shutdown_connections 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f6a4d0 0x5575f833ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f833b500 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 --2- 172.21.15.29:0/506577226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f49d80 0x5575f833afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- 172.21.15.29:0/506577226 >> 172.21.15.29:0/506577226 conn(0x5575f7f7f300 msgr2=0x5575f8337920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- 172.21.15.29:0/506577226 shutdown_connections 2024-09-21T01:53:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- 172.21.15.29:0/506577226 wait complete. 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.477+0000 7f1dbf481740 1 -- start start 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.478+0000 7f1dbf481740 1 Processor -- start 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.478+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 0x5575f8344060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.478+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f8342d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.478+0000 7f1dbf481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f8343260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.478+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 0x5575f8344060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1db69cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 0x5575f8344060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57562/0 (socket says 172.21.15.29:57562) 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1db69cd640 1 -- 172.21.15.29:0/575414524 learned_addr learned my addr 172.21.15.29:0/575414524 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1db59cb640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f8342d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c3bb0 con 0x5575f7f6a4d0 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c34b0 con 0x5575f7f622f0 2024-09-21T01:53:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.479+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01c0630 con 0x5575f7f49d80 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.480+0000 7f1db59cb640 1 -- 172.21.15.29:0/575414524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 msgr2=0x5575f8344060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.480+0000 7f1db59cb640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 0x5575f8344060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1db59cb640 1 -- 172.21.15.29:0/575414524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 msgr2=0x5575f8343260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1db69cd640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f8343260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1db59cb640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f8343260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1db59cb640 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f83437a0 con 0x5575f7f622f0 2024-09-21T01:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1dbf481740 10 client.0 _ll_register_callbacks cb 0x7f1da8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da8036040 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1db59cb640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f8342d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1da8052040 tx=0x7f1da8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70070b01fcacefbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.481+0000 7f1d95ffb640 10 client.26270 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1d95ffb640 1 -- 172.21.15.29:0/575414524 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da805f060 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1d95ffb640 1 -- 172.21.15.29:0/575414524 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1da8004240 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1d95ffb640 1 -- 172.21.15.29:0/575414524 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1da8054070 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1dbf481740 10 client.26270 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=38}) -- 0x5575f8342430 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1d95ffb640 1 -- 172.21.15.29:0/575414524 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f1da804e070 con 0x5575f7f622f0 2024-09-21T01:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1dbf481740 10 client.26270 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.482+0000 7f1dbf481740 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1dbf481740 20 client.26270 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f8b1b0 0x5575f83b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1dbf481740 4 client.26270 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575f82a30f0 con 0x5575f7f8b1b0 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1db69cd640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f8b1b0 0x5575f83b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.483+0000 7f1db69cd640 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f8b1b0 0x5575f83b6970 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbd7d05a3c2a522 server_cookie=21e9b2c7e40a6a40 in_seq=0 out_seq=0 2024-09-21T01:53:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.484+0000 7f1d95ffb640 10 client.26270 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.501+0000 7f1d95ffb640 1 -- 172.21.15.29:0/575414524 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575f82a30f0 con 0x5575f7f8b1b0 2024-09-21T01:53:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.501+0000 7f1d95ffb640 10 client.26270 handle_command_reply: tid=0 2024-09-21T01:53:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.501+0000 7f1dbf481740 1 client.26270 shutdown 2024-09-21T01:53:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.501+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 msgr2=0x5575f8342d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.501+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f8342d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1da8052040 tx=0x7f1da8000f40 comp rx=0 tx=0).stop 2024-09-21T01:53:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 shutdown_connections 2024-09-21T01:53:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5575f7f8b1b0 0x5575f83b6970 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575f7f6a4d0 0x5575f8343260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575f7f622f0 0x5575f8342d20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 --2- 172.21.15.29:0/575414524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575f7f49d80 0x5575f8344060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 >> 172.21.15.29:0/575414524 conn(0x5575f7f7f300 msgr2=0x5575f832f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.502+0000 7f1d95ffb640 0 client.26270 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.503+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 shutdown_connections 2024-09-21T01:53:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.503+0000 7f1dbf481740 1 -- 172.21.15.29:0/575414524 wait complete. 2024-09-21T01:53:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.503+0000 7f1dbf481740 20 client.26270 trim_cache size 0 max 16384 2024-09-21T01:53:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:53:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:53:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:53:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:53:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.505+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 msgr2=0x7f1d980b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.505+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 0x7f1d980b8aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1da40e3e50 tx=0x7f1da40a8bc0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.505+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 msgr2=0x7f1db0117920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.505+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d950 tx=0x7f1da400de10 comp rx=0 tx=0).stop 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.509+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 shutdown_connections 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.509+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d980b6590 0x7f1d980b8aa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.510+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1db010f9d0 0x7f1db0118b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.510+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db010cdb0 0x7f1db0117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.510+0000 7f1db8430640 1 --2- 172.21.15.29:0/2383083764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db010a3c0 0x7f1db0117920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.510+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 >> 172.21.15.29:0/2383083764 conn(0x7f1db0070810 msgr2=0x7f1db010b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.519+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 shutdown_connections 2024-09-21T01:53:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:41.522+0000 7f1db8430640 1 -- 172.21.15.29:0/2383083764 wait complete. 2024-09-21T01:53:41.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:41 smithi029 ceph-mon[31228]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.25k op/s 2024-09-21T01:53:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:41.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:41 smithi190 ceph-mon[34368]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.25k op/s 2024-09-21T01:53:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:41 smithi066 ceph-mon[35001]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.25k op/s 2024-09-21T01:53:42.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/2366417193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 msgr2=0x7fe9f81151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/2366417193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81151e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009900 tx=0x7fe9f402a410 comp rx=0 tx=0).stop 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/2366417193 shutdown_connections 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/2366417193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f8115720 0x7fe9f8117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/2366417193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81151e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/2366417193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f810bff0 0x7fe9f810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/2366417193 >> 172.21.15.29:0/2366417193 conn(0x7fe9f80712f0 msgr2=0x7fe9f8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/2366417193 shutdown_connections 2024-09-21T01:53:42.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.263+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/2366417193 wait complete. 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 Processor -- start 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 -- start start 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 0x7fe9f81a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f8115720 0x7fe9f81a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f811a0c0 con 0x7fe9f8115720 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8119f40 con 0x7fe9f810bff0 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9ff5a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f811a240 con 0x7fe9f8112dc0 2024-09-21T01:53:42.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9fcb3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9fcb3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56564/0 (socket says 172.21.15.29:56564) 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/607022002 learned_addr learned my addr 172.21.15.29:0/607022002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 0x7fe9f81a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.264+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f8115720 0x7fe9f81a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/607022002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 msgr2=0x7fe9f81a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 0x7fe9f81a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f8115720 msgr2=0x7fe9f81a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f8115720 0x7fe9f81a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/607022002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a6a40 con 0x7fe9f8112dc0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 0x7fe9f81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402a930 tx=0x7fe9f4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9ee7fc640 1 -- 172.21.15.29:0/607022002 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4035050 con 0x7fe9f8112dc0 2024-09-21T01:53:42.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.265+0000 7fe9ee7fc640 1 -- 172.21.15.29:0/607022002 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f4004490 con 0x7fe9f8112dc0 2024-09-21T01:53:42.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.266+0000 7fe9ee7fc640 1 -- 172.21.15.29:0/607022002 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40443e0 con 0x7fe9f8112dc0 2024-09-21T01:53:42.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.266+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a6cd0 con 0x7fe9f8112dc0 2024-09-21T01:53:42.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.266+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81ad490 con 0x7fe9f8112dc0 2024-09-21T01:53:42.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.271+0000 7fea05b15740 1 Processor -- start 2024-09-21T01:53:42.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.271+0000 7fea05b15740 1 -- start start 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c839a670 0x55d9c8762eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9c86d8e60 con 0x55d9c83bb350 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d9c86c8290 con 0x55d9c8392490 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fea05b15740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d9c86cc0a0 con 0x55d9c839a670 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9ee7fc640 1 -- 172.21.15.29:0/607022002 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe9f40035f0 con 0x7fe9f8112dc0 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fdb3e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fdb3e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55608/0 (socket says 172.21.15.29:55608) 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59164/0 (socket says 172.21.15.29:59164) 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fdb3e640 1 -- 172.21.15.29:0/3368698180 learned_addr learned my addr 172.21.15.29:0/3368698180 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:42.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.272+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c839a670 0x55d9c8762eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 -- 172.21.15.29:0/3368698180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c839a670 msgr2=0x55d9c8762eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c839a670 0x55d9c8762eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 -- 172.21.15.29:0/3368698180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 msgr2=0x55d9c8762430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 -- 172.21.15.29:0/3368698180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c8426930 con 0x55d9c8392490 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005fd0 tx=0x7fe9f0011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb6176ace9f498e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9ee7fc640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 0x7fe9e40b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9e37fe640 1 -- 172.21.15.29:0/3368698180 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003c30 con 0x55d9c8392490 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9e37fe640 1 -- 172.21.15.29:0/3368698180 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f000fe50 con 0x55d9c8392490 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9e37fe640 1 -- 172.21.15.29:0/3368698180 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00193e0 con 0x55d9c8392490 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.273+0000 7fe9ee7fc640 1 -- 172.21.15.29:0/607022002 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe9f410b8b0 con 0x7fe9f8112dc0 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 0x7fe9e40b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 -- 172.21.15.29:0/3368698180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 msgr2=0x55d9c8762970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005fd0 tx=0x7fe9f0011c70 comp rx=0 tx=0).stop 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 0x7fe9e40b8b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8012560 tx=0x7fe9e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 -- 172.21.15.29:0/3368698180 shutdown_connections 2024-09-21T01:53:42.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c839a670 0x55d9c8762eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c8392490 0x55d9c8762970 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 --2- 172.21.15.29:0/3368698180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c83bb350 0x55d9c8762430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 -- 172.21.15.29:0/3368698180 >> 172.21.15.29:0/3368698180 conn(0x55d9c83af740 msgr2=0x55d9c8760320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 -- 172.21.15.29:0/3368698180 shutdown_connections 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.274+0000 7fea05b15740 1 -- 172.21.15.29:0/3368698180 wait complete. 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.275+0000 7fea05b15740 1 -- start start 2024-09-21T01:53:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.275+0000 7fea05b15740 1 Processor -- start 2024-09-21T01:53:42.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.275+0000 7fea05b15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c876e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c83bb350 0x55d9c8779f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9c8529890 con 0x55d9c8392490 2024-09-21T01:53:42.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d9c8662d40 con 0x55d9c839a670 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d9c8770c10 con 0x55d9c83bb350 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fea05b15740 10 client.0 _ll_register_callbacks cb 0x55d9c86c6c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55620/0 (socket says 172.21.15.29:55620) 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 -- 172.21.15.29:0/2883302189 learned_addr learned my addr 172.21.15.29:0/2883302189 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c83bb350 0x55d9c8779f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c876e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 -- 172.21.15.29:0/2883302189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c83bb350 msgr2=0x55d9c8779f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c83bb350 0x55d9c8779f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 -- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 msgr2=0x55d9c876e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c876e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 -- 172.21.15.29:0/2883302189 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c877a570 con 0x55d9c8392490 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f80 tx=0x7fe9f00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78777535738405c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.276+0000 7fe9e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f001f040 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe9f000fae0 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f000fe50 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe9f0011040 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fea05b15740 10 client.16545 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d9c877ce80 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.277+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f000fe50 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fe9f0018070 con 0x55d9c8392490 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 10 client.16545 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 20 client.16545 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c8379cd0 0x55d9c8780290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 4 client.16545 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:42.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c86dadb0 con 0x55d9c8379cd0 2024-09-21T01:53:42.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.278+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c8379cd0 0x55d9c8780290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.281+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c8379cd0 0x55d9c8780290 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b35dabb4c814564 server_cookie=9a2ab1ef69676769 in_seq=0 out_seq=0 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.281+0000 7fe9e0ff9640 10 client.16545 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/2883302189 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9c86dadb0 con 0x55d9c8379cd0 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fe9e0ff9640 10 client.16545 handle_command_reply: tid=0 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 client.16545 shutdown 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 msgr2=0x55d9c876e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f80 tx=0x7fe9f00043c0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 shutdown_connections 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c8379cd0 0x55d9c8780290 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c83bb350 0x55d9c8779f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fe9e0ff9640 0 client.16545 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c876e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 --2- 172.21.15.29:0/2883302189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8392490 0x55d9c876e0e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 >> 172.21.15.29:0/2883302189 conn(0x55d9c83af740 msgr2=0x55d9c8765610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.282+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 shutdown_connections 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.283+0000 7fea05b15740 1 -- 172.21.15.29:0/2883302189 wait complete. 2024-09-21T01:53:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.283+0000 7fea05b15740 20 client.16545 trim_cache size 0 max 16384 2024-09-21T01:53:42.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.309+0000 7fea05b15740 1 Processor -- start 2024-09-21T01:53:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.309+0000 7fea05b15740 1 -- start start 2024-09-21T01:53:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.309+0000 7fea05b15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c87676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fea05b15740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8767bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fea05b15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c8765530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fea05b15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81be1f0 con 0x55d9c8379cd0 2024-09-21T01:53:42.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fea05b15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81bfe20 con 0x55d9c839a670 2024-09-21T01:53:42.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fea05b15740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81be8c0 con 0x55d9c8392490 2024-09-21T01:53:42.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fdb3e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c8765530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c87676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fdb3e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c8765530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59178/0 (socket says 172.21.15.29:59178) 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fdb3e640 1 -- 172.21.15.29:0/2562658918 learned_addr learned my addr 172.21.15.29:0/2562658918 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8767bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/2562658918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 msgr2=0x55d9c8765530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c8765530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/2562658918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 msgr2=0x55d9c87676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c87676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fcb3c640 1 -- 172.21.15.29:0/2562658918 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c8769770 con 0x55d9c8392490 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.310+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c87676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:42.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.311+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8767bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40d16f0 tx=0x7fe9f403bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0983b06d24b92a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.312+0000 7fe9e2ffd640 1 -- 172.21.15.29:0/2562658918 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40d1340 con 0x55d9c8392490 2024-09-21T01:53:42.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.312+0000 7fe9e2ffd640 1 -- 172.21.15.29:0/2562658918 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f40ba070 con 0x55d9c8392490 2024-09-21T01:53:42.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.312+0000 7fe9e2ffd640 1 -- 172.21.15.29:0/2562658918 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40f50b0 con 0x55d9c8392490 2024-09-21T01:53:42.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.314+0000 7fea05b15740 1 -- 172.21.15.29:0/2562658918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 msgr2=0x55d9c8767bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.314+0000 7fea05b15740 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8767bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40d16f0 tx=0x7fe9f403bf90 comp rx=0 tx=0).stop 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.314+0000 7fea05b15740 1 -- 172.21.15.29:0/2562658918 shutdown_connections 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c8765530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8767bf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 --2- 172.21.15.29:0/2562658918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c87676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 -- 172.21.15.29:0/2562658918 >> 172.21.15.29:0/2562658918 conn(0x55d9c83af740 msgr2=0x55d9c876c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 -- 172.21.15.29:0/2562658918 shutdown_connections 2024-09-21T01:53:42.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 -- 172.21.15.29:0/2562658918 wait complete. 2024-09-21T01:53:42.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.315+0000 7fea05b15740 1 -- start start 2024-09-21T01:53:42.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 Processor -- start 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8772e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c87e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81bf750 con 0x55d9c8379cd0 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81bf050 con 0x55d9c839a670 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81bc1d0 con 0x55d9c8392490 2024-09-21T01:53:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fea05b15740 10 client.0 _ll_register_callbacks cb 0x7fe9e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.316+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fd33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55642/0 (socket says 172.21.15.29:55642) 2024-09-21T01:53:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fd33d640 1 -- 172.21.15.29:0/1788619212 learned_addr learned my addr 172.21.15.29:0/1788619212 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c87e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fcb3c640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8772e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 -- 172.21.15.29:0/1788619212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 msgr2=0x55d9c8772e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8772e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 -- 172.21.15.29:0/1788619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 msgr2=0x55d9c8772890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 -- 172.21.15.29:0/1788619212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c87e4280 con 0x55d9c839a670 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9fdb3e640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c87e3c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009710 tx=0x7fe9e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc4fce6bc60709 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.317+0000 7fe9e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8004020 con 0x55d9c839a670 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe9e80041c0 con 0x55d9c839a670 2024-09-21T01:53:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f40ba070 con 0x55d9c839a670 2024-09-21T01:53:42.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe9e80043c0 con 0x55d9c839a670 2024-09-21T01:53:42.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 10 client.26288 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:42.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d9c87eb690 con 0x55d9c839a670 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e80054d0 con 0x55d9c839a670 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fe9e801d7a0 con 0x55d9c839a670 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 10 client.26288 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 20 client.26288 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c83bb350 0x55d9c87edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 4 client.26288 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.318+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe9f40ba070 con 0x55d9c83bb350 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.319+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c83bb350 0x55d9c87edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.319+0000 7fe9e0ff9640 10 client.26288 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.319+0000 7fe9fd33d640 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c83bb350 0x55d9c87edf00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed89d83db383c011 server_cookie=66f259a845f09467 in_seq=0 out_seq=0 2024-09-21T01:53:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fe9e0ff9640 1 -- 172.21.15.29:0/1788619212 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe9f40ba070 con 0x55d9c83bb350 2024-09-21T01:53:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fe9e0ff9640 10 client.26288 handle_command_reply: tid=0 2024-09-21T01:53:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 client.26288 shutdown 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 msgr2=0x55d9c87e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c87e3c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009710 tx=0x7fe9e8004730 comp rx=0 tx=0).stop 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 shutdown_connections 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d9c83bb350 0x55d9c87edf00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9c839a670 0x55d9c87e3c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c8392490 0x55d9c8772e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 --2- 172.21.15.29:0/1788619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9c8379cd0 0x55d9c8772890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 >> 172.21.15.29:0/1788619212 conn(0x55d9c83af740 msgr2=0x55d9c8763460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.321+0000 7fe9e0ff9640 0 client.26288 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.322+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 shutdown_connections 2024-09-21T01:53:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.322+0000 7fea05b15740 1 -- 172.21.15.29:0/1788619212 wait complete. 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.322+0000 7fea05b15740 20 client.26288 trim_cache size 0 max 16384 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "4700a670-a6ec-4427-bbc4-e0c61de06cab", 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:53:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:53:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 msgr2=0x7fe9e40b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 0x7fe9e40b8b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8012560 tx=0x7fe9e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 msgr2=0x7fe9f81a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402a930 tx=0x7fe9f4031a20 comp rx=0 tx=0).stop 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 shutdown_connections 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe9e40b6600 0x7fe9e40b8b10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9f8115720 0x7fe9f81a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f8112dc0 0x7fe9f81a1070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 --2- 172.21.15.29:0/607022002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9f810bff0 0x7fe9f81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.325+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 >> 172.21.15.29:0/607022002 conn(0x7fe9f80712f0 msgr2=0x7fe9f8115b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.326+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 shutdown_connections 2024-09-21T01:53:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:42.326+0000 7fe9ff5a1640 1 -- 172.21.15.29:0/607022002 wait complete. 2024-09-21T01:53:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 305.0 s, 2574.8 tps, lat 38.969 ms stddev 53.888 2024-09-21T01:53:43.163 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:53:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:53:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2024-09-21T01:53:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:53:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:43 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.756 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2024-09-21T01:53:43.756 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:53:43.756 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.756 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.757 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:43 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2024-09-21T01:53:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:53:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:43.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:43 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1672489418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c1083a0 msgr2=0x7fbf5c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1672489418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c1087b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009900 tx=0x7fbf5002a410 comp rx=0 tx=0).stop 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1672489418 shutdown_connections 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1672489418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1672489418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c1087b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1672489418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1672489418 >> 172.21.15.29:0/1672489418 conn(0x7fbf5c079730 msgr2=0x7fbf5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1672489418 shutdown_connections 2024-09-21T01:53:44.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.218+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1672489418 wait complete. 2024-09-21T01:53:44.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.219+0000 7fbf61dd4640 1 Processor -- start 2024-09-21T01:53:44.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.219+0000 7fbf61dd4640 1 -- start start 2024-09-21T01:53:44.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1129b0 con 0x7fbf5c1083a0 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c112830 con 0x7fbf5c102a60 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf61dd4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c112b30 con 0x7fbf5c108cf0 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55652/0 (socket says 172.21.15.29:55652) 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf5affd640 1 -- 172.21.15.29:0/1669984213 learned_addr learned my addr 172.21.15.29:0/1669984213 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.220+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 -- 172.21.15.29:0/1669984213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 msgr2=0x7fbf5c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 -- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 msgr2=0x7fbf5c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 -- 172.21.15.29:0/1669984213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c1a25e0 con 0x7fbf5c108cf0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5affd640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c1a1eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fd0 tx=0x7fbf5400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf58ff9640 1 -- 172.21.15.29:0/1669984213 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400ce20 con 0x7fbf5c108cf0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.221+0000 7fbf58ff9640 1 -- 172.21.15.29:0/1669984213 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54005020 con 0x7fbf5c108cf0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.222+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c1a28d0 con 0x7fbf5c108cf0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.222+0000 7fbf58ff9640 1 -- 172.21.15.29:0/1669984213 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400ce20 con 0x7fbf5c108cf0 2024-09-21T01:53:44.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.222+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a9030 con 0x7fbf5c108cf0 2024-09-21T01:53:44.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.224+0000 7fbf58ff9640 1 -- 172.21.15.29:0/1669984213 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbf54010430 con 0x7fbf5c108cf0 2024-09-21T01:53:44.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.225+0000 7fbf68348740 1 Processor -- start 2024-09-21T01:53:44.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.226+0000 7fbf68348740 1 -- start start 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca277053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca277058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ca27672e50 con 0x55ca2732c1b0 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca27662280 con 0x55ca27334390 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf68348740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca27666090 con 0x55ca27355070 2024-09-21T01:53:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.227+0000 7fbf58ff9640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 0x7fbf440b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56620/0 (socket says 172.21.15.29:56620) 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf5affd640 1 -- 172.21.15.29:0/3056048041 learned_addr learned my addr 172.21.15.29:0/3056048041 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf58ff9640 1 -- 172.21.15.29:0/1669984213 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbf540e7700 con 0x7fbf5c108cf0 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca277058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.228+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca277053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 -- 172.21.15.29:0/3056048041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 msgr2=0x55ca277058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca277058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 -- 172.21.15.29:0/3056048041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 msgr2=0x55ca277053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca277053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 -- 172.21.15.29:0/3056048041 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca273c0650 con 0x55ca27355070 2024-09-21T01:53:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 0x7fbf440b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.229+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005e00 tx=0x7fbf50005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7181883f182b5be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.237+0000 7fbf41ffb640 1 -- 172.21.15.29:0/3056048041 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50035030 con 0x55ca27355070 2024-09-21T01:53:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.237+0000 7fbf41ffb640 1 -- 172.21.15.29:0/3056048041 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf500054d0 con 0x55ca27355070 2024-09-21T01:53:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.237+0000 7fbf41ffb640 1 -- 172.21.15.29:0/3056048041 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50004c80 con 0x55ca27355070 2024-09-21T01:53:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.240+0000 7fbf68348740 1 -- 172.21.15.29:0/3056048041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 msgr2=0x55ca27704e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.240+0000 7fbf68348740 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005e00 tx=0x7fbf50005d90 comp rx=0 tx=0).stop 2024-09-21T01:53:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.244+0000 7fbf5bfff640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 0x7fbf440b8c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf5401e000 tx=0x7fbf54005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.245+0000 7fbf68348740 1 -- 172.21.15.29:0/3056048041 shutdown_connections 2024-09-21T01:53:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.245+0000 7fbf68348740 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca277058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.245+0000 7fbf68348740 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca277053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.245+0000 7fbf68348740 1 --2- 172.21.15.29:0/3056048041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca27355070 0x55ca27704e60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.245+0000 7fbf68348740 1 -- 172.21.15.29:0/3056048041 >> 172.21.15.29:0/3056048041 conn(0x55ca27349460 msgr2=0x55ca27702d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.246+0000 7fbf68348740 1 -- 172.21.15.29:0/3056048041 shutdown_connections 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.246+0000 7fbf68348740 1 -- 172.21.15.29:0/3056048041 wait complete. 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.246+0000 7fbf68348740 1 -- start start 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.246+0000 7fbf68348740 1 Processor -- start 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 0x55ca27710b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27355070 0x55ca27700de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ca274c3700 con 0x55ca27334390 2024-09-21T01:53:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca275fd110 con 0x55ca27355070 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca277135e0 con 0x55ca2732c1b0 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.247+0000 7fbf68348740 10 client.0 _ll_register_callbacks cb 0x55ca27616990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.248+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.248+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27355070 0x55ca27700de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.248+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56636/0 (socket says 172.21.15.29:56636) 2024-09-21T01:53:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.248+0000 7fbf5affd640 1 -- 172.21.15.29:0/3427472486 learned_addr learned my addr 172.21.15.29:0/3427472486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.248+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 0x55ca27710b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5affd640 1 -- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27355070 msgr2=0x55ca27700de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27355070 0x55ca27700de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5affd640 1 -- 172.21.15.29:0/3427472486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 msgr2=0x55ca27710b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 0x55ca27710b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5affd640 1 -- 172.21.15.29:0/3427472486 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2762ac40 con 0x55ca2732c1b0 2024-09-21T01:53:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.250+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 0x55ca27710b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:53:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.252+0000 7fbf5affd640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005ab0 tx=0x7fbf50004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a131dcfd9c824c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.252+0000 7fbf3affd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.256+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50035070 con 0x55ca2732c1b0 2024-09-21T01:53:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.256+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbf50003d10 con 0x55ca2732c1b0 2024-09-21T01:53:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.256+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf500054d0 con 0x55ca2732c1b0 2024-09-21T01:53:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.257+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbf50003830 con 0x55ca2732c1b0 2024-09-21T01:53:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.258+0000 7fbf68348740 10 client.26335 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.258+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ca27714bb0 con 0x55ca2732c1b0 2024-09-21T01:53:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.260+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50056070 con 0x55ca2732c1b0 2024-09-21T01:53:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.261+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fbf50031a20 con 0x55ca2732c1b0 2024-09-21T01:53:44.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.261+0000 7fbf68348740 10 client.26335 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:44.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.261+0000 7fbf68348740 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.261+0000 7fbf68348740 20 client.26335 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.262+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca273139f0 0x55ca27717fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.262+0000 7fbf68348740 4 client.26335 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.262+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbf500054d0 con 0x55ca273139f0 2024-09-21T01:53:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.262+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca273139f0 0x55ca27717fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.265+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca273139f0 0x55ca27717fd0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed922cac794c0a2 server_cookie=8e648f659e74c171 in_seq=0 out_seq=0 2024-09-21T01:53:44.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.265+0000 7fbf3affd640 10 client.26335 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:44.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.266+0000 7fbf3affd640 1 -- 172.21.15.29:0/3427472486 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbf500054d0 con 0x55ca273139f0 2024-09-21T01:53:44.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.266+0000 7fbf3affd640 10 client.26335 handle_command_reply: tid=0 2024-09-21T01:53:44.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.266+0000 7fbf68348740 1 client.26335 shutdown 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 msgr2=0x55ca27710620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005ab0 tx=0x7fbf50004920 comp rx=0 tx=0).stop 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 shutdown_connections 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca273139f0 0x55ca27717fd0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27355070 0x55ca27700de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf3affd640 0 client.26335 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca27334390 0x55ca27710b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 --2- 172.21.15.29:0/3427472486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca27710620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 >> 172.21.15.29:0/3427472486 conn(0x55ca27349460 msgr2=0x55ca27707ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.267+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 shutdown_connections 2024-09-21T01:53:44.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.268+0000 7fbf68348740 1 -- 172.21.15.29:0/3427472486 wait complete. 2024-09-21T01:53:44.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.268+0000 7fbf68348740 20 client.26335 trim_cache size 0 max 16384 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.299+0000 7fbf68348740 1 Processor -- start 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 -- start start 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca277025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca27702ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1aceb0 con 0x55ca273139f0 2024-09-21T01:53:44.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1aeae0 con 0x55ca27334390 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf68348740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ad580 con 0x55ca2732c1b0 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55678/0 (socket says 172.21.15.29:55678) 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 -- 172.21.15.29:0/1738182411 learned_addr learned my addr 172.21.15.29:0/1738182411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca277025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca27702ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 -- 172.21.15.29:0/1738182411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 msgr2=0x55ca277025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca277025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 -- 172.21.15.29:0/1738182411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 msgr2=0x55ca27702ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.300+0000 7fbf5affd640 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca27702ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.301+0000 7fbf5affd640 1 -- 172.21.15.29:0/1738182411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca27707cc0 con 0x55ca273139f0 2024-09-21T01:53:44.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.301+0000 7fbf5affd640 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbf500057c0 tx=0x7fbf50005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9646a83718832a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.301+0000 7fbf417fa640 1 -- 172.21.15.29:0/1738182411 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50035070 con 0x55ca273139f0 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.301+0000 7fbf417fa640 1 -- 172.21.15.29:0/1738182411 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50004990 con 0x55ca273139f0 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.301+0000 7fbf417fa640 1 -- 172.21.15.29:0/1738182411 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5004cab0 con 0x55ca273139f0 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 -- 172.21.15.29:0/1738182411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 msgr2=0x55ca27708670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbf500057c0 tx=0x7fbf50005a20 comp rx=0 tx=0).stop 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 -- 172.21.15.29:0/1738182411 shutdown_connections 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca27702ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca2732c1b0 0x55ca277025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 --2- 172.21.15.29:0/1738182411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca273139f0 0x55ca27708670 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 -- 172.21.15.29:0/1738182411 >> 172.21.15.29:0/1738182411 conn(0x55ca27349460 msgr2=0x55ca27707f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 -- 172.21.15.29:0/1738182411 shutdown_connections 2024-09-21T01:53:44.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.302+0000 7fbf68348740 1 -- 172.21.15.29:0/1738182411 wait complete. 2024-09-21T01:53:44.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.303+0000 7fbf68348740 1 -- start start 2024-09-21T01:53:44.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.303+0000 7fbf68348740 1 Processor -- start 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca2779e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca2771ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ae410 con 0x55ca2732c1b0 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1add10 con 0x55ca27334390 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ab000 con 0x55ca273139f0 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca2779e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56642/0 (socket says 172.21.15.29:56642) 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 -- 172.21.15.29:0/351677802 learned_addr learned my addr 172.21.15.29:0/351677802 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:44.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca2771ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf68348740 10 client.0 _ll_register_callbacks cb 0x7fbf4c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 -- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 msgr2=0x55ca2771ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca2771ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 -- 172.21.15.29:0/351677802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 msgr2=0x55ca2779e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca2779e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.304+0000 7fbf5affd640 1 -- 172.21.15.29:0/351677802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2771b350 con 0x55ca273139f0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf5affd640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf50032fd0 tx=0x7fbf50000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89fd3f72faae8954 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf3affd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50035020 con 0x55ca273139f0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbf5004ca80 con 0x55ca273139f0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf50004990 con 0x55ca273139f0 2024-09-21T01:53:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbf5004cc80 con 0x55ca273139f0 2024-09-21T01:53:44.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf68348740 10 client.26341 fetch_fsmap learned FSMap version 38 2024-09-21T01:53:44.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ca277a6a10 con 0x55ca273139f0 2024-09-21T01:53:44.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.305+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5006d030 con 0x55ca273139f0 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fbf50003a60 con 0x55ca273139f0 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 10 client.26341 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 20 client.26341 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca27355070 0x55ca277a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 4 client.26341 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbf50004990 con 0x55ca27355070 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca27355070 0x55ca277a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf3affd640 10 client.26341 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.307+0000 7fbf5b7fe640 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca27355070 0x55ca277a8f50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239263113102fdb3 server_cookie=933edb993a8d6974 in_seq=0 out_seq=0 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf3affd640 1 -- 172.21.15.29:0/351677802 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x7fbf50004990 con 0x55ca27355070 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf3affd640 10 client.26341 handle_command_reply: tid=0 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 client.26341 shutdown 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 msgr2=0x55ca2779e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf50032fd0 tx=0x7fbf50000bc0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 shutdown_connections 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ca27355070 0x55ca277a8f50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca27334390 0x55ca2771ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca2732c1b0 0x55ca2779e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 --2- 172.21.15.29:0/351677802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca273139f0 0x55ca2779e140 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 >> 172.21.15.29:0/351677802 conn(0x55ca27349460 msgr2=0x55ca27707f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf3affd640 0 client.26341 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 shutdown_connections 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 1 -- 172.21.15.29:0/351677802 wait complete. 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.308+0000 7fbf68348740 20 client.26341 trim_cache size 0 max 16384 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:53:44.314 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "4700a670-a6ec-4427-bbc4-e0c61de06cab", 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.317 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:53:44.319 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.309+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 msgr2=0x7fbf440b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.309+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 0x7fbf440b8c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf5401e000 tx=0x7fbf54005f70 comp rx=0 tx=0).stop 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.309+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 msgr2=0x7fbf5c1a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.309+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c1a1eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fd0 tx=0x7fbf5400ef10 comp rx=0 tx=0).stop 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.310+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 shutdown_connections 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.310+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbf440b6710 0x7fbf440b8c20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.310+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c1a1eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.310+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.311+0000 7fbf61dd4640 1 --2- 172.21.15.29:0/1669984213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf5c102a60 0x7fbf5c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:44.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.311+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 >> 172.21.15.29:0/1669984213 conn(0x7fbf5c079730 msgr2=0x7fbf5c06ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:44.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.311+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 shutdown_connections 2024-09-21T01:53:44.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:53:44.311+0000 7fbf61dd4640 1 -- 172.21.15.29:0/1669984213 wait complete. 2024-09-21T01:53:44.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:44 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:44.790 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:44 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:44.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:44 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:53:45.144 INFO:tasks.cephfs.filesystem:scrub status for tag:4700a670-a6ec-4427-bbc4-e0c61de06cab - {'path': '/', 'tag': '4700a670-a6ec-4427-bbc4-e0c61de06cab', 'options': 'recursive,force'} 2024-09-21T01:53:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:45 smithi190 ceph-mon[34368]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.07k op/s 2024-09-21T01:53:45.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:45 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:45.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:45 smithi066 ceph-mon[35001]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.07k op/s 2024-09-21T01:53:45.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:45 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:45 smithi029 ceph-mon[31228]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.07k op/s 2024-09-21T01:53:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:45 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:47.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:47 smithi190 ceph-mon[34368]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T01:53:47.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:47 smithi190 ceph-mon[34368]: Health check update: 6 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:47.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:47 smithi066 ceph-mon[35001]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T01:53:47.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:47 smithi066 ceph-mon[35001]: Health check update: 6 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:47.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:47 smithi029 ceph-mon[31228]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T01:53:47.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:47 smithi029 ceph-mon[31228]: Health check update: 6 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T01:53:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 310.0 s, 2576.8 tps, lat 38.777 ms stddev 44.738 2024-09-21T01:53:49.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:49 smithi190 ceph-mon[34368]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-09-21T01:53:49.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:49 smithi066 ceph-mon[35001]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-09-21T01:53:49.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:49 smithi029 ceph-mon[31228]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-09-21T01:53:50.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:50 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:50.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:50 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:50.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:50 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:50.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:50 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:50.848 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:50 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:50.848 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:50 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:53:51.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:51 smithi190 ceph-mon[34368]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-21T01:53:51.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:51.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:51.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:53:51.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:51 smithi066 ceph-mon[35001]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-21T01:53:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:53:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:51 smithi029 ceph-mon[31228]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-21T01:53:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 315.0 s, 3231.0 tps, lat 30.933 ms stddev 34.536 2024-09-21T01:53:53.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:53 smithi190 ceph-mon[34368]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T01:53:53.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:53 smithi066 ceph-mon[35001]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T01:53:53.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:53 smithi029 ceph-mon[31228]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T01:53:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.693+0000 7fcdf3fff640 1 -- 172.21.15.29:0/2062161023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 msgr2=0x7fcdec0ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.693+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/2062161023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec0ab270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d910 tx=0x7fcde402c390 comp rx=0 tx=0).stop 2024-09-21T01:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.697+0000 7fcdf3fff640 1 -- 172.21.15.29:0/2062161023 shutdown_connections 2024-09-21T01:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.698+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/2062161023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdec0ab980 0x7fcdec0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.698+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/2062161023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec0ab270 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.698+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/2062161023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec0a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.698+0000 7fcdf3fff640 1 -- 172.21.15.29:0/2062161023 >> 172.21.15.29:0/2062161023 conn(0x7fcdec0a0480 msgr2=0x7fcdec0a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.699+0000 7fcdf3fff640 1 -- 172.21.15.29:0/2062161023 shutdown_connections 2024-09-21T01:53:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.700+0000 7fcdf3fff640 1 -- 172.21.15.29:0/2062161023 wait complete. 2024-09-21T01:53:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.701+0000 7fcdf3fff640 1 Processor -- start 2024-09-21T01:53:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.702+0000 7fcdf3fff640 1 -- start start 2024-09-21T01:53:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.702+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdec0ab980 0x7fcdec140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0b0840 con 0x7fcdec0a6430 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0b06c0 con 0x7fcdec0a8d90 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0b09c0 con 0x7fcdec0ab980 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec13f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37724/0 (socket says 172.21.15.29:37724) 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.703+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/1435211414 learned_addr learned my addr 172.21.15.29:0/1435211414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 -- 172.21.15.29:0/1435211414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdec0ab980 msgr2=0x7fcdec140890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdec0ab980 0x7fcdec140890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 -- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 msgr2=0x7fcdec13f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec13f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.704+0000 7fcdf27fc640 1 -- 172.21.15.29:0/1435211414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec1410b0 con 0x7fcdec0a8d90 2024-09-21T01:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.705+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec13fce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcde402c8b0 tx=0x7fcde4034a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.705+0000 7fcddbfff640 1 -- 172.21.15.29:0/1435211414 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4034e00 con 0x7fcdec0a8d90 2024-09-21T01:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.705+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec141340 con 0x7fcdec0a8d90 2024-09-21T01:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.705+0000 7fcddbfff640 1 -- 172.21.15.29:0/1435211414 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde40035f0 con 0x7fcdec0a8d90 2024-09-21T01:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.705+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1478b0 con 0x7fcdec0a8d90 2024-09-21T01:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.707+0000 7fcddbfff640 1 -- 172.21.15.29:0/1435211414 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde403c680 con 0x7fcdec0a8d90 2024-09-21T01:53:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.710+0000 7fcddbfff640 1 -- 172.21.15.29:0/1435211414 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcde400c050 con 0x7fcdec0a8d90 2024-09-21T01:53:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.710+0000 7fce00431740 1 Processor -- start 2024-09-21T01:53:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 -- start start 2024-09-21T01:53:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5598445426f0 con 0x5598441ff2f0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559844531b20 con 0x5598442281b0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.711+0000 7fce00431740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559844535930 con 0x5598442074d0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41538/0 (socket says 172.21.15.29:41538) 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 -- 172.21.15.29:0/332459571 learned_addr learned my addr 172.21.15.29:0/332459571 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 -- 172.21.15.29:0/332459571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 msgr2=0x5598445d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 -- 172.21.15.29:0/332459571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 msgr2=0x5598445d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.712+0000 7fcdf37fe640 1 -- 172.21.15.29:0/332459571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598444f7530 con 0x5598442281b0 2024-09-21T01:53:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.713+0000 7fcddbfff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 0x7fcddc0b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.713+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcde80096f0 tx=0x7fcde8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d4d26e3536dec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.714+0000 7fcdd97fa640 1 -- 172.21.15.29:0/332459571 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8015020 con 0x5598442281b0 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.714+0000 7fcdd97fa640 1 -- 172.21.15.29:0/332459571 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde800bd80 con 0x5598442281b0 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.714+0000 7fcdd97fa640 1 -- 172.21.15.29:0/332459571 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8010380 con 0x5598442281b0 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.714+0000 7fcddbfff640 1 -- 172.21.15.29:0/1435211414 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcde4106d70 con 0x7fcdec0a8d90 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.714+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 0x7fcddc0b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.715+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 0x7fcddc0b8970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4068db0 tx=0x7fcdf40692f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.715+0000 7fce00431740 1 -- 172.21.15.29:0/332459571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 msgr2=0x5598445d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.715+0000 7fce00431740 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcde80096f0 tx=0x7fcde8009bb0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 -- 172.21.15.29:0/332459571 shutdown_connections 2024-09-21T01:53:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 --2- 172.21.15.29:0/332459571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445d0040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 -- 172.21.15.29:0/332459571 >> 172.21.15.29:0/332459571 conn(0x55984421c300 msgr2=0x5598445cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 -- 172.21.15.29:0/332459571 shutdown_connections 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 -- 172.21.15.29:0/332459571 wait complete. 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 -- start start 2024-09-21T01:53:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.716+0000 7fce00431740 1 Processor -- start 2024-09-21T01:53:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 0x5598445d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559844469ab0 con 0x5598442281b0 2024-09-21T01:53:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5598444d7730 con 0x5598441ff2f0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5598445d1d20 con 0x5598442074d0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fce00431740 10 client.0 _ll_register_callbacks cb 0x55984452b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 0x5598445d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445daa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37742/0 (socket says 172.21.15.29:37742) 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.717+0000 7fcdf37fe640 1 -- 172.21.15.29:0/4140773016 learned_addr learned my addr 172.21.15.29:0/4140773016 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/4140773016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 msgr2=0x5598445d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/4140773016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 msgr2=0x5598445daa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445daa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/4140773016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559844293790 con 0x5598441ff2f0 2024-09-21T01:53:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.718+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.719+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 0x5598445d9810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4069100 tx=0x7fcdf4069620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c3cea485593b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.719+0000 7fcdd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.719+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4085b20 con 0x5598441ff2f0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde800bd80 con 0x5598441ff2f0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcdf4085cc0 con 0x5598441ff2f0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdf408e440 con 0x5598441ff2f0 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fce00431740 10 client.26318 fetch_fsmap learned FSMap version 39 2024-09-21T01:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55984430c620 con 0x5598441ff2f0 2024-09-21T01:53:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.720+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf408ea90 con 0x5598441ff2f0 2024-09-21T01:53:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcdf408c020 con 0x5598441ff2f0 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 10 client.26318 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 20 client.26318 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441e6d80 0x5598445e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 4 client.26318 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.721+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcde800bd80 con 0x5598441e6d80 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.722+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441e6d80 0x5598445e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.723+0000 7fcdd6ff5640 10 client.26318 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.723+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441e6d80 0x5598445e6f40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5726088fa79d081 server_cookie=5853137d0afdfa54 in_seq=0 out_seq=0 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/4140773016 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcde800bd80 con 0x5598441e6d80 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fcdd6ff5640 10 client.26318 handle_command_reply: tid=0 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 client.26318 shutdown 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 msgr2=0x5598445d9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 0x5598445d9810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4069100 tx=0x7fcdf4069620 comp rx=0 tx=0).stop 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 shutdown_connections 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441e6d80 0x5598445e6f40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445daa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445d9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 --2- 172.21.15.29:0/4140773016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598441ff2f0 0x5598445d9810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 >> 172.21.15.29:0/4140773016 conn(0x55984421c300 msgr2=0x5598445ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fcdd6ff5640 0 client.26318 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 shutdown_connections 2024-09-21T01:53:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 1 -- 172.21.15.29:0/4140773016 wait complete. 2024-09-21T01:53:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.724+0000 7fce00431740 20 client.26318 trim_cache size 0 max 16384 2024-09-21T01:53:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.751+0000 7fce00431740 1 Processor -- start 2024-09-21T01:53:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.752+0000 7fce00431740 1 -- start start 2024-09-21T01:53:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.752+0000 7fce00431740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598441ff2f0 0x5598445cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.752+0000 7fce00431740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fce00431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598441ff2f0 0x5598445cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41558/0 (socket says 172.21.15.29:41558) 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 -- 172.21.15.29:0/760439056 learned_addr learned my addr 172.21.15.29:0/760439056 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14c210 con 0x5598442281b0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14de10 con 0x5598442074d0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14c8e0 con 0x5598441ff2f0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 -- 172.21.15.29:0/760439056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598441ff2f0 msgr2=0x5598445cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598441ff2f0 0x5598445cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 -- 172.21.15.29:0/760439056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 msgr2=0x5598445ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.753+0000 7fcdf27fc640 1 -- 172.21.15.29:0/760439056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598445d7a20 con 0x5598442074d0 2024-09-21T01:53:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.754+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ca120 tx=0x7fcde41046f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f95a8a8b6ac8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.754+0000 7fcdd8ff9640 1 -- 172.21.15.29:0/760439056 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde40ccc80 con 0x5598442074d0 2024-09-21T01:53:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.754+0000 7fcdd8ff9640 1 -- 172.21.15.29:0/760439056 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde4030040 con 0x5598442074d0 2024-09-21T01:53:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.754+0000 7fcdd8ff9640 1 -- 172.21.15.29:0/760439056 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde40cc620 con 0x5598442074d0 2024-09-21T01:53:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 msgr2=0x5598445e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ca120 tx=0x7fcde41046f0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 shutdown_connections 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598442281b0 0x5598445ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442074d0 0x5598445e6d60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 --2- 172.21.15.29:0/760439056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598441ff2f0 0x5598445cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 >> 172.21.15.29:0/760439056 conn(0x55984421c300 msgr2=0x5598445d8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 shutdown_connections 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.756+0000 7fce00431740 1 -- 172.21.15.29:0/760439056 wait complete. 2024-09-21T01:53:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.757+0000 7fce00431740 1 -- start start 2024-09-21T01:53:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.757+0000 7fce00431740 1 Processor -- start 2024-09-21T01:53:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445dd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14d740 con 0x5598441ff2f0 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14d070 con 0x5598442281b0 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec14a210 con 0x5598442074d0 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fcdf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37758/0 (socket says 172.21.15.29:37758) 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fce00431740 10 client.0 _ll_register_callbacks cb 0x7fcdf4075650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/3022274587 learned_addr learned my addr 172.21.15.29:0/3022274587 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.758+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445dd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/3022274587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 msgr2=0x5598445dcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445dcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 msgr2=0x5598445dd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445dd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 -- 172.21.15.29:0/3022274587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598445ddfb0 con 0x5598441ff2f0 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf37fe640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445dd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde4030040 con 0x5598441ff2f0 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdf2ffd640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40774d0 tx=0x7fcdf40806e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69324aff253b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.759+0000 7fcdd6ff5640 10 client.16584 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.760+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/3022274587 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4085d30 con 0x5598441ff2f0 2024-09-21T01:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.760+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/3022274587 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcdf408e5e0 con 0x5598441ff2f0 2024-09-21T01:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.760+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/3022274587 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdf408e7e0 con 0x5598441ff2f0 2024-09-21T01:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.761+0000 7fce00431740 10 client.16584 fetch_fsmap learned FSMap version 39 2024-09-21T01:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.761+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5598445e11f0 con 0x5598441ff2f0 2024-09-21T01:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.762+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/3022274587 <== mon.0 v2:172.21.15.29:3300/0 4 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcdf408c020 con 0x5598441ff2f0 2024-09-21T01:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 10 client.16584 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T01:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 20 client.16584 populate_metadata read hostname 'smithi029' 2024-09-21T01:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441bc1c0 0x559844653b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 4 client.16584 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559844544640 con 0x5598441bc1c0 2024-09-21T01:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.763+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441bc1c0 0x559844653b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.764+0000 7fcdf27fc640 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441bc1c0 0x559844653b20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862d76718cc2a67c server_cookie=20ae751ab85d5ce in_seq=0 out_seq=0 2024-09-21T01:53:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.764+0000 7fcdd6ff5640 10 client.16584 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.779+0000 7fcdd6ff5640 1 -- 172.21.15.29:0/3022274587 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559844544640 con 0x5598441bc1c0 2024-09-21T01:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.779+0000 7fcdd6ff5640 10 client.16584 handle_command_reply: tid=0 2024-09-21T01:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 client.16584 shutdown 2024-09-21T01:53:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 msgr2=0x5598445dc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40774d0 tx=0x7fcdf40806e0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 shutdown_connections 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5598441bc1c0 0x559844653b20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.780+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5598442281b0 0x5598445dd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fcdd6ff5640 0 client.16584 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598442074d0 0x5598445dcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 1 --2- 172.21.15.29:0/3022274587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598441ff2f0 0x5598445dc620 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 >> 172.21.15.29:0/3022274587 conn(0x55984421c300 msgr2=0x5598445cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 shutdown_connections 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 1 -- 172.21.15.29:0/3022274587 wait complete. 2024-09-21T01:53:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.781+0000 7fce00431740 20 client.16584 trim_cache size 0 max 16384 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.782+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 msgr2=0x7fcddc0b8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.782+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 0x7fcddc0b8970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4068db0 tx=0x7fcdf40692f0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.783+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 msgr2=0x7fcdec13fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.783+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec13fce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcde402c8b0 tx=0x7fcde4034a00 comp rx=0 tx=0).stop 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 shutdown_connections 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcddc0b6460 0x7fcddc0b8970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdec0ab980 0x7fcdec140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdec0a8d90 0x7fcdec13fce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 --2- 172.21.15.29:0/1435211414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdec0a6430 0x7fcdec13f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:53:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 >> 172.21.15.29:0/1435211414 conn(0x7fcdec0a0480 msgr2=0x7fcdec0a8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:53:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 shutdown_connections 2024-09-21T01:53:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:53:54.784+0000 7fcdf3fff640 1 -- 172.21.15.29:0/1435211414 wait complete. 2024-09-21T01:53:55.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:55 smithi190 ceph-mon[34368]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T01:53:55.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:55 smithi066 ceph-mon[35001]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T01:53:55.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:55 smithi029 ceph-mon[31228]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T01:53:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:57 smithi190 ceph-mon[34368]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 995 op/s 2024-09-21T01:53:57.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:57 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:57.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:57 smithi066 ceph-mon[35001]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 995 op/s 2024-09-21T01:53:57.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:57 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:57.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:57 smithi029 ceph-mon[31228]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 995 op/s 2024-09-21T01:53:57.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:57 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:53:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 320.0 s, 2927.2 tps, lat 34.198 ms stddev 40.056 2024-09-21T01:53:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:53:59 smithi190 ceph-mon[34368]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 620 op/s 2024-09-21T01:53:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:53:59 smithi066 ceph-mon[35001]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 620 op/s 2024-09-21T01:53:59.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:53:59 smithi029 ceph-mon[31228]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 620 op/s 2024-09-21T01:54:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:01 smithi190 ceph-mon[34368]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 867 op/s 2024-09-21T01:54:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:01 smithi066 ceph-mon[35001]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 867 op/s 2024-09-21T01:54:01.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:01 smithi029 ceph-mon[31228]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 867 op/s 2024-09-21T01:54:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 325.0 s, 3637.5 tps, lat 27.406 ms stddev 30.306 2024-09-21T01:54:03.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:03 smithi190 ceph-mon[34368]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 681 op/s 2024-09-21T01:54:03.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:03 smithi066 ceph-mon[35001]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 681 op/s 2024-09-21T01:54:03.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:03 smithi029 ceph-mon[31228]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 681 op/s 2024-09-21T01:54:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:05 smithi190 ceph-mon[34368]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 679 op/s 2024-09-21T01:54:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:05 smithi066 ceph-mon[35001]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 679 op/s 2024-09-21T01:54:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:05 smithi029 ceph-mon[31228]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 679 op/s 2024-09-21T01:54:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:07 smithi190 ceph-mon[34368]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-09-21T01:54:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:07 smithi066 ceph-mon[35001]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-09-21T01:54:07.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:07 smithi029 ceph-mon[31228]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 -- 172.21.15.29:0/3181868697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 msgr2=0x7f14c0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 --2- 172.21.15.29:0/3181868697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0101910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009900 tx=0x7f14b002a410 comp rx=0 tx=0).stop 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 -- 172.21.15.29:0/3181868697 shutdown_connections 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 --2- 172.21.15.29:0/3181868697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0102eb0 0x7f14c01033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 --2- 172.21.15.29:0/3181868697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0101f50 0x7f14c0102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 --2- 172.21.15.29:0/3181868697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0101910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.933+0000 7f14c5de6640 1 -- 172.21.15.29:0/3181868697 >> 172.21.15.29:0/3181868697 conn(0x7f14c00fd270 msgr2=0x7f14c00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.934+0000 7f14c5de6640 1 -- 172.21.15.29:0/3181868697 shutdown_connections 2024-09-21T01:54:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.934+0000 7f14c5de6640 1 -- 172.21.15.29:0/3181868697 wait complete. 2024-09-21T01:54:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 Processor -- start 2024-09-21T01:54:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 -- start start 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 0x7f14c010a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 0x7f14c01a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0104580 con 0x7f14c0101f50 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0104400 con 0x7f14c0069f00 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.936+0000 7f14c5de6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0104700 con 0x7f14c0102eb0 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 0x7f14c010a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49126/0 (socket says 172.21.15.29:49126) 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 0x7f14c010a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51756/0 (socket says 172.21.15.29:51756) 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 0x7f14c01a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bf7fe640 1 -- 172.21.15.29:0/1384903031 learned_addr learned my addr 172.21.15.29:0/1384903031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 -- 172.21.15.29:0/1384903031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 msgr2=0x7f14c0109fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 -- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 msgr2=0x7f14c010a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 0x7f14c010a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 -- 172.21.15.29:0/1384903031 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c01a1fb0 con 0x7f14c0102eb0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.937+0000 7f14bffff640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 0x7f14c01a1790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009fd0 tx=0x7f14b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.938+0000 7f14bcff9640 1 -- 172.21.15.29:0/1384903031 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b800ce20 con 0x7f14c0102eb0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.938+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c01a22a0 con 0x7f14c0102eb0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.938+0000 7f14bcff9640 1 -- 172.21.15.29:0/1384903031 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b8010430 con 0x7f14c0102eb0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.938+0000 7f14bcff9640 1 -- 172.21.15.29:0/1384903031 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b800f620 con 0x7f14c0102eb0 2024-09-21T01:54:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.938+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01a8990 con 0x7f14c0102eb0 2024-09-21T01:54:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.942+0000 7f14bcff9640 1 -- 172.21.15.29:0/1384903031 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f14b800f7c0 con 0x7f14c0102eb0 2024-09-21T01:54:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.943+0000 7f14cce2f740 1 Processor -- start 2024-09-21T01:54:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.943+0000 7f14cce2f740 1 -- start start 2024-09-21T01:54:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.943+0000 7f14bcff9640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 0x7f14a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.943+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d74de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d3854d0 0x55f88d74f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14cce2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f88d6c06f0 con 0x55f88d3a61b0 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14cce2f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f88d6afb20 con 0x55f88d37d2f0 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14cce2f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f88d6b3930 con 0x55f88d3854d0 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 0x7f14a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14bcff9640 1 -- 172.21.15.29:0/1384903031 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f14b80e7630 con 0x7f14c0102eb0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49138/0 (socket says 172.21.15.29:49138) 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 -- 172.21.15.29:0/979194260 learned_addr learned my addr 172.21.15.29:0/979194260 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d74de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14bffff640 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d3854d0 0x55f88d74f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 -- 172.21.15.29:0/979194260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d3854d0 msgr2=0x55f88d74f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d3854d0 0x55f88d74f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 -- 172.21.15.29:0/979194260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 msgr2=0x55f88d74de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d74de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.944+0000 7f14beffd640 1 -- 172.21.15.29:0/979194260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88d675530 con 0x55f88d37d2f0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.945+0000 7f14beffd640 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14b40137b0 tx=0x7f14b4013c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61a2dce0905761f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.945+0000 7f14a5ffb640 1 -- 172.21.15.29:0/979194260 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4005250 con 0x55f88d37d2f0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.945+0000 7f14a5ffb640 1 -- 172.21.15.29:0/979194260 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4023070 con 0x55f88d37d2f0 2024-09-21T01:54:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.945+0000 7f14a5ffb640 1 -- 172.21.15.29:0/979194260 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004730 con 0x55f88d37d2f0 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 -- 172.21.15.29:0/979194260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 msgr2=0x55f88d74e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14beffd640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 0x7f14a80b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f14b4022000 tx=0x7f14b401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14b40137b0 tx=0x7f14b4013c70 comp rx=0 tx=0).stop 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 -- 172.21.15.29:0/979194260 shutdown_connections 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d3854d0 0x55f88d74f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d74e4c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 --2- 172.21.15.29:0/979194260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d74de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 -- 172.21.15.29:0/979194260 >> 172.21.15.29:0/979194260 conn(0x55f88d39a300 msgr2=0x55f88d7498e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 -- 172.21.15.29:0/979194260 shutdown_connections 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.946+0000 7f14cce2f740 1 -- 172.21.15.29:0/979194260 wait complete. 2024-09-21T01:54:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.947+0000 7f14cce2f740 1 -- start start 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.947+0000 7f14cce2f740 1 Processor -- start 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.947+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d7685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d769810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f88d5e7ab0 con 0x55f88d3a61b0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f88d655730 con 0x55f88d3854d0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f88d74fb30 con 0x55f88d37d2f0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14cce2f740 10 client.0 _ll_register_callbacks cb 0x55f88d6a96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14bffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d769810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49152/0 (socket says 172.21.15.29:49152) 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 -- 172.21.15.29:0/3356993285 learned_addr learned my addr 172.21.15.29:0/3356993285 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d7685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 -- 172.21.15.29:0/3356993285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 msgr2=0x55f88d7685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d7685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 -- 172.21.15.29:0/3356993285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 msgr2=0x55f88d769810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d769810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 -- 172.21.15.29:0/3356993285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88d411790 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f14beffd640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14b4011fc0 tx=0x7f14b40097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa51d01a39278a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.948+0000 7f149effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4037060 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f14b401e7d0 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4023070 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14b4005050 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f14cce2f740 10 client.26336 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f88d48a620 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4023070 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f14b4031030 con 0x55f88d3854d0 2024-09-21T01:54:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.949+0000 7f14cce2f740 10 client.26336 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14cce2f740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14cce2f740 20 client.26336 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d364d80 0x55f88d775b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14cce2f740 4 client.26336 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88d6c2640 con 0x55f88d364d80 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d364d80 0x55f88d775b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f149effd640 10 client.26336 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.950+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d364d80 0x55f88d775b00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b6fcc64a40b7ef server_cookie=abd2485345ce1e6d in_seq=0 out_seq=0 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f149effd640 1 -- 172.21.15.29:0/3356993285 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f88d6c2640 con 0x55f88d364d80 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f149effd640 10 client.26336 handle_command_reply: tid=0 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 client.26336 shutdown 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 msgr2=0x55f88d768ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14b4011fc0 tx=0x7f14b40097a0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 shutdown_connections 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d364d80 0x55f88d775b00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f149effd640 0 client.26336 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3a61b0 0x55f88d769810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d3854d0 0x55f88d768ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 --2- 172.21.15.29:0/3356993285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d7685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.951+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 >> 172.21.15.29:0/3356993285 conn(0x55f88d39a300 msgr2=0x55f88d747ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.952+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 shutdown_connections 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.952+0000 7f14cce2f740 1 -- 172.21.15.29:0/3356993285 wait complete. 2024-09-21T01:54:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.952+0000 7f14cce2f740 20 client.26336 trim_cache size 0 max 16384 2024-09-21T01:54:07.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 330.0 s, 3360.6 tps, lat 29.694 ms stddev 34.873 2024-09-21T01:54:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.974+0000 7f14cce2f740 1 Processor -- start 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.974+0000 7f14cce2f740 1 -- start start 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.974+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 0x55f88d7445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.974+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.974+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 0x55f88d7445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 0x55f88d7445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52426/0 (socket says 172.21.15.29:52426) 2024-09-21T01:54:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49168/0 (socket says 172.21.15.29:49168) 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14bf7fe640 1 -- 172.21.15.29:0/446198488 learned_addr learned my addr 172.21.15.29:0/446198488 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14cce2f740 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d74a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01a8e30 con 0x55f88d3854d0 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01aa9a0 con 0x55f88d37d2f0 2024-09-21T01:54:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01a9500 con 0x55f88d364d80 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 -- 172.21.15.29:0/446198488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 msgr2=0x55f88d7445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 0x55f88d7445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 -- 172.21.15.29:0/446198488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 msgr2=0x55f88d74a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d74a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.975+0000 7f14beffd640 1 -- 172.21.15.29:0/446198488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88d744f50 con 0x55f88d37d2f0 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.976+0000 7f14beffd640 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14b4012010 tx=0x7f14b40092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40c8178b1e1c1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.976+0000 7f14a57fa640 1 -- 172.21.15.29:0/446198488 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4042070 con 0x55f88d37d2f0 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.976+0000 7f14a57fa640 1 -- 172.21.15.29:0/446198488 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4023040 con 0x55f88d37d2f0 2024-09-21T01:54:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.976+0000 7f14a57fa640 1 -- 172.21.15.29:0/446198488 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4037070 con 0x55f88d37d2f0 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.977+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 msgr2=0x55f88d749dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.977+0000 7f14cce2f740 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14b4012010 tx=0x7f14b40092a0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 shutdown_connections 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d74a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d37d2f0 0x55f88d749dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 --2- 172.21.15.29:0/446198488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d364d80 0x55f88d7445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 >> 172.21.15.29:0/446198488 conn(0x55f88d39a300 msgr2=0x55f88d749220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 shutdown_connections 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 -- 172.21.15.29:0/446198488 wait complete. 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 -- start start 2024-09-21T01:54:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.979+0000 7f14cce2f740 1 Processor -- start 2024-09-21T01:54:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d76dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d76e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01aa2d0 con 0x55f88d3854d0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01a9c00 con 0x55f88d364d80 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14c019f720 con 0x55f88d37d2f0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14cce2f740 10 client.0 _ll_register_callbacks cb 0x7f14b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49182/0 (socket says 172.21.15.29:49182) 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 -- 172.21.15.29:0/1620372487 learned_addr learned my addr 172.21.15.29:0/1620372487 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14beffd640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d76dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bffff640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d76e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 -- 172.21.15.29:0/1620372487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 msgr2=0x55f88d76dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d76dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 -- 172.21.15.29:0/1620372487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 msgr2=0x55f88d76e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d76e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 -- 172.21.15.29:0/1620372487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88d76ed80 con 0x55f88d364d80 2024-09-21T01:54:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f14bf7fe640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14b0004360 tx=0x7f14b0004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1666a3d6ff82b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.980+0000 7f149effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0035040 con 0x55f88d364d80 2024-09-21T01:54:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f14b0003a00 con 0x55f88d364d80 2024-09-21T01:54:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4023040 con 0x55f88d364d80 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14b00044b0 con 0x55f88d364d80 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f14cce2f740 10 client.26342 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.981+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f88d770110 con 0x55f88d364d80 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0003c00 con 0x55f88d364d80 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f14b004d430 con 0x55f88d364d80 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f14cce2f740 10 client.26342 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f14cce2f740 10 client.26342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f14cce2f740 20 client.26342 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d3a61b0 0x55f88d7e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.982+0000 7f14cce2f740 4 client.26342 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.983+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f14b4023040 con 0x55f88d3a61b0 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.983+0000 7f14beffd640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d3a61b0 0x55f88d7e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.983+0000 7f149effd640 10 client.26342 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.983+0000 7f14beffd640 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d3a61b0 0x55f88d7e2920 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e615205b5529d1 server_cookie=85df0aa654ee5ad8 in_seq=0 out_seq=0 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f149effd640 1 -- 172.21.15.29:0/1620372487 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f14b4023040 con 0x55f88d3a61b0 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f149effd640 10 client.26342 handle_command_reply: tid=0 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 client.26342 shutdown 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 msgr2=0x55f88d76d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14b0004360 tx=0x7f14b0004390 comp rx=0 tx=0).stop 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 shutdown_connections 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f88d3a61b0 0x55f88d7e2920 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f149effd640 0 client.26342 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f88d3854d0 0x55f88d76e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f88d37d2f0 0x55f88d76dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 --2- 172.21.15.29:0/1620372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88d364d80 0x55f88d76d3f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 >> 172.21.15.29:0/1620372487 conn(0x55f88d39a300 msgr2=0x55f88d7445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 shutdown_connections 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 1 -- 172.21.15.29:0/1620372487 wait complete. 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.996+0000 7f14cce2f740 20 client.26342 trim_cache size 0 max 16384 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:54:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 msgr2=0x7f14a80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 0x7f14a80b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f14b4022000 tx=0x7f14b401e000 comp rx=0 tx=0).stop 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 msgr2=0x7f14c01a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 0x7f14c01a1790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009fd0 tx=0x7f14b800ef10 comp rx=0 tx=0).stop 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 shutdown_connections 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f14a80b6510 0x7f14a80b8a20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0102eb0 0x7f14c01a1790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14c0101f50 0x7f14c010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 --2- 172.21.15.29:0/1384903031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14c0069f00 0x7f14c0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 >> 172.21.15.29:0/1384903031 conn(0x7f14c00fd270 msgr2=0x7f14c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 shutdown_connections 2024-09-21T01:54:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:07.999+0000 7f14c5de6640 1 -- 172.21.15.29:0/1384903031 wait complete. 2024-09-21T01:54:09.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:09 smithi190 ceph-mon[34368]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-21T01:54:09.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:09 smithi066 ceph-mon[35001]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-21T01:54:09.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:09 smithi029 ceph-mon[31228]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-21T01:54:11.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:11 smithi190 ceph-mon[34368]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-21T01:54:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:11 smithi066 ceph-mon[35001]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-21T01:54:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:11 smithi029 ceph-mon[31228]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-21T01:54:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 335.0 s, 2930.8 tps, lat 34.212 ms stddev 41.870 2024-09-21T01:54:13.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:13 smithi190 ceph-mon[34368]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-09-21T01:54:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:13 smithi066 ceph-mon[35001]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-09-21T01:54:13.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:13 smithi029 ceph-mon[31228]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-09-21T01:54:15.146 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:54:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:15.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:15 smithi190 ceph-mon[34368]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T01:54:15.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:15 smithi066 ceph-mon[35001]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T01:54:15.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:15 smithi029 ceph-mon[31228]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T01:54:16.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.193+0000 7f871135f640 1 -- 172.21.15.29:0/4218209750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040ab1e0 msgr2=0x7f87040aba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.193+0000 7f871135f640 1 --2- 172.21.15.29:0/4218209750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040ab1e0 0x7f87040aba00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f870c066480 tx=0x7f870c08f980 comp rx=0 tx=0).stop 2024-09-21T01:54:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.197+0000 7f871135f640 1 -- 172.21.15.29:0/4218209750 shutdown_connections 2024-09-21T01:54:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.197+0000 7f871135f640 1 --2- 172.21.15.29:0/4218209750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040ab1e0 0x7f87040aba00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.197+0000 7f871135f640 1 --2- 172.21.15.29:0/4218209750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040aa890 0x7f87040aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.197+0000 7f871135f640 1 --2- 172.21.15.29:0/4218209750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040a5120 0x7f87040a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.197+0000 7f871135f640 1 -- 172.21.15.29:0/4218209750 >> 172.21.15.29:0/4218209750 conn(0x7f870401cca0 msgr2=0x7f870401d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.198+0000 7f871135f640 1 -- 172.21.15.29:0/4218209750 shutdown_connections 2024-09-21T01:54:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.198+0000 7f871135f640 1 -- 172.21.15.29:0/4218209750 wait complete. 2024-09-21T01:54:16.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.199+0000 7f871135f640 1 Processor -- start 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.199+0000 7f871135f640 1 -- start start 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040a5120 0x7f87041440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ab1e0 0x7f8704149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87040bd650 con 0x7f87040a5120 2024-09-21T01:54:16.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87040bd4d0 con 0x7f87040ab1e0 2024-09-21T01:54:16.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.200+0000 7f871135f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87040bd7d0 con 0x7f87040aa890 2024-09-21T01:54:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.201+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.201+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ab1e0 0x7f8704149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.201+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33986/0 (socket says 172.21.15.29:33986) 2024-09-21T01:54:16.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.201+0000 7f870a7fc640 1 -- 172.21.15.29:0/2580218850 learned_addr learned my addr 172.21.15.29:0/2580218850 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.201+0000 7f870affd640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040a5120 0x7f87041440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 -- 172.21.15.29:0/2580218850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ab1e0 msgr2=0x7f8704149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ab1e0 0x7f8704149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 -- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040a5120 msgr2=0x7f87041440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040a5120 0x7f87041440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 -- 172.21.15.29:0/2580218850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870414a050 con 0x7f87040aa890 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.202+0000 7f870a7fc640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f870000e9f0 tx=0x7f870000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.203+0000 7f86f3fff640 1 -- 172.21.15.29:0/2580218850 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870000ccc0 con 0x7f87040aa890 2024-09-21T01:54:16.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.203+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870414a340 con 0x7f87040aa890 2024-09-21T01:54:16.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.203+0000 7f86f3fff640 1 -- 172.21.15.29:0/2580218850 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87000108e0 con 0x7f87040aa890 2024-09-21T01:54:16.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.203+0000 7f86f3fff640 1 -- 172.21.15.29:0/2580218850 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000188e0 con 0x7f87040aa890 2024-09-21T01:54:16.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.203+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8704150aa0 con 0x7f87040aa890 2024-09-21T01:54:16.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.206+0000 7f87178d3740 1 Processor -- start 2024-09-21T01:54:16.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.206+0000 7f87178d3740 1 -- start start 2024-09-21T01:54:16.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e2218950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e50390 0x5613e2218e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5613e218ee50 con 0x5613e1e50390 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5613e217e280 con 0x5613e1e71070 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f87178d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5613e2182090 con 0x5613e1e481b0 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54336/0 (socket says 172.21.15.29:54336) 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f870b7fe640 1 -- 172.21.15.29:0/3107523359 learned_addr learned my addr 172.21.15.29:0/3107523359 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.207+0000 7f870affd640 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e2218950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 -- 172.21.15.29:0/3107523359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 msgr2=0x5613e2218950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e2218950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 -- 172.21.15.29:0/3107523359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e50390 msgr2=0x5613e2218e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e50390 0x5613e2218e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 -- 172.21.15.29:0/3107523359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e1edc650 con 0x5613e1e71070 2024-09-21T01:54:16.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f870b7fe640 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f870c069050 tx=0x7f870c069570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9670e77d436b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.208+0000 7f86f17fa640 1 -- 172.21.15.29:0/3107523359 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c06c420 con 0x5613e1e71070 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.209+0000 7f86f17fa640 1 -- 172.21.15.29:0/3107523359 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c063cb0 con 0x5613e1e71070 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.209+0000 7f86f17fa640 1 -- 172.21.15.29:0/3107523359 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c06a5d0 con 0x5613e1e71070 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.209+0000 7f86f3fff640 1 -- 172.21.15.29:0/2580218850 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8700010430 con 0x7f87040aa890 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f86f3fff640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 0x7f86f40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f87178d3740 1 -- 172.21.15.29:0/3107523359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 msgr2=0x5613e2218410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f87178d3740 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f870c069050 tx=0x7f870c069570 comp rx=0 tx=0).stop 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f86f3fff640 1 -- 172.21.15.29:0/2580218850 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f87000e75c0 con 0x7f87040aa890 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f870affd640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 0x7f86f40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.210+0000 7f87178d3740 1 -- 172.21.15.29:0/3107523359 shutdown_connections 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e50390 0x5613e2218e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e2218950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 --2- 172.21.15.29:0/3107523359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e2218410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 -- 172.21.15.29:0/3107523359 >> 172.21.15.29:0/3107523359 conn(0x5613e1e65460 msgr2=0x5613e2216300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 -- 172.21.15.29:0/3107523359 shutdown_connections 2024-09-21T01:54:16.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 -- 172.21.15.29:0/3107523359 wait complete. 2024-09-21T01:54:16.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f870affd640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 0x7f86f40b8b60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5613e2217090 tx=0x7f86fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.211+0000 7f87178d3740 1 -- start start 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 Processor -- start 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2223bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2224130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5613e1fdf700 con 0x5613e1e481b0 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5613e2119110 con 0x5613e1e71070 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5613e2226b90 con 0x5613e1e50390 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.212+0000 7f87178d3740 10 client.0 _ll_register_callbacks cb 0x5613e2132990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54352/0 (socket says 172.21.15.29:54352) 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 -- 172.21.15.29:0/1752565169 learned_addr learned my addr 172.21.15.29:0/1752565169 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870a7fc640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2224130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 -- 172.21.15.29:0/1752565169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 msgr2=0x5613e2224130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2224130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 -- 172.21.15.29:0/1752565169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 msgr2=0x5613e2223bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2223bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 -- 172.21.15.29:0/1752565169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e2146c40 con 0x5613e1e71070 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f870c000b60 tx=0x7f870c064300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85356c0257b892c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.213+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0b9060 con 0x5613e1e71070 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f870c063cb0 con 0x5613e1e71070 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f870c06a5d0 con 0x5613e1e71070 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f870c064700 con 0x5613e1e71070 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f87178d3740 10 client.26351 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5613e2232980 con 0x5613e1e71070 2024-09-21T01:54:16.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.214+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c063cb0 con 0x5613e1e71070 2024-09-21T01:54:16.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f870c095770 con 0x5613e1e71070 2024-09-21T01:54:16.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 10 client.26351 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:16.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:16.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 20 client.26351 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e2f9f0 0x5613e2235e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 4 client.26351 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e2190da0 con 0x5613e1e2f9f0 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.215+0000 7f870affd640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e2f9f0 0x5613e2235e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.216+0000 7f870affd640 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e2f9f0 0x5613e2235e10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848fc27e621705f0 server_cookie=a98bcd500c7c043a in_seq=0 out_seq=0 2024-09-21T01:54:16.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.216+0000 7f86eeff5640 10 client.26351 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.217+0000 7f86eeff5640 1 -- 172.21.15.29:0/1752565169 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613e2190da0 con 0x5613e1e2f9f0 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.217+0000 7f86eeff5640 10 client.26351 handle_command_reply: tid=0 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.217+0000 7f87178d3740 1 client.26351 shutdown 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 msgr2=0x5613e222fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f870c000b60 tx=0x7f870c064300 comp rx=0 tx=0).stop 2024-09-21T01:54:16.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 shutdown_connections 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e2f9f0 0x5613e2235e10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e71070 0x5613e222fee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2224130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 --2- 172.21.15.29:0/1752565169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2223bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 >> 172.21.15.29:0/1752565169 conn(0x5613e1e65460 msgr2=0x5613e221b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f86eeff5640 0 client.26351 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.218+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 shutdown_connections 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.219+0000 7f87178d3740 1 -- 172.21.15.29:0/1752565169 wait complete. 2024-09-21T01:54:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.219+0000 7f87178d3740 20 client.26351 trim_cache size 0 max 16384 2024-09-21T01:54:16.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.244+0000 7f87178d3740 1 Processor -- start 2024-09-21T01:54:16.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 -- start start 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e65460 0x5613e2219e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2220de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2221320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800e3d0 con 0x5613e1e481b0 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800ff40 con 0x5613e1e65460 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f87178d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800eaa0 con 0x5613e1e50390 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2220de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2220de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60762/0 (socket says 172.21.15.29:60762) 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f870a7fc640 1 -- 172.21.15.29:0/874856927 learned_addr learned my addr 172.21.15.29:0/874856927 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f870b7fe640 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2221320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.245+0000 7f870affd640 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e65460 0x5613e2219e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 -- 172.21.15.29:0/874856927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e65460 msgr2=0x5613e2219e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e65460 0x5613e2219e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 -- 172.21.15.29:0/874856927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 msgr2=0x5613e2220de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2220de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 -- 172.21.15.29:0/874856927 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e2224eb0 con 0x5613e1e50390 2024-09-21T01:54:16.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.246+0000 7f870b7fe640 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2221320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f870c052910 tx=0x7f870c08f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e549eefa9cad013d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.247+0000 7f86f0ff9640 1 -- 172.21.15.29:0/874856927 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0c4070 con 0x5613e1e50390 2024-09-21T01:54:16.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.247+0000 7f86f0ff9640 1 -- 172.21.15.29:0/874856927 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c095dd0 con 0x5613e1e50390 2024-09-21T01:54:16.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.247+0000 7f86f0ff9640 1 -- 172.21.15.29:0/874856927 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0b9070 con 0x5613e1e50390 2024-09-21T01:54:16.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 -- 172.21.15.29:0/874856927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 msgr2=0x5613e2221320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2221320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f870c052910 tx=0x7f870c08f980 comp rx=0 tx=0).stop 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 -- 172.21.15.29:0/874856927 shutdown_connections 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e50390 0x5613e2221320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e481b0 0x5613e2220de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 --2- 172.21.15.29:0/874856927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e65460 0x5613e2219e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 -- 172.21.15.29:0/874856927 >> 172.21.15.29:0/874856927 conn(0x5613e22114f0 msgr2=0x5613e221bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 -- 172.21.15.29:0/874856927 shutdown_connections 2024-09-21T01:54:16.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.248+0000 7f87178d3740 1 -- 172.21.15.29:0/874856927 wait complete. 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 -- start start 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 Processor -- start 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e222a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 0x5613e222aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e65460 0x5613e229f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800f870 con 0x5613e1e65460 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800f1a0 con 0x5613e1e50390 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.249+0000 7f87178d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86f800c3b0 con 0x5613e1e481b0 2024-09-21T01:54:16.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f87178d3740 10 client.0 _ll_register_callbacks cb 0x7f86fc00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 0x5613e222aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 0x5613e222aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54368/0 (socket says 172.21.15.29:54368) 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e222a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870a7fc640 1 -- 172.21.15.29:0/4056023503 learned_addr learned my addr 172.21.15.29:0/4056023503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870b7fe640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e65460 0x5613e229f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 -- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 msgr2=0x5613e222aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 0x5613e222aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 -- 172.21.15.29:0/4056023503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e65460 msgr2=0x5613e229f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e65460 0x5613e229f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.250+0000 7f870affd640 1 -- 172.21.15.29:0/4056023503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e229fd60 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.251+0000 7f870affd640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e222a540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5613e2236370 tx=0x7f86fc016850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44fd011168e31c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.251+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc02e060 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f86fc005b80 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f870c095dd0 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f86fc01c030 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f87178d3740 10 client.26389 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5613e22a1020 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.252+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc01d040 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f86fc0144c0 con 0x5613e1e481b0 2024-09-21T01:54:16.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 10 client.26389 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 10 client.26389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 20 client.26389 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e71070 0x5613e22a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 4 client.26389 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f870c095dd0 con 0x5613e1e71070 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f870a7fc640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e71070 0x5613e22a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f870a7fc640 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e71070 0x5613e22a37c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787eb69a2c139c01 server_cookie=2c9da1058ef70d27 in_seq=0 out_seq=0 2024-09-21T01:54:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.253+0000 7f86eeff5640 10 client.26389 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:16.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.254+0000 7f86eeff5640 1 -- 172.21.15.29:0/4056023503 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x7f870c095dd0 con 0x5613e1e71070 2024-09-21T01:54:16.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.254+0000 7f86eeff5640 10 client.26389 handle_command_reply: tid=0 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.254+0000 7f87178d3740 1 client.26389 shutdown 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 msgr2=0x5613e222a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e222a540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5613e2236370 tx=0x7f86fc016850 comp rx=0 tx=0).stop 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 shutdown_connections 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5613e1e71070 0x5613e22a37c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5613e1e65460 0x5613e229f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613e1e50390 0x5613e222aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 --2- 172.21.15.29:0/4056023503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5613e1e481b0 0x5613e222a540 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f86eeff5640 0 client.26389 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.255+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 >> 172.21.15.29:0/4056023503 conn(0x5613e22114f0 msgr2=0x5613e2225080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.256+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 shutdown_connections 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.256+0000 7f87178d3740 1 -- 172.21.15.29:0/4056023503 wait complete. 2024-09-21T01:54:16.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.256+0000 7f87178d3740 20 client.26389 trim_cache size 0 max 16384 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.259 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:16.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.256+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 msgr2=0x7f86f40b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.256+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 0x7f86f40b8b60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5613e2217090 tx=0x7f86fc008040 comp rx=0 tx=0).stop 2024-09-21T01:54:16.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 msgr2=0x7f8704144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f870000e9f0 tx=0x7f870000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 shutdown_connections 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f40b6650 0x7f86f40b8b60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ab1e0 0x7f8704149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87040aa890 0x7f8704144680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 --2- 172.21.15.29:0/2580218850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87040a5120 0x7f87041440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 >> 172.21.15.29:0/2580218850 conn(0x7f870401cca0 msgr2=0x7f8704004900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 shutdown_connections 2024-09-21T01:54:16.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:16.257+0000 7f871135f640 1 -- 172.21.15.29:0/2580218850 wait complete. 2024-09-21T01:54:17.280 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:54:17.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:54:17.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:17 smithi029 ceph-mon[31228]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 892 op/s 2024-09-21T01:54:17.725 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:17.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:17 smithi190 ceph-mon[34368]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 892 op/s 2024-09-21T01:54:17.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:17 smithi066 ceph-mon[35001]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 892 op/s 2024-09-21T01:54:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 340.0 s, 3541.3 tps, lat 28.323 ms stddev 37.422 2024-09-21T01:54:18.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.387+0000 7f8432e77640 1 -- 172.21.15.29:0/1507315632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c103b90 msgr2=0x7f842c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:18.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.387+0000 7f8432e77640 1 --2- 172.21.15.29:0/1507315632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c103b90 0x7f842c103fa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.389+0000 7f8432e77640 1 -- 172.21.15.29:0/1507315632 shutdown_connections 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.389+0000 7f8432e77640 1 --2- 172.21.15.29:0/1507315632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c109e20 0x7f842c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.389+0000 7f8432e77640 1 --2- 172.21.15.29:0/1507315632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c1094d0 0x7f842c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.389+0000 7f8432e77640 1 --2- 172.21.15.29:0/1507315632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c103b90 0x7f842c103fa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.389+0000 7f8432e77640 1 -- 172.21.15.29:0/1507315632 >> 172.21.15.29:0/1507315632 conn(0x7f842c0ff600 msgr2=0x7f842c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.390+0000 7f8432e77640 1 -- 172.21.15.29:0/1507315632 shutdown_connections 2024-09-21T01:54:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.390+0000 7f8432e77640 1 -- 172.21.15.29:0/1507315632 wait complete. 2024-09-21T01:54:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 Processor -- start 2024-09-21T01:54:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 -- start start 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 0x7f842c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1094d0 0x7f842c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 0x7f842c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f842c113ae0 con 0x7f842c103b90 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f842c113960 con 0x7f842c1094d0 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8432e77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f842c113c60 con 0x7f842c109e20 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8430c13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 0x7f842c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.391+0000 7f8430c13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 0x7f842c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60784/0 (socket says 172.21.15.29:60784) 2024-09-21T01:54:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8430c13640 1 -- 172.21.15.29:0/4174925009 learned_addr learned my addr 172.21.15.29:0/4174925009 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 0x7f842c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 -- 172.21.15.29:0/4174925009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1094d0 msgr2=0x7f842c075c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1094d0 0x7f842c075c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 -- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 msgr2=0x7f842c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 0x7f842c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 -- 172.21.15.29:0/4174925009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a8770 con 0x7f842c109e20 2024-09-21T01:54:18.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.392+0000 7f8431414640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 0x7f842c076190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8424009fd0 tx=0x7f842400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:18.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.393+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400ce20 con 0x7f842c109e20 2024-09-21T01:54:18.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.393+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8424005020 con 0x7f842c109e20 2024-09-21T01:54:18.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.393+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424010690 con 0x7f842c109e20 2024-09-21T01:54:18.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.393+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a8a60 con 0x7f842c109e20 2024-09-21T01:54:18.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.393+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a8f50 con 0x7f842c109e20 2024-09-21T01:54:18.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.396+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84100052c0 con 0x7f842c109e20 2024-09-21T01:54:18.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.398+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8424010830 con 0x7f842c109e20 2024-09-21T01:54:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.399+0000 7f8429ffb640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 0x7f841c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:18.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.399+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f84240e7850 con 0x7f842c109e20 2024-09-21T01:54:18.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.400+0000 7f8430c13640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 0x7f841c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:18.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.401+0000 7f8430c13640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 0x7f841c0b8b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8420009230 tx=0x7f84200058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:18.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.403+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84240ae090 con 0x7f842c109e20 2024-09-21T01:54:18.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.838+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84100055b0 con 0x7f842c109e20 2024-09-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.839+0000 7f8429ffb640 1 -- 172.21.15.29:0/4174925009 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11971 (secure 0 0 0) 0x7f84240b2470 con 0x7f842c109e20 2024-09-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":40,"btime":"2024-09-21T01:53:56:421645+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:53:56.253083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 40 2024-09-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 msgr2=0x7f841c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 0x7f841c0b8b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8420009230 tx=0x7f84200058d0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 msgr2=0x7f842c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 0x7f842c076190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8424009fd0 tx=0x7f842400ef10 comp rx=0 tx=0).stop 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 shutdown_connections 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f841c0b6610 0x7f841c0b8b20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842c109e20 0x7f842c076190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1094d0 0x7f842c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 --2- 172.21.15.29:0/4174925009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c103b90 0x7f842c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 >> 172.21.15.29:0/4174925009 conn(0x7f842c0ff600 msgr2=0x7f842c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 shutdown_connections 2024-09-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:18.844+0000 7f8432e77640 1 -- 172.21.15.29:0/4174925009 wait complete. 2024-09-21T01:54:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:19 smithi066 ceph-mon[35001]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 625 op/s 2024-09-21T01:54:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:19 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/4174925009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:54:19.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:19 smithi029 ceph-mon[31228]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 625 op/s 2024-09-21T01:54:19.840 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:19 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/4174925009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:54:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:19 smithi190 ceph-mon[34368]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 625 op/s 2024-09-21T01:54:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:19 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/4174925009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:54:20.330 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:54:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 -- 172.21.15.29:0/1892352339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c104560 msgr2=0x7fc82c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 --2- 172.21.15.29:0/1892352339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c104560 0x7fc82c104970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009900 tx=0x7fc81c02a410 comp rx=0 tx=0).stop 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 -- 172.21.15.29:0/1892352339 shutdown_connections 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 --2- 172.21.15.29:0/1892352339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104f00 0x7fc82c10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 --2- 172.21.15.29:0/1892352339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c104560 0x7fc82c104970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 --2- 172.21.15.29:0/1892352339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82c1034e0 0x7fc82c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 -- 172.21.15.29:0/1892352339 >> 172.21.15.29:0/1892352339 conn(0x7fc82c0fee20 msgr2=0x7fc82c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 -- 172.21.15.29:0/1892352339 shutdown_connections 2024-09-21T01:54:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.161+0000 7fc83359d640 1 -- 172.21.15.29:0/1892352339 wait complete. 2024-09-21T01:54:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.162+0000 7fc83359d640 1 Processor -- start 2024-09-21T01:54:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.162+0000 7fc83359d640 1 -- start start 2024-09-21T01:54:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.162+0000 7fc83359d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.163+0000 7fc83359d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 0x7fc82c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.163+0000 7fc83359d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82c104f00 0x7fc82c10dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc83359d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c111410 con 0x7fc82c104560 2024-09-21T01:54:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc83359d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c111290 con 0x7fc82c1034e0 2024-09-21T01:54:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc83359d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c111590 con 0x7fc82c104f00 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc831d9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 0x7fc82c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc831d9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 0x7fc82c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60804/0 (socket says 172.21.15.29:60804) 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54388/0 (socket says 172.21.15.29:54388) 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.164+0000 7fc831d9a640 1 -- 172.21.15.29:0/655861052 learned_addr learned my addr 172.21.15.29:0/655861052 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 -- 172.21.15.29:0/655861052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82c104f00 msgr2=0x7fc82c10dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82c104f00 0x7fc82c10dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 -- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 msgr2=0x7fc82c10d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 0x7fc82c10d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 -- 172.21.15.29:0/655861052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c10e480 con 0x7fc82c1034e0 2024-09-21T01:54:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc83259b640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc82800d6d0 tx=0x7fc82800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.165+0000 7fc8237fe640 1 -- 172.21.15.29:0/655861052 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828017070 con 0x7fc82c1034e0 2024-09-21T01:54:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.166+0000 7fc8237fe640 1 -- 172.21.15.29:0/655861052 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc828009820 con 0x7fc82c1034e0 2024-09-21T01:54:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.166+0000 7fc8237fe640 1 -- 172.21.15.29:0/655861052 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828004c40 con 0x7fc82c1034e0 2024-09-21T01:54:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.166+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c10e770 con 0x7fc82c1034e0 2024-09-21T01:54:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.166+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c10ee30 con 0x7fc82c1034e0 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.169+0000 7fc83b843740 1 Processor -- start 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 -- start start 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b9466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b947200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f70b8b86f0 con 0x55f70b57d4d0 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f70b8a7b20 con 0x55f70b59e1b0 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc83b843740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f70b8ab930 con 0x55f70b5752f0 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc832d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc832d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54394/0 (socket says 172.21.15.29:54394) 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc832d9c640 1 -- 172.21.15.29:0/3988333272 learned_addr learned my addr 172.21.15.29:0/3988333272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.170+0000 7fc831d9a640 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b947200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 -- 172.21.15.29:0/3988333272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 msgr2=0x55f70b9466b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b9466b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 -- 172.21.15.29:0/3988333272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 msgr2=0x55f70b947200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b947200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 -- 172.21.15.29:0/3988333272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70b86d530 con 0x55f70b59e1b0 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.171+0000 7fc832d9c640 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc824009d00 tx=0x7fc82400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96606cc74f8bedf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.172+0000 7fc820ff9640 1 -- 172.21.15.29:0/3988333272 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400bd40 con 0x55f70b59e1b0 2024-09-21T01:54:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.172+0000 7fc8237fe640 1 -- 172.21.15.29:0/655861052 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc828010460 con 0x7fc82c1034e0 2024-09-21T01:54:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.172+0000 7fc820ff9640 1 -- 172.21.15.29:0/3988333272 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc824017070 con 0x55f70b59e1b0 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.172+0000 7fc820ff9640 1 -- 172.21.15.29:0/3988333272 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824011610 con 0x55f70b59e1b0 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 -- 172.21.15.29:0/3988333272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 msgr2=0x55f70b946040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc824009d00 tx=0x7fc82400b420 comp rx=0 tx=0).stop 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 -- 172.21.15.29:0/3988333272 shutdown_connections 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b947200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b9466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 --2- 172.21.15.29:0/3988333272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b59e1b0 0x55f70b946040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 -- 172.21.15.29:0/3988333272 >> 172.21.15.29:0/3988333272 conn(0x55f70b592300 msgr2=0x55f70b941ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 -- 172.21.15.29:0/3988333272 shutdown_connections 2024-09-21T01:54:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.173+0000 7fc83b843740 1 -- 172.21.15.29:0/3988333272 wait complete. 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.174+0000 7fc8237fe640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 0x7fc8180b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.174+0000 7fc83b843740 1 -- start start 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.174+0000 7fc8237fe640 1 -- 172.21.15.29:0/655861052 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc8280e6eb0 con 0x7fc82c1034e0 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.174+0000 7fc83b843740 1 Processor -- start 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc832d9c640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 0x7fc8180b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b57d4d0 0x55f70b94ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b950b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f70b7dfab0 con 0x55f70b59e1b0 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f70b84d730 con 0x55f70b5752f0 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f70b947d20 con 0x55f70b57d4d0 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83b843740 10 client.0 _ll_register_callbacks cb 0x55f70b8a16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54410/0 (socket says 172.21.15.29:54410) 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.175+0000 7fc83259b640 1 -- 172.21.15.29:0/3745029228 learned_addr learned my addr 172.21.15.29:0/3745029228 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc832d9c640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b950b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc832d9c640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 0x7fc8180b8a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc824009f50 tx=0x7fc824012b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 -- 172.21.15.29:0/3745029228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b57d4d0 msgr2=0x55f70b94ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b57d4d0 0x55f70b94ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 -- 172.21.15.29:0/3745029228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 msgr2=0x55f70b950b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b950b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 -- 172.21.15.29:0/3745029228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70b609790 con 0x55f70b5752f0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc832d9c640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b950b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83259b640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8280aa260 tx=0x7fc828009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578c2a570830228c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828020070 con 0x55f70b5752f0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc8280a8940 con 0x55f70b5752f0 2024-09-21T01:54:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.176+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc824017070 con 0x55f70b5752f0 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc8280b1ad0 con 0x55f70b5752f0 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc83b843740 10 client.26372 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f70b682620 con 0x55f70b5752f0 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc824017070 con 0x55f70b5752f0 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fc8280981f0 con 0x55f70b5752f0 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc83b843740 10 client.26372 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc83b843740 10 client.26372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.177+0000 7fc83b843740 20 client.26372 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.178+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b55cd80 0x55f70b95d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.178+0000 7fc83b843740 4 client.26372 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.178+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f70b8ba640 con 0x55f70b55cd80 2024-09-21T01:54:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.179+0000 7fc831d9a640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b55cd80 0x55f70b95d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.180+0000 7fc831d9a640 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b55cd80 0x55f70b95d0b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b406cd0484e356a7 server_cookie=27299ae21c4916cc in_seq=0 out_seq=0 2024-09-21T01:54:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.180+0000 7fc815ffb640 10 client.26372 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.181+0000 7fc815ffb640 1 -- 172.21.15.29:0/3745029228 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f70b8ba640 con 0x55f70b55cd80 2024-09-21T01:54:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.181+0000 7fc815ffb640 10 client.26372 handle_command_reply: tid=0 2024-09-21T01:54:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 client.26372 shutdown 2024-09-21T01:54:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 msgr2=0x55f70b94f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8280aa260 tx=0x7fc828009f00 comp rx=0 tx=0).stop 2024-09-21T01:54:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 shutdown_connections 2024-09-21T01:54:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b55cd80 0x55f70b95d0b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.182+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b950b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b57d4d0 0x55f70b94ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 1 --2- 172.21.15.29:0/3745029228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b94f8f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 >> 172.21.15.29:0/3745029228 conn(0x55f70b592300 msgr2=0x55f70b940150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc815ffb640 0 client.26372 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 shutdown_connections 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 1 -- 172.21.15.29:0/3745029228 wait complete. 2024-09-21T01:54:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.183+0000 7fc83b843740 20 client.26372 trim_cache size 0 max 16384 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 Processor -- start 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 -- start start 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b95d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 0x55f70b940900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b940e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.209+0000 7fc83b843740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b1420 con 0x55f70b59e1b0 2024-09-21T01:54:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83b843740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b2f90 con 0x55f70b57d4d0 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83b843740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b1af0 con 0x55f70b5752f0 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc831d9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 0x55f70b940900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc831d9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 0x55f70b940900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54424/0 (socket says 172.21.15.29:54424) 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83259b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b95d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc831d9a640 1 -- 172.21.15.29:0/827839242 learned_addr learned my addr 172.21.15.29:0/827839242 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc832d9c640 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b940e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83259b640 1 -- 172.21.15.29:0/827839242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 msgr2=0x55f70b940900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83259b640 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 0x55f70b940900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83259b640 1 -- 172.21.15.29:0/827839242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 msgr2=0x55f70b940e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.210+0000 7fc83259b640 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b940e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.211+0000 7fc83259b640 1 -- 172.21.15.29:0/827839242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70b94f260 con 0x55f70b5752f0 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.211+0000 7fc832d9c640 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b940e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.211+0000 7fc83259b640 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b95d030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8280acf80 tx=0x7fc8280abf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed238fe5bf00c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.212+0000 7fc817fff640 1 -- 172.21.15.29:0/827839242 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828020050 con 0x55f70b5752f0 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.212+0000 7fc817fff640 1 -- 172.21.15.29:0/827839242 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8280b4e50 con 0x55f70b5752f0 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.212+0000 7fc817fff640 1 -- 172.21.15.29:0/827839242 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828005220 con 0x55f70b5752f0 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 -- 172.21.15.29:0/827839242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 msgr2=0x55f70b95d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b95d030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8280acf80 tx=0x7fc8280abf30 comp rx=0 tx=0).stop 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 -- 172.21.15.29:0/827839242 shutdown_connections 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b59e1b0 0x55f70b940e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b57d4d0 0x55f70b940900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 --2- 172.21.15.29:0/827839242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b5752f0 0x55f70b95d030 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 -- 172.21.15.29:0/827839242 >> 172.21.15.29:0/827839242 conn(0x55f70b592300 msgr2=0x55f70b94f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 -- 172.21.15.29:0/827839242 shutdown_connections 2024-09-21T01:54:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.213+0000 7fc83b843740 1 -- 172.21.15.29:0/827839242 wait complete. 2024-09-21T01:54:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.214+0000 7fc83b843740 1 -- start start 2024-09-21T01:54:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.215+0000 7fc83b843740 1 Processor -- start 2024-09-21T01:54:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b9547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b954e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b9547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83259b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b9547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54430/0 (socket says 172.21.15.29:54430) 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b59e1b0 0x55f70b955960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b28c0 con 0x55f70b57d4d0 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b21f0 con 0x55f70b5752f0 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1af400 con 0x55f70b59e1b0 2024-09-21T01:54:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.216+0000 7fc83b843740 10 client.0 _ll_register_callbacks cb 0x7fc8280057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.217+0000 7fc831d9a640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b954e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.217+0000 7fc83259b640 1 -- 172.21.15.29:0/1954824556 learned_addr learned my addr 172.21.15.29:0/1954824556 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.218+0000 7fc831d9a640 1 -- 172.21.15.29:0/1954824556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b59e1b0 msgr2=0x55f70b955960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:54:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.218+0000 7fc831d9a640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b59e1b0 0x55f70b955960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.218+0000 7fc831d9a640 1 -- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 msgr2=0x55f70b9547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.218+0000 7fc831d9a640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b9547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.218+0000 7fc831d9a640 1 -- 172.21.15.29:0/1954824556 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70b9560c0 con 0x55f70b57d4d0 2024-09-21T01:54:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.219+0000 7fc831d9a640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b954e60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02a9d0 tx=0x7fc81c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c851d9a5cc5673 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.219+0000 7fc815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:54:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.219+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c035070 con 0x55f70b57d4d0 2024-09-21T01:54:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.220+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc81c0046c0 con 0x55f70b57d4d0 2024-09-21T01:54:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.220+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8280b4e50 con 0x55f70b57d4d0 2024-09-21T01:54:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.220+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc81c005590 con 0x55f70b57d4d0 2024-09-21T01:54:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.220+0000 7fc83b843740 10 client.16632 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f70b957480 con 0x55f70b57d4d0 2024-09-21T01:54:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c004d70 con 0x55f70b57d4d0 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fc81c031510 con 0x55f70b57d4d0 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc83b843740 10 client.16632 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc83b843740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.221+0000 7fc83b843740 20 client.16632 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b5321c0 0x55f70b9c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc83b843740 4 client.16632 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:54:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc8280b4e50 con 0x55f70b5321c0 2024-09-21T01:54:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc83259b640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b5321c0 0x55f70b9c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc815ffb640 10 client.16632 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.222+0000 7fc83259b640 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b5321c0 0x55f70b9c9a80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c2a6f493ba2cb server_cookie=a7293c9bebf211c0 in_seq=0 out_seq=0 2024-09-21T01:54:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.234+0000 7fc815ffb640 1 -- 172.21.15.29:0/1954824556 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc8280b4e50 con 0x55f70b5321c0 2024-09-21T01:54:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.234+0000 7fc815ffb640 10 client.16632 handle_command_reply: tid=0 2024-09-21T01:54:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 client.16632 shutdown 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 msgr2=0x55f70b954e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b954e60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02a9d0 tx=0x7fc81c004310 comp rx=0 tx=0).stop 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 shutdown_connections 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f70b5321c0 0x55f70b9c9a80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f70b59e1b0 0x55f70b955960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f70b57d4d0 0x55f70b954e60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 --2- 172.21.15.29:0/1954824556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f70b5752f0 0x55f70b9547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 >> 172.21.15.29:0/1954824556 conn(0x55f70b592300 msgr2=0x55f70b9408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc815ffb640 0 client.16632 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 shutdown_connections 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.235+0000 7fc83b843740 1 -- 172.21.15.29:0/1954824556 wait complete. 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.236+0000 7fc83b843740 20 client.16632 trim_cache size 0 max 16384 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:54:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:54:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 msgr2=0x7fc8180b8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 0x7fc8180b8a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc824009f50 tx=0x7fc824012b50 comp rx=0 tx=0).stop 2024-09-21T01:54:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 msgr2=0x7fc82c10ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc82800d6d0 tx=0x7fc82800db90 comp rx=0 tx=0).stop 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 shutdown_connections 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.238+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8180b6520 0x7fc8180b8a30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82c104f00 0x7fc82c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc82c104560 0x7fc82c10d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 --2- 172.21.15.29:0/655861052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82c1034e0 0x7fc82c10ca40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 >> 172.21.15.29:0/655861052 conn(0x7fc82c0fee20 msgr2=0x7fc82c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 shutdown_connections 2024-09-21T01:54:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:21.239+0000 7fc83359d640 1 -- 172.21.15.29:0/655861052 wait complete. 2024-09-21T01:54:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 -- 172.21.15.29:0/2077560568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 msgr2=0x7f67741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 --2- 172.21.15.29:0/2077560568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f67741087b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6768009860 tx=0x7f676802a340 comp rx=0 tx=0).stop 2024-09-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 -- 172.21.15.29:0/2077560568 shutdown_connections 2024-09-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 --2- 172.21.15.29:0/2077560568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774108cf0 0x7f677410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 --2- 172.21.15.29:0/2077560568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f67741087b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 --2- 172.21.15.29:0/2077560568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774102a60 0x7f6774102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.365+0000 7f6778eee640 1 -- 172.21.15.29:0/2077560568 >> 172.21.15.29:0/2077560568 conn(0x7f6774079730 msgr2=0x7f6774079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.366+0000 7f6778eee640 1 -- 172.21.15.29:0/2077560568 shutdown_connections 2024-09-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.366+0000 7f6778eee640 1 -- 172.21.15.29:0/2077560568 wait complete. 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 Processor -- start 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 -- start start 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f677419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 0x7f67741a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112960 con 0x7f6774108cf0 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67741127e0 con 0x7f6774102a60 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.367+0000 7f6778eee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112ae0 con 0x7f67741083a0 2024-09-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 0x7f67741a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54442/0 (socket says 172.21.15.29:54442) 2024-09-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 -- 172.21.15.29:0/3881110731 learned_addr learned my addr 172.21.15.29:0/3881110731 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f6771d9b640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f677419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 -- 172.21.15.29:0/3881110731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 msgr2=0x7f677419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f677419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 -- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 msgr2=0x7f67741a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 0x7f67741a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 -- 172.21.15.29:0/3881110731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741a26a0 con 0x7f6774102a60 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f6772d9d640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 0x7f67741a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f677259c640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f676400e9c0 tx=0x7f676400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.368+0000 7f67637fe640 1 -- 172.21.15.29:0/3881110731 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400ccc0 con 0x7f6774102a60 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.369+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741a2990 con 0x7f6774102a60 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.369+0000 7f67637fe640 1 -- 172.21.15.29:0/3881110731 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67640054d0 con 0x7f6774102a60 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.369+0000 7f67637fe640 1 -- 172.21.15.29:0/3881110731 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764010430 con 0x7f6774102a60 2024-09-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.369+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a90f0 con 0x7f6774102a60 2024-09-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.372+0000 7f677f462740 1 Processor -- start 2024-09-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 -- start start 2024-09-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632833800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e632833d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e6327a9ea0 con 0x55e6324631b0 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e6327992d0 con 0x55e63248c070 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f677f462740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e63279d0e0 con 0x55e63246b390 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54448/0 (socket says 172.21.15.29:54448) 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.373+0000 7f6772d9d640 1 -- 172.21.15.29:0/2509112002 learned_addr learned my addr 172.21.15.29:0/2509112002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e632833d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 -- 172.21.15.29:0/2509112002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 msgr2=0x55e632833d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e632833d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 -- 172.21.15.29:0/2509112002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 msgr2=0x55e632833800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:21.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632833800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 -- 172.21.15.29:0/2509112002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6324f7650 con 0x55e63248c070 2024-09-21T01:54:21.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e632833d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:21.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.374+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f676c010710 tx=0x7f676c010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d89e04377a0c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.376+0000 7f6760ff9640 1 -- 172.21.15.29:0/2509112002 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00e8d0 con 0x55e63248c070 2024-09-21T01:54:21.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.376+0000 7f6760ff9640 1 -- 172.21.15.29:0/2509112002 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c012450 con 0x55e63248c070 2024-09-21T01:54:21.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.376+0000 7f6760ff9640 1 -- 172.21.15.29:0/2509112002 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c01c670 con 0x55e63248c070 2024-09-21T01:54:21.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.377+0000 7f67637fe640 1 -- 172.21.15.29:0/3881110731 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6764004f80 con 0x7f6774102a60 2024-09-21T01:54:21.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.377+0000 7f677f462740 1 -- 172.21.15.29:0/2509112002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 msgr2=0x55e6328332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.377+0000 7f677f462740 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f676c010710 tx=0x7f676c010bd0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.377+0000 7f67637fe640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 0x7f675c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 -- 172.21.15.29:0/2509112002 shutdown_connections 2024-09-21T01:54:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e632833d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632833800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 --2- 172.21.15.29:0/2509112002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63248c070 0x55e6328332c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 -- 172.21.15.29:0/2509112002 >> 172.21.15.29:0/2509112002 conn(0x55e632480460 msgr2=0x55e6328311b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677259c640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 0x7f675c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 -- 172.21.15.29:0/2509112002 shutdown_connections 2024-09-21T01:54:21.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f677f462740 1 -- 172.21.15.29:0/2509112002 wait complete. 2024-09-21T01:54:21.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.378+0000 7f67637fe640 1 -- 172.21.15.29:0/3881110731 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f67640e7780 con 0x7f6774102a60 2024-09-21T01:54:21.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.379+0000 7f677f462740 1 -- start start 2024-09-21T01:54:21.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.379+0000 7f677f462740 1 Processor -- start 2024-09-21T01:54:21.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.380+0000 7f677f462740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e63283ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.380+0000 7f677f462740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677f462740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e6328535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677f462740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e6325fa700 con 0x55e6324631b0 2024-09-21T01:54:21.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f6771d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f6771d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54450/0 (socket says 172.21.15.29:54450) 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f6771d9b640 1 -- 172.21.15.29:0/2396936097 learned_addr learned my addr 172.21.15.29:0/2396936097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e6328535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677f462740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e632734160 con 0x55e63246b390 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e632841aa0 con 0x55e63248c070 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677259c640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 0x7f675c0b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67640ad850 tx=0x7f67640a9f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.381+0000 7f677f462740 10 client.0 _ll_register_callbacks cb 0x55e63274d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e63283ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 -- 172.21.15.29:0/2396936097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 msgr2=0x55e6328535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e6328535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 -- 172.21.15.29:0/2396936097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 msgr2=0x55e63283ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e63283ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 -- 172.21.15.29:0/2396936097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e632761c90 con 0x55e63246b390 2024-09-21T01:54:21.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e63283ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f676802a930 tx=0x7f6768004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101abeecd76ec9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768037040 con 0x55e63246b390 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c012450 con 0x55e63246b390 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.382+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6768003a00 con 0x55e63246b390 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6768003c00 con 0x55e63246b390 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 10 client.26390 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e632856000 con 0x55e63246b390 2024-09-21T01:54:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c012450 con 0x55e63246b390 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f676803d4d0 con 0x55e63246b390 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 10 client.26390 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 10 client.26390 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 20 client.26390 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63244a9f0 0x55e632859440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 4 client.26390 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6327abdf0 con 0x55e63244a9f0 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.383+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63244a9f0 0x55e632859440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.384+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63244a9f0 0x55e632859440 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95969202ca4022da server_cookie=c6db197a196227bb in_seq=0 out_seq=0 2024-09-21T01:54:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.384+0000 7f6759ffb640 10 client.26390 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:54:21.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.385+0000 7f6759ffb640 1 -- 172.21.15.29:0/2396936097 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6327abdf0 con 0x55e63244a9f0 2024-09-21T01:54:21.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.385+0000 7f6759ffb640 10 client.26390 handle_command_reply: tid=0 2024-09-21T01:54:21.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.385+0000 7f677f462740 1 client.26390 shutdown 2024-09-21T01:54:21.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 msgr2=0x55e63283ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f676802a930 tx=0x7f6768004580 comp rx=0 tx=0).stop 2024-09-21T01:54:21.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 shutdown_connections 2024-09-21T01:54:21.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63244a9f0 0x55e632859440 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e6328535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e63283ef60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 --2- 172.21.15.29:0/2396936097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e63283ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 >> 172.21.15.29:0/2396936097 conn(0x55e632480460 msgr2=0x55e6328364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f6759ffb640 0 client.26390 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:54:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 shutdown_connections 2024-09-21T01:54:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 1 -- 172.21.15.29:0/2396936097 wait complete. 2024-09-21T01:54:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.386+0000 7f677f462740 20 client.26390 trim_cache size 0 max 16384 2024-09-21T01:54:21.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.409+0000 7f677f462740 1 Processor -- start 2024-09-21T01:54:21.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.409+0000 7f677f462740 1 -- start start 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e632853520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60874/0 (socket says 172.21.15.29:60874) 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f6772d9d640 1 -- 172.21.15.29:0/641103752 learned_addr learned my addr 172.21.15.29:0/641103752 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e632830af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ad280 con 0x55e6324631b0 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f6771d9b640 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e632853520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aedf0 con 0x55e63246b390 2024-09-21T01:54:21.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.410+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ad950 con 0x55e63248c070 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 -- 172.21.15.29:0/641103752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 msgr2=0x55e632830af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e632830af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 -- 172.21.15.29:0/641103752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 msgr2=0x55e632853520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e632853520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 -- 172.21.15.29:0/641103752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63282c5d0 con 0x55e6324631b0 2024-09-21T01:54:21.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.411+0000 7f6772d9d640 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e6c0 tx=0x7f676c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae4721dcc6ec452 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.412+0000 7f675bfff640 1 -- 172.21.15.29:0/641103752 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ea80 con 0x55e6324631b0 2024-09-21T01:54:21.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.412+0000 7f675bfff640 1 -- 172.21.15.29:0/641103752 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c00ec20 con 0x55e6324631b0 2024-09-21T01:54:21.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.412+0000 7f675bfff640 1 -- 172.21.15.29:0/641103752 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0128e0 con 0x55e6324631b0 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 msgr2=0x55e632836ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e6c0 tx=0x7f676c000f40 comp rx=0 tx=0).stop 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 shutdown_connections 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63248c070 0x55e632830af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e63246b390 0x55e632853520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 --2- 172.21.15.29:0/641103752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6324631b0 0x55e632836ad0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.413+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 >> 172.21.15.29:0/641103752 conn(0x55e632480460 msgr2=0x55e6328363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.414+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 shutdown_connections 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.414+0000 7f677f462740 1 -- 172.21.15.29:0/641103752 wait complete. 2024-09-21T01:54:21.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.414+0000 7f677f462740 1 -- start start 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.414+0000 7f677f462740 1 Processor -- start 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e63284b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e63248c070 0x55e63284b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ae720 con 0x55e63248c070 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ae050 con 0x55e6324631b0 2024-09-21T01:54:21.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ab260 con 0x55e63246b390 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f677f462740 10 client.0 _ll_register_callbacks cb 0x7f67680096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54480/0 (socket says 172.21.15.29:54480) 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f6772d9d640 1 -- 172.21.15.29:0/2770161096 learned_addr learned my addr 172.21.15.29:0/2770161096 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e63248c070 0x55e63284b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.415+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e63284b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 -- 172.21.15.29:0/2770161096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 msgr2=0x55e63284b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e63284b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 -- 172.21.15.29:0/2770161096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e63248c070 msgr2=0x55e63284b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e63248c070 0x55e63284b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 -- 172.21.15.29:0/2770161096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63284bee0 con 0x55e6324631b0 2024-09-21T01:54:21.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6772d9d640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f676c005b40 tx=0x7f676c012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f7e42ac16d5c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0049a0 con 0x55e6324631b0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c00ec20 con 0x55e6324631b0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.416+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f676c004b40 con 0x55e6324631b0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.417+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f676c004d40 con 0x55e6324631b0 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.417+0000 7f677f462740 10 client.26396 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.417+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6328c3cf0 con 0x55e6324631b0 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c00ec20 con 0x55e6324631b0 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f676c00f040 con 0x55e6324631b0 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 10 client.26396 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 10 client.26396 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 20 client.26396 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63241fda0 0x55e6328c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 4 client.26396 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:54:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6327a78a0 con 0x55e63241fda0 2024-09-21T01:54:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.419+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63241fda0 0x55e6328c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f6771d9b640 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63241fda0 0x55e6328c62d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0b678bd748aa1bb server_cookie=f4e49d85c6b4e812 in_seq=0 out_seq=0 2024-09-21T01:54:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f6759ffb640 10 client.26396 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f6759ffb640 1 -- 172.21.15.29:0/2770161096 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6327a78a0 con 0x55e63241fda0 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f6759ffb640 10 client.26396 handle_command_reply: tid=0 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f677f462740 1 client.26396 shutdown 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 msgr2=0x55e63284fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f676c005b40 tx=0x7f676c012450 comp rx=0 tx=0).stop 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 shutdown_connections 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.420+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e63241fda0 0x55e6328c62d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e63248c070 0x55e63284b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f6759ffb640 0 client.26396 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e63246b390 0x55e63284b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 --2- 172.21.15.29:0/2770161096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6324631b0 0x55e63284fd10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 >> 172.21.15.29:0/2770161096 conn(0x55e632480460 msgr2=0x55e63283e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 shutdown_connections 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 1 -- 172.21.15.29:0/2770161096 wait complete. 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.421+0000 7f677f462740 20 client.26396 trim_cache size 0 max 16384 2024-09-21T01:54:21.423 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 msgr2=0x7f675c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 0x7f675c0b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67640ad850 tx=0x7f67640a9f10 comp rx=0 tx=0).stop 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 msgr2=0x7f677419c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f676400e9c0 tx=0x7f676400ee80 comp rx=0 tx=0).stop 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 shutdown_connections 2024-09-21T01:54:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f675c0b6590 0x7f675c0b8aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6774108cf0 0x7f67741a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67741083a0 0x7f677419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 --2- 172.21.15.29:0/3881110731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6774102a60 0x7f677419c730 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.423+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 >> 172.21.15.29:0/3881110731 conn(0x7f6774079730 msgr2=0x7f6774104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.424+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 shutdown_connections 2024-09-21T01:54:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:21.424+0000 7f6778eee640 1 -- 172.21.15.29:0/3881110731 wait complete. 2024-09-21T01:54:21.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:21 smithi190 ceph-mon[34368]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 856 op/s 2024-09-21T01:54:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:21 smithi066 ceph-mon[35001]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 856 op/s 2024-09-21T01:54:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:21 smithi029 ceph-mon[31228]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 856 op/s 2024-09-21T01:54:22.400 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:54:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 345.0 s, 3078.6 tps, lat 32.329 ms stddev 44.749 2024-09-21T01:54:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:23 smithi066 ceph-mon[35001]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-09-21T01:54:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:23 smithi029 ceph-mon[31228]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-09-21T01:54:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:23 smithi190 ceph-mon[34368]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 -- 172.21.15.29:0/418001413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e810a3c0 msgr2=0x7fa4e810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/418001413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e810a3c0 0x7fa4e810a7d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc02a410 comp rx=0 tx=0).stop 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 -- 172.21.15.29:0/418001413 shutdown_connections 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/418001413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e806af90 0x7fa4e810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/418001413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e810a3c0 0x7fa4e810a7d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/418001413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 -- 172.21.15.29:0/418001413 >> 172.21.15.29:0/418001413 conn(0x7fa4e80712f0 msgr2=0x7fa4e8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.024+0000 7fa4ecb13640 1 -- 172.21.15.29:0/418001413 shutdown_connections 2024-09-21T01:54:24.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.025+0000 7fa4ecb13640 1 -- 172.21.15.29:0/418001413 wait complete. 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 Processor -- start 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 -- start start 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810a3c0 0x7fa4e811a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e811aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8118e10 con 0x7fa4e810f150 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8118c90 con 0x7fa4e806af90 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4ecb13640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8118f90 con 0x7fa4e810a3c0 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39098/0 (socket says 172.21.15.29:39098) 2024-09-21T01:54:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.026+0000 7fa4e659c640 1 -- 172.21.15.29:0/77708011 learned_addr learned my addr 172.21.15.29:0/77708011 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e811aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e659c640 1 -- 172.21.15.29:0/77708011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810a3c0 msgr2=0x7fa4e811a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e659c640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810a3c0 0x7fa4e811a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e659c640 1 -- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 msgr2=0x7fa4e811aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e659c640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e811aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e659c640 1 -- 172.21.15.29:0/77708011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e81bddc0 con 0x7fa4e806af90 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.027+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e811aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.028+0000 7fa4e659c640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9f0 tx=0x7fa4d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.029+0000 7fa4d77fe640 1 -- 172.21.15.29:0/77708011 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800cd80 con 0x7fa4e806af90 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.029+0000 7fa4d77fe640 1 -- 172.21.15.29:0/77708011 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80054d0 con 0x7fa4e806af90 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.029+0000 7fa4d77fe640 1 -- 172.21.15.29:0/77708011 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8010690 con 0x7fa4e806af90 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.029+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e81bdf90 con 0x7fa4e806af90 2024-09-21T01:54:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.029+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81be390 con 0x7fa4e806af90 2024-09-21T01:54:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.033+0000 7fa4d77fe640 1 -- 172.21.15.29:0/77708011 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa4d8004e10 con 0x7fa4e806af90 2024-09-21T01:54:24.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.033+0000 7fa4f3087740 1 Processor -- start 2024-09-21T01:54:24.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 -- start start 2024-09-21T01:54:24.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 0x55bfaa82e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa82ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa7a4ea0 con 0x55bfaa466390 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa7942d0 con 0x55bfaa487070 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4f3087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa7980e0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4e6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 0x55bfaa82e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4e5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa82ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.034+0000 7fa4e6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 0x55bfaa82e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39112/0 (socket says 172.21.15.29:39112) 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e6d9d640 1 -- 172.21.15.29:0/1211810646 learned_addr learned my addr 172.21.15.29:0/1211810646 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 -- 172.21.15.29:0/1211810646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 msgr2=0x55bfaa82e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 -- 172.21.15.29:0/1211810646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 msgr2=0x55bfaa82e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 0x55bfaa82e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 -- 172.21.15.29:0/1211810646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfaa4f2650 con 0x55bfaa466390 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4d77fe640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 0x7fa4d00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.035+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa82ed40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4e810ad90 tx=0x7fa4dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6a7dd6814400f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.036+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 0x7fa4d00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.036+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/1211810646 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc03e070 con 0x55bfaa466390 2024-09-21T01:54:24.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.036+0000 7fa4d77fe640 1 -- 172.21.15.29:0/77708011 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4d80e73e0 con 0x7fa4e806af90 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.036+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/1211810646 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4dc005500 con 0x55bfaa466390 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.036+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 0x7fa4d00b8c30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0011930 tx=0x7fa4e0009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.037+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/1211810646 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc035040 con 0x55bfaa466390 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 -- 172.21.15.29:0/1211810646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 msgr2=0x55bfaa82ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa82ed40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4e810ad90 tx=0x7fa4dc000bc0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 -- 172.21.15.29:0/1211810646 shutdown_connections 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa82ed40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 --2- 172.21.15.29:0/1211810646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa487070 0x55bfaa82e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.038+0000 7fa4f3087740 1 -- 172.21.15.29:0/1211810646 >> 172.21.15.29:0/1211810646 conn(0x55bfaa47b460 msgr2=0x55bfaa82c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.039+0000 7fa4f3087740 1 -- 172.21.15.29:0/1211810646 shutdown_connections 2024-09-21T01:54:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.039+0000 7fa4f3087740 1 -- 172.21.15.29:0/1211810646 wait complete. 2024-09-21T01:54:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 -- start start 2024-09-21T01:54:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 Processor -- start 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa839830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa839d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa5f5700 con 0x55bfaa466390 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa72f160 con 0x55bfaa45e1b0 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.040+0000 7fa4f3087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bfaa83ca90 con 0x55bfaa487070 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4f3087740 10 client.0 _ll_register_callbacks cb 0x55bfaa7489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33908/0 (socket says 172.21.15.29:33908) 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 -- 172.21.15.29:0/3589753592 learned_addr learned my addr 172.21.15.29:0/3589753592 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa839d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 -- 172.21.15.29:0/3589753592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 msgr2=0x55bfaa839830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa839830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 -- 172.21.15.29:0/3589753592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 msgr2=0x55bfaa839d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa839d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 -- 172.21.15.29:0/3589753592 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfaa75cc90 con 0x55bfaa487070 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.041+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001e6b0 tx=0x7fa4e0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e736db2deb4be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e002b060 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4e001c730 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4dc005500 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4e001c930 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4f3087740 10 client.26425 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bfaa848730 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.042+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4dc005500 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa4e001a040 con 0x55bfaa487070 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 10 client.26425 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 10 client.26425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:54:24.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 20 client.26425 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa4459f0 0x55bfaa84bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 4 client.26425 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfaa7a6df0 con 0x55bfaa4459f0 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4e659c640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa4459f0 0x55bfaa84bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4cdffb640 10 client.26425 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.043+0000 7fa4e659c640 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa4459f0 0x55bfaa84bd50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff5e146e75aa4b85 server_cookie=337387b1fdd60787 in_seq=0 out_seq=0 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.044+0000 7fa4cdffb640 1 -- 172.21.15.29:0/3589753592 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bfaa7a6df0 con 0x55bfaa4459f0 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.044+0000 7fa4cdffb640 10 client.26425 handle_command_reply: tid=0 2024-09-21T01:54:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 client.26425 shutdown 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 msgr2=0x55bfaa83a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001e6b0 tx=0x7fa4e0004a40 comp rx=0 tx=0).stop 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 shutdown_connections 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa4459f0 0x55bfaa84bd50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83a2b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa839d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 --2- 172.21.15.29:0/3589753592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa839830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 >> 172.21.15.29:0/3589753592 conn(0x55bfaa47b460 msgr2=0x55bfaa831450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4cdffb640 0 client.26425 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:54:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 shutdown_connections 2024-09-21T01:54:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.045+0000 7fa4f3087740 1 -- 172.21.15.29:0/3589753592 wait complete. 2024-09-21T01:54:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.046+0000 7fa4f3087740 20 client.26425 trim_cache size 0 max 16384 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.065+0000 7fa4f3087740 1 Processor -- start 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.065+0000 7fa4f3087740 1 -- start start 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 0x55bfaa82b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa487070 0x55bfaa82bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800e410 con 0x55bfaa487070 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800ff80 con 0x55bfaa466390 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4f3087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800eae0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 0x55bfaa82b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33924/0 (socket says 172.21.15.29:33924) 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 -- 172.21.15.29:0/2517450650 learned_addr learned my addr 172.21.15.29:0/2517450650 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 0x55bfaa82b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39144/0 (socket says 172.21.15.29:39144) 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa487070 0x55bfaa82bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 -- 172.21.15.29:0/2517450650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 msgr2=0x55bfaa82b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 0x55bfaa82b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 -- 172.21.15.29:0/2517450650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa487070 msgr2=0x55bfaa82bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa487070 0x55bfaa82bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.066+0000 7fa4e659c640 1 -- 172.21.15.29:0/2517450650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfaa8305e0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.067+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80e4e30 tx=0x7fa4d8004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18d03f79bce2311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.067+0000 7fa4cffff640 1 -- 172.21.15.29:0/2517450650 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80a93c0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.067+0000 7fa4cffff640 1 -- 172.21.15.29:0/2517450650 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80ad030 con 0x55bfaa45e1b0 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.067+0000 7fa4cffff640 1 -- 172.21.15.29:0/2517450650 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80acc90 con 0x55bfaa45e1b0 2024-09-21T01:54:24.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 -- 172.21.15.29:0/2517450650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 msgr2=0x55bfaa82fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80e4e30 tx=0x7fa4d8004b30 comp rx=0 tx=0).stop 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 -- 172.21.15.29:0/2517450650 shutdown_connections 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa487070 0x55bfaa82bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa466390 0x55bfaa82b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2517450650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa45e1b0 0x55bfaa82fa00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.069+0000 7fa4f3087740 1 -- 172.21.15.29:0/2517450650 >> 172.21.15.29:0/2517450650 conn(0x55bfaa47b460 msgr2=0x55bfaa82f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.070+0000 7fa4f3087740 1 -- 172.21.15.29:0/2517450650 shutdown_connections 2024-09-21T01:54:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.070+0000 7fa4f3087740 1 -- 172.21.15.29:0/2517450650 wait complete. 2024-09-21T01:54:24.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.070+0000 7fa4f3087740 1 -- start start 2024-09-21T01:54:24.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.071+0000 7fa4f3087740 1 Processor -- start 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.071+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.071+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa83db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39148/0 (socket says 172.21.15.29:39148) 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4f3087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 -- 172.21.15.29:0/2990583715 learned_addr learned my addr 172.21.15.29:0/2990583715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800f8b0 con 0x55bfaa466390 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800f1e0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c800c3f0 con 0x55bfaa487070 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e6d9d640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa83db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4f3087740 10 client.0 _ll_register_callbacks cb 0x7fa4e0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:24.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 -- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 msgr2=0x55bfaa83e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 -- 172.21.15.29:0/2990583715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 msgr2=0x55bfaa83db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:24.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa83db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.072+0000 7fa4e659c640 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfaa83e980 con 0x55bfaa45e1b0 2024-09-21T01:54:24.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.073+0000 7fa4e659c640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80aa790 tx=0x7fa4d80a99c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdc588eef047d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:24.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.073+0000 7fa4cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:24.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.073+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80ace70 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.073+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4d80046c0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.073+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d80ad030 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.074+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4d80ad2f0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.074+0000 7fa4f3087740 10 client.26411 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.074+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bfaa8b67c0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.074+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80b2000 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.074+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa4d80ac1f0 con 0x55bfaa45e1b0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 10 client.26411 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 10 client.26411 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 20 client.26411 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa41ada0 0x55bfaa8b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 4 client.26411 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa4d80ad030 con 0x55bfaa41ada0 2024-09-21T01:54:24.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa41ada0 0x55bfaa8b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4e5d9b640 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa41ada0 0x55bfaa8b8e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76cedde3b3599205 server_cookie=3efc527d532799b in_seq=0 out_seq=0 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.075+0000 7fa4cdffb640 10 client.26411 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4cdffb640 1 -- 172.21.15.29:0/2990583715 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa4d80ad030 con 0x55bfaa41ada0 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4cdffb640 10 client.26411 handle_command_reply: tid=0 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 client.26411 shutdown 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 msgr2=0x55bfaa8426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80aa790 tx=0x7fa4d80a99c0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 shutdown_connections 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55bfaa41ada0 0x55bfaa8b8e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfaa487070 0x55bfaa83e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bfaa466390 0x55bfaa83db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 --2- 172.21.15.29:0/2990583715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfaa45e1b0 0x55bfaa8426c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 >> 172.21.15.29:0/2990583715 conn(0x55bfaa47b460 msgr2=0x55bfaa82f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4cdffb640 0 client.26411 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 shutdown_connections 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 1 -- 172.21.15.29:0/2990583715 wait complete. 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.076+0000 7fa4f3087740 20 client.26411 trim_cache size 0 max 16384 2024-09-21T01:54:24.079 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 msgr2=0x7fa4d00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 0x7fa4d00b8c30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0011930 tx=0x7fa4e0009600 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 msgr2=0x7fa4e81206d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9f0 tx=0x7fa4d800edf0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 shutdown_connections 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4d00b6720 0x7fa4d00b8c30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e810f150 0x7fa4e811aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810a3c0 0x7fa4e811a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 --2- 172.21.15.29:0/77708011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e806af90 0x7fa4e81206d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 >> 172.21.15.29:0/77708011 conn(0x7fa4e80712f0 msgr2=0x7fa4e810bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 shutdown_connections 2024-09-21T01:54:24.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:24.077+0000 7fa4ecb13640 1 -- 172.21.15.29:0/77708011 wait complete. 2024-09-21T01:54:25.273 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:54:25.691 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:25 smithi066 ceph-mon[35001]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-09-21T01:54:25.825 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:25 smithi029 ceph-mon[31228]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-09-21T01:54:25.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:25 smithi190 ceph-mon[34368]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-09-21T01:54:26.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.393+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/72289987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 msgr2=0x7f8a9c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.393+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/72289987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c10f520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009be0 tx=0x7f8a9402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.394+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/72289987 shutdown_connections 2024-09-21T01:54:26.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.394+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/72289987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c10f520 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.394+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/72289987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 0x7f8a9c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.394+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/72289987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 0x7f8a9c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.394+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/72289987 >> 172.21.15.29:0/72289987 conn(0x7f8a9c079730 msgr2=0x7f8a9c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.395+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/72289987 shutdown_connections 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.395+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/72289987 wait complete. 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.396+0000 7f8aa0d0e640 1 Processor -- start 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.396+0000 7f8aa0d0e640 1 -- start start 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 0x7f8a9c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 0x7f8a9c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c112960 con 0x7f8a9c102a60 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1127e0 con 0x7f8a9c108cf0 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8aa0d0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c112ae0 con 0x7f8a9c1083a0 2024-09-21T01:54:26.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.397+0000 7f8a9ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39174/0 (socket says 172.21.15.29:39174) 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a99d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 0x7f8a9c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/253750676 learned_addr learned my addr 172.21.15.29:0/253750676 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 0x7f8a9c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/253750676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 msgr2=0x7f8a9c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 0x7f8a9c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 msgr2=0x7f8a9c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 0x7f8a9c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/253750676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c19f030 con 0x7f8a9c108cf0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.398+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009560 tx=0x7f8a94003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.399+0000 7f8a8b7fe640 1 -- 172.21.15.29:0/253750676 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94044070 con 0x7f8a9c108cf0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.399+0000 7f8a8b7fe640 1 -- 172.21.15.29:0/253750676 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a94030970 con 0x7f8a9c108cf0 2024-09-21T01:54:26.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.399+0000 7f8a8b7fe640 1 -- 172.21.15.29:0/253750676 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94036040 con 0x7f8a9c108cf0 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.399+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c19f2c0 con 0x7f8a9c108cf0 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.399+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c06ae60 con 0x7f8a9c108cf0 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.403+0000 7f8aa7282740 1 Processor -- start 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.403+0000 7f8a8b7fe640 1 -- 172.21.15.29:0/253750676 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8a94047070 con 0x7f8a9c108cf0 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 -- start start 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee13ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55983ed81ea0 con 0x55983ea3b1b0 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55983ed712d0 con 0x55983ea43390 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8aa7282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55983ed750e0 con 0x55983ea64070 2024-09-21T01:54:26.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.404+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee14010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43074/0 (socket says 172.21.15.29:43074) 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39184/0 (socket says 172.21.15.29:39184) 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a9a59c640 1 -- 172.21.15.29:0/2211660702 learned_addr learned my addr 172.21.15.29:0/2211660702 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee13ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 -- 172.21.15.29:0/2211660702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 msgr2=0x55983ee13ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee13ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 -- 172.21.15.29:0/2211660702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 msgr2=0x55983ee14010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee14010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 -- 172.21.15.29:0/2211660702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983eacf650 con 0x55983ea43390 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a8b7fe640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 0x7f8a840b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005d00 tx=0x7f8a90005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a8b8904d3ac962 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 0x7f8a840b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.405+0000 7f8a8b7fe640 1 -- 172.21.15.29:0/253750676 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a941071a0 con 0x7f8a9c108cf0 2024-09-21T01:54:26.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.406+0000 7f8a88ff9640 1 -- 172.21.15.29:0/2211660702 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90012d20 con 0x55983ea43390 2024-09-21T01:54:26.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.406+0000 7f8a88ff9640 1 -- 172.21.15.29:0/2211660702 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90005020 con 0x55983ea43390 2024-09-21T01:54:26.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 -- 172.21.15.29:0/2211660702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 msgr2=0x55983ee14550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005d00 tx=0x7f8a90005c90 comp rx=0 tx=0).stop 2024-09-21T01:54:26.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 -- 172.21.15.29:0/2211660702 shutdown_connections 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee14550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee14010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2211660702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee13ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 -- 172.21.15.29:0/2211660702 >> 172.21.15.29:0/2211660702 conn(0x55983ea58460 msgr2=0x55983ee119c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 0x7f8a840b8aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55983ee12750 tx=0x7f8a8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 -- 172.21.15.29:0/2211660702 shutdown_connections 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.407+0000 7f8aa7282740 1 -- 172.21.15.29:0/2211660702 wait complete. 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.408+0000 7f8aa7282740 1 -- start start 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.408+0000 7f8aa7282740 1 Processor -- start 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.408+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.408+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 0x55983ee1f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 0x55983ee2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8aa7282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55983ebd2700 con 0x55983ea3b1b0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8aa7282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55983ed0c160 con 0x55983ea64070 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8aa7282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55983ee22350 con 0x55983ea43390 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8aa7282740 10 client.0 _ll_register_callbacks cb 0x55983ed259e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a99d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 0x55983ee1f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 0x55983ee2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a99d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 0x55983ee1f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:33976/0 (socket says 172.21.15.29:33976) 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a99d9b640 1 -- 172.21.15.29:0/3886070793 learned_addr learned my addr 172.21.15.29:0/3886070793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 msgr2=0x55983ee1f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 0x55983ee1f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/3886070793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 msgr2=0x55983ee1f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee1f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.409+0000 7f8a9ad9d640 1 -- 172.21.15.29:0/3886070793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983ed39c90 con 0x55983ea64070 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8a9ad9d640 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 0x55983ee2b690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8a940ce9c0 tx=0x7f8a940d1160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810223fde30b6fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94106070 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a940d09b0 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a90005020 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a9404e040 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8aa7282740 10 client.26423 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.410+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55983ee2e070 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.411+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90005020 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.411+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8a940d04b0 con 0x55983ea64070 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.412+0000 7f8aa7282740 10 client.26423 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.412+0000 7f8aa7282740 10 client.26423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.412+0000 7f8aa7282740 20 client.26423 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.413+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983ea229f0 0x55983ee31500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.413+0000 7f8aa7282740 4 client.26423 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:26.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.413+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55983ed83df0 con 0x55983ea229f0 2024-09-21T01:54:26.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.413+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983ea229f0 0x55983ee31500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.414+0000 7f8a81ffb640 10 client.26423 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.414+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983ea229f0 0x55983ee31500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18242651fa0f641e server_cookie=4d8ae7e5fed3109f in_seq=0 out_seq=0 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.415+0000 7f8a81ffb640 1 -- 172.21.15.29:0/3886070793 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55983ed83df0 con 0x55983ea229f0 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.415+0000 7f8a81ffb640 10 client.26423 handle_command_reply: tid=0 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.415+0000 7f8aa7282740 1 client.26423 shutdown 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.415+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 msgr2=0x55983ee2b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 0x55983ee2b690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8a940ce9c0 tx=0x7f8a940d1160 comp rx=0 tx=0).stop 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 shutdown_connections 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983ea229f0 0x55983ee31500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8a81ffb640 0 client.26423 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:54:26.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea64070 0x55983ee2b690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea43390 0x55983ee1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 --2- 172.21.15.29:0/3886070793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 >> 172.21.15.29:0/3886070793 conn(0x55983ea58460 msgr2=0x55983ee16c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 shutdown_connections 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 1 -- 172.21.15.29:0/3886070793 wait complete. 2024-09-21T01:54:26.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.416+0000 7f8aa7282740 20 client.26423 trim_cache size 0 max 16384 2024-09-21T01:54:26.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 Processor -- start 2024-09-21T01:54:26.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 -- start start 2024-09-21T01:54:26.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1ad0f0 con 0x55983ea3b1b0 2024-09-21T01:54:26.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1aec60 con 0x55983ea43390 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.438+0000 7f8aa7282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1ad7c0 con 0x55983ea64070 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a9ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43106/0 (socket says 172.21.15.29:43106) 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a9a59c640 1 -- 172.21.15.29:0/656732141 learned_addr learned my addr 172.21.15.29:0/656732141 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 -- 172.21.15.29:0/656732141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 msgr2=0x55983ee2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 -- 172.21.15.29:0/656732141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 msgr2=0x55983ee154c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee154c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.439+0000 7f8a99d9b640 1 -- 172.21.15.29:0/656732141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983ee045d0 con 0x55983ea43390 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.440+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee20d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000bfd0 tx=0x7f8a90001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf6d8929030f494 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.440+0000 7f8a83fff640 1 -- 172.21.15.29:0/656732141 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004240 con 0x55983ea43390 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.440+0000 7f8a83fff640 1 -- 172.21.15.29:0/656732141 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90016040 con 0x55983ea43390 2024-09-21T01:54:26.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.440+0000 7f8a83fff640 1 -- 172.21.15.29:0/656732141 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90003940 con 0x55983ea43390 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 -- 172.21.15.29:0/656732141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 msgr2=0x55983ee20d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee20d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000bfd0 tx=0x7f8a90001e50 comp rx=0 tx=0).stop 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 -- 172.21.15.29:0/656732141 shutdown_connections 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea43390 0x55983ee20d70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 --2- 172.21.15.29:0/656732141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea3b1b0 0x55983ee154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 -- 172.21.15.29:0/656732141 >> 172.21.15.29:0/656732141 conn(0x55983ea58460 msgr2=0x55983ee20660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 -- 172.21.15.29:0/656732141 shutdown_connections 2024-09-21T01:54:26.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.441+0000 7f8aa7282740 1 -- 172.21.15.29:0/656732141 wait complete. 2024-09-21T01:54:26.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.442+0000 7f8aa7282740 1 -- start start 2024-09-21T01:54:26.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.442+0000 7f8aa7282740 1 Processor -- start 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea43390 0x55983ee24290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee9aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1ae590 con 0x55983ea43390 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1adec0 con 0x55983ea3b1b0 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8aa7282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1ab0d0 con 0x55983ea64070 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8a9a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39208/0 (socket says 172.21.15.29:39208) 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8a9a59c640 1 -- 172.21.15.29:0/2475354631 learned_addr learned my addr 172.21.15.29:0/2475354631 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.443+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea43390 0x55983ee24290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8aa7282740 10 client.0 _ll_register_callbacks cb 0x7f8a94009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 -- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 msgr2=0x55983ee9aec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee9aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 -- 172.21.15.29:0/2475354631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea43390 msgr2=0x55983ee24290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea43390 0x55983ee24290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 -- 172.21.15.29:0/2475354631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983ee9b450 con 0x55983ea3b1b0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a9a59c640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c01f780 tx=0x7f8a8c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d61864c45ea11a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004030 con 0x55983ea3b1b0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.444+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8a8c018030 con 0x55983ea3b1b0 2024-09-21T01:54:26.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.445+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a90016040 con 0x55983ea3b1b0 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.445+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c0041d0 con 0x55983ea3b1b0 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.445+0000 7f8aa7282740 10 client.26429 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.445+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55983ee9c710 con 0x55983ea3b1b0 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.445+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55983ed83df0 con 0x55983ea3b1b0 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8a8c02b3b0 con 0x55983ea3b1b0 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 10 client.26429 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 10 client.26429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 20 client.26429 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:26.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983e9f7da0 0x55983ee9ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 4 client.26429 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8a90016040 con 0x55983e9f7da0 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983e9f7da0 0x55983ee9ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8a99d9b640 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983e9f7da0 0x55983ee9ef00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15ef228e35cf40d8 server_cookie=8ffde6df8c8b7a59 in_seq=0 out_seq=0 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.446+0000 7f8a81ffb640 10 client.26429 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8a81ffb640 1 -- 172.21.15.29:0/2475354631 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8a90016040 con 0x55983e9f7da0 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8a81ffb640 10 client.26429 handle_command_reply: tid=0 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 client.26429 shutdown 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 msgr2=0x55983ee23d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c01f780 tx=0x7f8a8c0046c0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 shutdown_connections 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55983e9f7da0 0x55983ee9ef00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55983ea64070 0x55983ee9aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55983ea43390 0x55983ee24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 --2- 172.21.15.29:0/2475354631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55983ea3b1b0 0x55983ee23d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 >> 172.21.15.29:0/2475354631 conn(0x55983ea58460 msgr2=0x55983ee20500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.447+0000 7f8a81ffb640 0 client.26429 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.448+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 shutdown_connections 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.448+0000 7f8aa7282740 1 -- 172.21.15.29:0/2475354631 wait complete. 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.448+0000 7f8aa7282740 20 client.26429 trim_cache size 0 max 16384 2024-09-21T01:54:26.451 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 msgr2=0x7f8a840b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 0x7f8a840b8aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55983ee12750 tx=0x7f8a8c009260 comp rx=0 tx=0).stop 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 msgr2=0x7f8a9c19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009560 tx=0x7f8a94003df0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 shutdown_connections 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8a840b6590 0x7f8a840b8aa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a9c108cf0 0x7f8a9c19e900 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c1083a0 0x7f8a9c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 --2- 172.21.15.29:0/253750676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a9c102a60 0x7f8a9c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:26.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 >> 172.21.15.29:0/253750676 conn(0x7f8a9c079730 msgr2=0x7f8a9c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:26.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 shutdown_connections 2024-09-21T01:54:26.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:26.449+0000 7f8aa0d0e640 1 -- 172.21.15.29:0/253750676 wait complete. 2024-09-21T01:54:27.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:54:27.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:27 smithi029 ceph-mon[31228]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-21T01:54:27.754 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:27 smithi066 ceph-mon[35001]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-21T01:54:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 350.0 s, 3417.6 tps, lat 29.386 ms stddev 33.037 2024-09-21T01:54:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:27 smithi190 ceph-mon[34368]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-21T01:54:28.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.341+0000 7fd7d7eca640 1 -- 172.21.15.29:0/1934272171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d01027a0 msgr2=0x7fd7d0102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.341+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/1934272171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d01027a0 0x7fd7d0102bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009930 tx=0x7fd7cc02a420 comp rx=0 tx=0).stop 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 -- 172.21.15.29:0/1934272171 shutdown_connections 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/1934272171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d0107be0 0x7fd7d01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/1934272171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01030f0 0x7fd7d0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/1934272171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d01027a0 0x7fd7d0102bb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 -- 172.21.15.29:0/1934272171 >> 172.21.15.29:0/1934272171 conn(0x7fd7d0079730 msgr2=0x7fd7d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 -- 172.21.15.29:0/1934272171 shutdown_connections 2024-09-21T01:54:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.342+0000 7fd7d7eca640 1 -- 172.21.15.29:0/1934272171 wait complete. 2024-09-21T01:54:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.343+0000 7fd7d7eca640 1 Processor -- start 2024-09-21T01:54:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.343+0000 7fd7d7eca640 1 -- start start 2024-09-21T01:54:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.343+0000 7fd7d7eca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 0x7fd7d01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.343+0000 7fd7d7eca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.343+0000 7fd7d7eca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 0x7fd7d01a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39240/0 (socket says 172.21.15.29:39240) 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 0x7fd7d01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 0x7fd7d01a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34010/0 (socket says 172.21.15.29:34010) 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d6467640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 0x7fd7d01a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d6467640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 0x7fd7d01a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43124/0 (socket says 172.21.15.29:43124) 2024-09-21T01:54:28.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d5465640 1 -- 172.21.15.29:0/913299422 learned_addr learned my addr 172.21.15.29:0/913299422 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:28.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d011a100 con 0x7fd7d0107be0 2024-09-21T01:54:28.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0119f80 con 0x7fd7d01030f0 2024-09-21T01:54:28.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.344+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d011a280 con 0x7fd7d01027a0 2024-09-21T01:54:28.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.345+0000 7fd7d5465640 1 -- 172.21.15.29:0/913299422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 msgr2=0x7fd7d01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.345+0000 7fd7d5465640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 0x7fd7d01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.345+0000 7fd7d5465640 1 -- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 msgr2=0x7fd7d01a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.345+0000 7fd7d5465640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 0x7fd7d01a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.345+0000 7fd7d5465640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d01a6a80 con 0x7fd7d01030f0 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.346+0000 7fd7d5465640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9c0 tx=0x7fd7c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.347+0000 7fd7c6ffd640 1 -- 172.21.15.29:0/913299422 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000ccc0 con 0x7fd7d01030f0 2024-09-21T01:54:28.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.347+0000 7fd7c6ffd640 1 -- 172.21.15.29:0/913299422 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c00054d0 con 0x7fd7d01030f0 2024-09-21T01:54:28.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.347+0000 7fd7c6ffd640 1 -- 172.21.15.29:0/913299422 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0010430 con 0x7fd7d01030f0 2024-09-21T01:54:28.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.347+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d01a6d70 con 0x7fd7d01030f0 2024-09-21T01:54:28.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.347+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d01ad4d0 con 0x7fd7d01030f0 2024-09-21T01:54:28.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.352+0000 7fd7c6ffd640 1 -- 172.21.15.29:0/913299422 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd7c00046c0 con 0x7fd7d01030f0 2024-09-21T01:54:28.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.352+0000 7fd7de43e740 1 Processor -- start 2024-09-21T01:54:28.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.352+0000 7fd7de43e740 1 -- start start 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 0x5579a345d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a345e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43136/0 (socket says 172.21.15.29:43136) 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7d5c66640 1 -- 172.21.15.29:0/3850549936 learned_addr learned my addr 172.21.15.29:0/3850549936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7d5465640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a345e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5579a33d3ea0 con 0x5579a308d1b0 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.353+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5579a33c32d0 con 0x5579a3095390 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5579a33c70e0 con 0x5579a30b6070 2024-09-21T01:54:28.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 0x5579a345d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 -- 172.21.15.29:0/3850549936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 msgr2=0x5579a345e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a345e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 -- 172.21.15.29:0/3850549936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 msgr2=0x5579a345dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.354+0000 7fd7d6467640 1 -- 172.21.15.29:0/3850549936 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a3121650 con 0x5579a30b6070 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7d6467640 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 0x5579a345d6a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009f00 tx=0x7fd7c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5615d5780e8d7b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7bbfff640 1 -- 172.21.15.29:0/3850549936 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004d20 con 0x5579a30b6070 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7bbfff640 1 -- 172.21.15.29:0/3850549936 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c8004030 con 0x5579a30b6070 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7bbfff640 1 -- 172.21.15.29:0/3850549936 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8005280 con 0x5579a30b6070 2024-09-21T01:54:28.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.355+0000 7fd7c6ffd640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 0x7fd7bc0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.356+0000 7fd7c6ffd640 1 -- 172.21.15.29:0/913299422 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd7c00e75f0 con 0x7fd7d01030f0 2024-09-21T01:54:28.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.356+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 0x7fd7bc0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 msgr2=0x5579a345d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 0x5579a345d6a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009f00 tx=0x7fd7c8011810 comp rx=0 tx=0).stop 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 shutdown_connections 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a345e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a345dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 --2- 172.21.15.29:0/3850549936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a30b6070 0x5579a345d6a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 >> 172.21.15.29:0/3850549936 conn(0x5579a30aa460 msgr2=0x5579a345b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.357+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 0x7fd7bc0b8b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5579a345c320 tx=0x7fd7cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.358+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 shutdown_connections 2024-09-21T01:54:28.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.358+0000 7fd7de43e740 1 -- 172.21.15.29:0/3850549936 wait complete. 2024-09-21T01:54:28.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.359+0000 7fd7de43e740 1 -- start start 2024-09-21T01:54:28.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.359+0000 7fd7de43e740 1 Processor -- start 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 0x5579a3468d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a3469800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5579a3224700 con 0x5579a30b6070 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5579a335e160 con 0x5579a3095390 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5579a346be80 con 0x5579a308d1b0 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7de43e740 10 client.0 _ll_register_callbacks cb 0x5579a33779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 0x5579a3468d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 0x5579a3468d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34020/0 (socket says 172.21.15.29:34020) 2024-09-21T01:54:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d5c66640 1 -- 172.21.15.29:0/4192061402 learned_addr learned my addr 172.21.15.29:0/4192061402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d6467640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a3469800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.360+0000 7fd7d5465640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39254/0 (socket says 172.21.15.29:39254) 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 -- 172.21.15.29:0/4192061402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 msgr2=0x5579a3468d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 0x5579a3468d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 -- 172.21.15.29:0/4192061402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 msgr2=0x5579a3469800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a3469800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 -- 172.21.15.29:0/4192061402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a338bc90 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7d5465640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00eafd0 tx=0x7fd7c00b6f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a737045cfd259298 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.361+0000 7fd7b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00a95d0 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c8004030 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd7c00e70d0 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd7c00acea0 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7de43e740 10 client.26441 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.362+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5579a3477be0 con 0x5579a3095390 2024-09-21T01:54:28.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c0005680 con 0x5579a3095390 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd7c00ac400 con 0x5579a3095390 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 10 client.26441 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 10 client.26441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 20 client.26441 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a30749f0 0x5579a347b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 4 client.26441 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd7c8004030 con 0x5579a30749f0 2024-09-21T01:54:28.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.363+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a30749f0 0x5579a347b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.364+0000 7fd7b97fa640 10 client.26441 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.364+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a30749f0 0x5579a347b4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f6a72cbed84eb07 server_cookie=2ca3dd52419fa0ca in_seq=0 out_seq=0 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.364+0000 7fd7b97fa640 1 -- 172.21.15.29:0/4192061402 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fd7c8004030 con 0x5579a30749f0 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.364+0000 7fd7b97fa640 10 client.26441 handle_command_reply: tid=0 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.364+0000 7fd7de43e740 1 client.26441 shutdown 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 msgr2=0x5579a3469280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00eafd0 tx=0x7fd7c00b6f40 comp rx=0 tx=0).stop 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 shutdown_connections 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a30749f0 0x5579a347b4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a3469800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a3095390 0x5579a3469280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4192061402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a308d1b0 0x5579a3468d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7b97fa640 0 client.26441 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:54:28.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 >> 172.21.15.29:0/4192061402 conn(0x5579a30aa460 msgr2=0x5579a3460830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 shutdown_connections 2024-09-21T01:54:28.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 1 -- 172.21.15.29:0/4192061402 wait complete. 2024-09-21T01:54:28.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.365+0000 7fd7de43e740 20 client.26441 trim_cache size 0 max 16384 2024-09-21T01:54:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.386+0000 7fd7de43e740 1 Processor -- start 2024-09-21T01:54:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 -- start start 2024-09-21T01:54:28.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 0x5579a3469200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a345abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a345b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b5ad0 con 0x5579a30b6070 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b7640 con 0x5579a308d1b0 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.387+0000 7fd7de43e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b61a0 con 0x5579a3095390 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 0x5579a3469200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a345abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a345abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34034/0 (socket says 172.21.15.29:34034) 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5465640 1 -- 172.21.15.29:0/4173821434 learned_addr learned my addr 172.21.15.29:0/4173821434 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:28.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d6467640 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a345b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 -- 172.21.15.29:0/4173821434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 msgr2=0x5579a345abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a345abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 -- 172.21.15.29:0/4173821434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 msgr2=0x5579a345b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a345b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.388+0000 7fd7d5c66640 1 -- 172.21.15.29:0/4173821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a345c350 con 0x5579a308d1b0 2024-09-21T01:54:28.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.389+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 0x5579a3469200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5579a345bff0 tx=0x7fd7cc03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b24a6c659f088d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.389+0000 7fd7bb7fe640 1 -- 172.21.15.29:0/4173821434 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc035040 con 0x5579a308d1b0 2024-09-21T01:54:28.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.389+0000 7fd7bb7fe640 1 -- 172.21.15.29:0/4173821434 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc004eb0 con 0x5579a308d1b0 2024-09-21T01:54:28.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.389+0000 7fd7bb7fe640 1 -- 172.21.15.29:0/4173821434 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0324b0 con 0x5579a308d1b0 2024-09-21T01:54:28.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 -- 172.21.15.29:0/4173821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 msgr2=0x5579a3469200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 0x5579a3469200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5579a345bff0 tx=0x7fd7cc03f000 comp rx=0 tx=0).stop 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 -- 172.21.15.29:0/4173821434 shutdown_connections 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a30b6070 0x5579a345b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a345abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 --2- 172.21.15.29:0/4173821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a308d1b0 0x5579a3469200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 -- 172.21.15.29:0/4173821434 >> 172.21.15.29:0/4173821434 conn(0x5579a30aa460 msgr2=0x5579a3468af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 -- 172.21.15.29:0/4173821434 shutdown_connections 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.390+0000 7fd7de43e740 1 -- 172.21.15.29:0/4173821434 wait complete. 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.391+0000 7fd7de43e740 1 -- start start 2024-09-21T01:54:28.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.391+0000 7fd7de43e740 1 Processor -- start 2024-09-21T01:54:28.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.392+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a3471770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.392+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a346cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7de43e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 0x5579a346d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7de43e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b6f70 con 0x5579a308d1b0 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7de43e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b68a0 con 0x5579a30b6070 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7de43e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01b3ab0 con 0x5579a3095390 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7de43e740 10 client.0 _ll_register_callbacks cb 0x7fd7c00eaca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d6467640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 0x5579a346d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a346cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d6467640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 0x5579a346d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39278/0 (socket says 172.21.15.29:39278) 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d6467640 1 -- 172.21.15.29:0/713867505 learned_addr learned my addr 172.21.15.29:0/713867505 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 -- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 msgr2=0x5579a346d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 0x5579a346d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 -- 172.21.15.29:0/713867505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 msgr2=0x5579a3471770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a3471770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.393+0000 7fd7d5465640 1 -- 172.21.15.29:0/713867505 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a346d750 con 0x5579a3095390 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7d5465640 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a346cc20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00aa9a0 tx=0x7fd7c0005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd9b849ea410a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:28.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00ad080 con 0x5579a3095390 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd7c00b33a0 con 0x5579a3095390 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc004eb0 con 0x5579a3095390 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.394+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd7c00ad500 con 0x5579a3095390 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.395+0000 7fd7de43e740 10 client.26464 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:28.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.395+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5579a346d750 con 0x5579a3095390 2024-09-21T01:54:28.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc004eb0 con 0x5579a3095390 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd7c00b6690 con 0x5579a3095390 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 10 client.26464 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 10 client.26464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 20 client.26464 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a3049da0 0x5579a34e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 4 client.26464 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579a33d5df0 con 0x5579a3049da0 2024-09-21T01:54:28.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.397+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a3049da0 0x5579a34e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7b97fa640 10 client.26464 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7d5c66640 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a3049da0 0x5579a34e7fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b76c446fec53e1b6 server_cookie=a7aa53e2a02be025 in_seq=0 out_seq=0 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7b97fa640 1 -- 172.21.15.29:0/713867505 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579a33d5df0 con 0x5579a3049da0 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7b97fa640 10 client.26464 handle_command_reply: tid=0 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7de43e740 1 client.26464 shutdown 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 msgr2=0x5579a346cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.398+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a346cc20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00aa9a0 tx=0x7fd7c0005440 comp rx=0 tx=0).stop 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 shutdown_connections 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5579a3049da0 0x5579a34e7fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579a30b6070 0x5579a346d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579a3095390 0x5579a346cc20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 --2- 172.21.15.29:0/713867505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5579a308d1b0 0x5579a3471770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 >> 172.21.15.29:0/713867505 conn(0x5579a30aa460 msgr2=0x5579a346a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7b97fa640 0 client.26464 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:54:28.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 shutdown_connections 2024-09-21T01:54:28.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 1 -- 172.21.15.29:0/713867505 wait complete. 2024-09-21T01:54:28.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.399+0000 7fd7de43e740 20 client.26464 trim_cache size 0 max 16384 2024-09-21T01:54:28.401 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 msgr2=0x7fd7bc0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 0x7fd7bc0b8b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5579a345c320 tx=0x7fd7cc000bc0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 msgr2=0x7fd7d01a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9c0 tx=0x7fd7c000ee80 comp rx=0 tx=0).stop 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 shutdown_connections 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7bc0b6650 0x7fd7bc0b8b60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d0107be0 0x7fd7d01a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d01030f0 0x7fd7d01a10b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 --2- 172.21.15.29:0/913299422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7d01027a0 0x7fd7d01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:28.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.401+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 >> 172.21.15.29:0/913299422 conn(0x7fd7d0079730 msgr2=0x7fd7d01055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:28.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.402+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 shutdown_connections 2024-09-21T01:54:28.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:28.402+0000 7fd7d7eca640 1 -- 172.21.15.29:0/913299422 wait complete. 2024-09-21T01:54:29.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:54:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:29 smithi066 ceph-mon[35001]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 698 op/s 2024-09-21T01:54:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:29 smithi029 ceph-mon[31228]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 698 op/s 2024-09-21T01:54:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:29 smithi190 ceph-mon[34368]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 698 op/s 2024-09-21T01:54:31.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 -- 172.21.15.29:0/1865090968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 msgr2=0x7fd58c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 --2- 172.21.15.29:0/1865090968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1087b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009980 tx=0x7fd57c02c630 comp rx=0 tx=0).stop 2024-09-21T01:54:31.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 -- 172.21.15.29:0/1865090968 shutdown_connections 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 --2- 172.21.15.29:0/1865090968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 0x7fd58c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 --2- 172.21.15.29:0/1865090968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1087b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 --2- 172.21.15.29:0/1865090968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 -- 172.21.15.29:0/1865090968 >> 172.21.15.29:0/1865090968 conn(0x7fd58c079730 msgr2=0x7fd58c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 -- 172.21.15.29:0/1865090968 shutdown_connections 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.579+0000 7fd5945ac640 1 -- 172.21.15.29:0/1865090968 wait complete. 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.580+0000 7fd5945ac640 1 Processor -- start 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.580+0000 7fd5945ac640 1 -- start start 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.580+0000 7fd5945ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.580+0000 7fd5945ac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd5945ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 0x7fd58c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd591b47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd591b47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39304/0 (socket says 172.21.15.29:39304) 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd591b47640 1 -- 172.21.15.29:0/617349503 learned_addr learned my addr 172.21.15.29:0/617349503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c111860 con 0x7fd58c108cf0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.581+0000 7fd592348640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1116e0 con 0x7fd58c1083a0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1119e0 con 0x7fd58c102a60 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592b49640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 0x7fd58c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592348640 1 -- 172.21.15.29:0/617349503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 msgr2=0x7fd58c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592348640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592348640 1 -- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 msgr2=0x7fd58c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592348640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 0x7fd58c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.582+0000 7fd592348640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c19e280 con 0x7fd58c102a60 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.583+0000 7fd592348640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c198310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d6d0 tx=0x7fd58800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.583+0000 7fd5837fe640 1 -- 172.21.15.29:0/617349503 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588017070 con 0x7fd58c102a60 2024-09-21T01:54:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.583+0000 7fd5837fe640 1 -- 172.21.15.29:0/617349503 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd588009820 con 0x7fd58c102a60 2024-09-21T01:54:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.583+0000 7fd5837fe640 1 -- 172.21.15.29:0/617349503 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588004c40 con 0x7fd58c102a60 2024-09-21T01:54:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.584+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c19e570 con 0x7fd58c102a60 2024-09-21T01:54:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.584+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1a4cd0 con 0x7fd58c102a60 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd5837fe640 1 -- 172.21.15.29:0/617349503 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd588005220 con 0x7fd58c102a60 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 Processor -- start 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 -- start start 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 0x55b6a09a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09a7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b6a091cea0 con 0x55b6a05ff070 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b6a090c2d0 con 0x55b6a05de390 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.587+0000 7fd59ab20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6a09100e0 con 0x55b6a05d61b0 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592b49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 0x55b6a09a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34054/0 (socket says 172.21.15.29:34054) 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 -- 172.21.15.29:0/3973246580 learned_addr learned my addr 172.21.15.29:0/3973246580 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd591b47640 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 -- 172.21.15.29:0/3973246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 msgr2=0x55b6a09a7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09a7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 -- 172.21.15.29:0/3973246580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 msgr2=0x55b6a09a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd5837fe640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 0x7fd5780b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 0x55b6a09a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592348640 1 -- 172.21.15.29:0/3973246580 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a066a650 con 0x55b6a05d61b0 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd592b49640 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 0x55b6a09a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.588+0000 7fd591b47640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 0x7fd5780b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.589+0000 7fd5837fe640 1 -- 172.21.15.29:0/617349503 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd5880e6ce0 con 0x7fd58c102a60 2024-09-21T01:54:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.589+0000 7fd592348640 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5880ba000 tx=0x7fd5880ad070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb7d65fffdadcb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.590+0000 7fd580ff9640 1 -- 172.21.15.29:0/3973246580 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588020070 con 0x55b6a05d61b0 2024-09-21T01:54:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.590+0000 7fd580ff9640 1 -- 172.21.15.29:0/3973246580 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5880ba460 con 0x55b6a05d61b0 2024-09-21T01:54:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.590+0000 7fd580ff9640 1 -- 172.21.15.29:0/3973246580 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5880ba260 con 0x55b6a05d61b0 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 -- 172.21.15.29:0/3973246580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 msgr2=0x55b6a09a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd591b47640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 0x7fd5780b8b70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd57c02cfd0 tx=0x7fd57c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5880ba000 tx=0x7fd5880ad070 comp rx=0 tx=0).stop 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 -- 172.21.15.29:0/3973246580 shutdown_connections 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09a7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05d61b0 0x55b6a09a6be0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3973246580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05ff070 0x55b6a09a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 -- 172.21.15.29:0/3973246580 >> 172.21.15.29:0/3973246580 conn(0x55b6a05f3460 msgr2=0x55b6a09a4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 -- 172.21.15.29:0/3973246580 shutdown_connections 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.591+0000 7fd59ab20740 1 -- 172.21.15.29:0/3973246580 wait complete. 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 -- start start 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 Processor -- start 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05d61b0 0x55b6a09b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05ff070 0x55b6a09b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b6a076d700 con 0x55b6a05de390 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b6a08a7160 con 0x55b6a05d61b0 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6a09b4f20 con 0x55b6a05ff070 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd59ab20740 10 client.0 _ll_register_callbacks cb 0x55b6a08c09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43208/0 (socket says 172.21.15.29:43208) 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd591b47640 1 -- 172.21.15.29:0/3134625216 learned_addr learned my addr 172.21.15.29:0/3134625216 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd592348640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05d61b0 0x55b6a09b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.592+0000 7fd592b49640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05ff070 0x55b6a09b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 -- 172.21.15.29:0/3134625216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05ff070 msgr2=0x55b6a09b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05ff070 0x55b6a09b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 -- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05d61b0 msgr2=0x55b6a09b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05d61b0 0x55b6a09b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 -- 172.21.15.29:0/3134625216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a08d4c90 con 0x55b6a05de390 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd591b47640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0096c0 tx=0x7fd57c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2422ae4a3136e7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c04e030 con 0x55b6a05de390 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd57c0048d0 con 0x55b6a05de390 2024-09-21T01:54:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.593+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6a091edf0 con 0x55b6a05de390 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd57c037040 con 0x55b6a05de390 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 10 client.16707 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b6a09c0bc0 con 0x55b6a05de390 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6a091edf0 con 0x55b6a05de390 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd57c003680 con 0x55b6a05de390 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 10 client.16707 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 20 client.16707 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05bd9f0 0x55b6a09c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 4 client.16707 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.594+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a091a8a0 con 0x55b6a05bd9f0 2024-09-21T01:54:31.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.595+0000 7fd592348640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05bd9f0 0x55b6a09c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.596+0000 7fd592348640 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05bd9f0 0x55b6a09c4210 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9067d266deafe671 server_cookie=10f5fee4c88b303f in_seq=0 out_seq=0 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.596+0000 7fd575ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.597+0000 7fd575ffb640 1 -- 172.21.15.29:0/3134625216 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6a091a8a0 con 0x55b6a05bd9f0 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.597+0000 7fd575ffb640 10 client.16707 handle_command_reply: tid=0 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.597+0000 7fd59ab20740 1 client.16707 shutdown 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.598+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 msgr2=0x55b6a09b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.598+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0096c0 tx=0x7fd57c009260 comp rx=0 tx=0).stop 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.598+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 shutdown_connections 2024-09-21T01:54:31.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.598+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05bd9f0 0x55b6a09c4210 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd575ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05ff070 0x55b6a09b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05de390 0x55b6a09b2220 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 --2- 172.21.15.29:0/3134625216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05d61b0 0x55b6a09b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 >> 172.21.15.29:0/3134625216 conn(0x55b6a05f3460 msgr2=0x55b6a09a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 shutdown_connections 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 1 -- 172.21.15.29:0/3134625216 wait complete. 2024-09-21T01:54:31.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.599+0000 7fd59ab20740 20 client.16707 trim_cache size 0 max 16384 2024-09-21T01:54:31.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.616+0000 7fd59ab20740 1 Processor -- start 2024-09-21T01:54:31.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.616+0000 7fd59ab20740 1 -- start start 2024-09-21T01:54:31.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.616+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.616+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.616+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd59ab20740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1ad140 con 0x55b6a05d61b0 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd59ab20740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1aecb0 con 0x55b6a05de390 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd59ab20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1ad810 con 0x55b6a05bd9f0 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43220/0 (socket says 172.21.15.29:43220) 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 -- 172.21.15.29:0/412537839 learned_addr learned my addr 172.21.15.29:0/412537839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd592b49640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd592348640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 -- 172.21.15.29:0/412537839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 msgr2=0x55b6a09a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 -- 172.21.15.29:0/412537839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 msgr2=0x55b6a09bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 -- 172.21.15.29:0/412537839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a09b2fa0 con 0x55b6a05d61b0 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd592348640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd592b49640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:31.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.617+0000 7fd591b47640 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd57c033890 tx=0x7fd57c02c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8628875e04fe9054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.618+0000 7fd577fff640 1 -- 172.21.15.29:0/412537839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c062020 con 0x55b6a05d61b0 2024-09-21T01:54:31.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.618+0000 7fd577fff640 1 -- 172.21.15.29:0/412537839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57c0048d0 con 0x55b6a05d61b0 2024-09-21T01:54:31.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.618+0000 7fd577fff640 1 -- 172.21.15.29:0/412537839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c04e040 con 0x55b6a05d61b0 2024-09-21T01:54:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.619+0000 7fd59ab20740 1 -- 172.21.15.29:0/412537839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 msgr2=0x55b6a09b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.619+0000 7fd59ab20740 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd57c033890 tx=0x7fd57c02c630 comp rx=0 tx=0).stop 2024-09-21T01:54:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.626+0000 7fd59ab20740 1 -- 172.21.15.29:0/412537839 shutdown_connections 2024-09-21T01:54:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.626+0000 7fd59ab20740 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a09bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.626+0000 7fd59ab20740 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b3bc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.626+0000 7fd59ab20740 1 --2- 172.21.15.29:0/412537839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.626+0000 7fd59ab20740 1 -- 172.21.15.29:0/412537839 >> 172.21.15.29:0/412537839 conn(0x55b6a05f3460 msgr2=0x55b6a09bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.627+0000 7fd59ab20740 1 -- 172.21.15.29:0/412537839 shutdown_connections 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.629+0000 7fd59ab20740 1 -- 172.21.15.29:0/412537839 wait complete. 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.629+0000 7fd59ab20740 1 -- start start 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd59ab20740 1 Processor -- start 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd59ab20740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a0a38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd591b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43232/0 (socket says 172.21.15.29:43232) 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.630+0000 7fd591b47640 1 -- 172.21.15.29:0/2610522434 learned_addr learned my addr 172.21.15.29:0/2610522434 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.631+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1ae5e0 con 0x55b6a05d61b0 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.631+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1adf10 con 0x55b6a05de390 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.631+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1ab120 con 0x55b6a05bd9f0 2024-09-21T01:54:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.631+0000 7fd59ab20740 10 client.0 _ll_register_callbacks cb 0x7fd57c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:31.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.631+0000 7fd592b49640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a0a38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.632+0000 7fd592348640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 -- 172.21.15.29:0/2610522434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 msgr2=0x55b6a09b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 -- 172.21.15.29:0/2610522434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 msgr2=0x55b6a09b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a0a39220 con 0x55b6a05de390 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd592b49640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a0a38bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b6a09a8780 tx=0x7fd58400b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dbb9cad5968d546 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.633+0000 7fd575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584004d20 con 0x55b6a05de390 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57c0048d0 con 0x55b6a05de390 2024-09-21T01:54:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd5840122e0 con 0x55b6a05de390 2024-09-21T01:54:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd584012520 con 0x55b6a05de390 2024-09-21T01:54:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd59ab20740 10 client.26465 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.634+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b6a0a405f0 con 0x55b6a05de390 2024-09-21T01:54:31.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57c0048d0 con 0x55b6a05de390 2024-09-21T01:54:31.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd584021410 con 0x55b6a05de390 2024-09-21T01:54:31.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 10 client.26465 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:31.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:31.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 20 client.26465 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:31.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05ff070 0x55b6a0a42dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:31.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 4 client.26465 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:54:31.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.639+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6a0917cd0 con 0x55b6a05ff070 2024-09-21T01:54:31.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.642+0000 7fd592348640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05ff070 0x55b6a0a42dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:31.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.644+0000 7fd592348640 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05ff070 0x55b6a0a42dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6885e466be683c server_cookie=baa69849b418b00a in_seq=0 out_seq=0 2024-09-21T01:54:31.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.645+0000 7fd575ffb640 10 client.26465 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:31.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.645+0000 7fd575ffb640 1 -- 172.21.15.29:0/2610522434 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6a0917cd0 con 0x55b6a05ff070 2024-09-21T01:54:31.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.645+0000 7fd575ffb640 10 client.26465 handle_command_reply: tid=0 2024-09-21T01:54:31.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.645+0000 7fd59ab20740 1 client.26465 shutdown 2024-09-21T01:54:31.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.647+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 msgr2=0x55b6a0a38bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a0a38bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b6a09a8780 tx=0x7fd58400b6f0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 shutdown_connections 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b6a05ff070 0x55b6a0a42dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6a05de390 0x55b6a0a38bb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b6a05d61b0 0x55b6a09b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 --2- 172.21.15.29:0/2610522434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a05bd9f0 0x55b6a09b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 >> 172.21.15.29:0/2610522434 conn(0x55b6a05f3460 msgr2=0x55b6a09b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.648+0000 7fd575ffb640 0 client.26465 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:31.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.650+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 shutdown_connections 2024-09-21T01:54:31.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.650+0000 7fd59ab20740 1 -- 172.21.15.29:0/2610522434 wait complete. 2024-09-21T01:54:31.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.650+0000 7fd59ab20740 20 client.26465 trim_cache size 0 max 16384 2024-09-21T01:54:31.651 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 msgr2=0x7fd5780b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 0x7fd5780b8b70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd57c02cfd0 tx=0x7fd57c0058d0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 msgr2=0x7fd58c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c198310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d6d0 tx=0x7fd58800db90 comp rx=0 tx=0).stop 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 shutdown_connections 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd5780b6660 0x7fd5780b8b70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd58c108cf0 0x7fd58c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd58c1083a0 0x7fd58c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 --2- 172.21.15.29:0/617349503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd58c102a60 0x7fd58c198310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:31.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 >> 172.21.15.29:0/617349503 conn(0x7fd58c079730 msgr2=0x7fd58c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:31.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.652+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 shutdown_connections 2024-09-21T01:54:31.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:31.653+0000 7fd5945ac640 1 -- 172.21.15.29:0/617349503 wait complete. 2024-09-21T01:54:31.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:31 smithi066 ceph-mon[35001]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-09-21T01:54:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:31 smithi029 ceph-mon[31228]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-09-21T01:54:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:31 smithi190 ceph-mon[34368]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-09-21T01:54:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 355.0 s, 1908.6 tps, lat 51.978 ms stddev 57.984 2024-09-21T01:54:33.418 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:54:33.419 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:54:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:33 smithi066 ceph-mon[35001]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 742 op/s 2024-09-21T01:54:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:33.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:33 smithi029 ceph-mon[31228]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 742 op/s 2024-09-21T01:54:33.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:33.876 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:33 smithi190 ceph-mon[34368]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 742 op/s 2024-09-21T01:54:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.418+0000 7f636e12b640 1 -- 172.21.15.29:0/1443832067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f636806c070 msgr2=0x7f6368106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.418+0000 7f636e12b640 1 --2- 172.21.15.29:0/1443832067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f636806c070 0x7f6368106350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6358009860 tx=0x7f635802a360 comp rx=0 tx=0).stop 2024-09-21T01:54:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.424+0000 7f636e12b640 1 -- 172.21.15.29:0/1443832067 shutdown_connections 2024-09-21T01:54:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.424+0000 7f636e12b640 1 --2- 172.21.15.29:0/1443832067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63681093f0 0x7f636810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.424+0000 7f636e12b640 1 --2- 172.21.15.29:0/1443832067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f6368108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.424+0000 7f636e12b640 1 --2- 172.21.15.29:0/1443832067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f636806c070 0x7f6368106350 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.424+0000 7f636e12b640 1 -- 172.21.15.29:0/1443832067 >> 172.21.15.29:0/1443832067 conn(0x7f63680fdf30 msgr2=0x7f63681003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.426+0000 7f636e12b640 1 -- 172.21.15.29:0/1443832067 shutdown_connections 2024-09-21T01:54:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.426+0000 7f636e12b640 1 -- 172.21.15.29:0/1443832067 wait complete. 2024-09-21T01:54:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.428+0000 7f636e12b640 1 Processor -- start 2024-09-21T01:54:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.428+0000 7f636e12b640 1 -- start start 2024-09-21T01:54:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f636e12b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f636e12b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f636819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f636e12b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63681093f0 0x7f636819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f63677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f63677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52324/0 (socket says 172.21.15.29:52324) 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f636819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f636819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58316/0 (socket says 172.21.15.29:58316) 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.429+0000 7f63677fe640 1 -- 172.21.15.29:0/3489666477 learned_addr learned my addr 172.21.15.29:0/3489666477 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f63681115a0 con 0x7f63681093f0 2024-09-21T01:54:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6368111420 con 0x7f6368106890 2024-09-21T01:54:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6368111720 con 0x7f636806c070 2024-09-21T01:54:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 -- 172.21.15.29:0/3489666477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 msgr2=0x7f636819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f636819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 -- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63681093f0 msgr2=0x7f636819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63681093f0 0x7f636819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636819dc30 con 0x7f636806c070 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.430+0000 7f63677fe640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6358009980 tx=0x7f6358032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.432+0000 7f6364ff9640 1 -- 172.21.15.29:0/3489666477 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358037050 con 0x7f636806c070 2024-09-21T01:54:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.432+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636819dec0 con 0x7f636806c070 2024-09-21T01:54:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.433+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636819e1d0 con 0x7f636806c070 2024-09-21T01:54:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.433+0000 7f6364ff9640 1 -- 172.21.15.29:0/3489666477 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6358043070 con 0x7f636806c070 2024-09-21T01:54:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.433+0000 7f6364ff9640 1 -- 172.21.15.29:0/3489666477 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358003df0 con 0x7f636806c070 2024-09-21T01:54:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.439+0000 7f6375099740 1 Processor -- start 2024-09-21T01:54:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6364ff9640 1 -- 172.21.15.29:0/3489666477 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6358004040 con 0x7f636806c070 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 -- start start 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dc031b0 0x55fd1dfaae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd1df1d6f0 con 0x55fd1dc031b0 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fd1df0cb20 con 0x55fd1dbe24d0 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.440+0000 7f6375099740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd1df10930 con 0x55fd1dbda2f0 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58318/0 (socket says 172.21.15.29:58318) 2024-09-21T01:54:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 -- 172.21.15.29:0/2407969614 learned_addr learned my addr 172.21.15.29:0/2407969614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 -- 172.21.15.29:0/2407969614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 msgr2=0x55fd1dfab4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfab4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 -- 172.21.15.29:0/2407969614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dc031b0 msgr2=0x55fd1dfaae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dc031b0 0x55fd1dfaae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 -- 172.21.15.29:0/2407969614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1ded2530 con 0x55fd1dbe24d0 2024-09-21T01:54:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.441+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63600103d0 tx=0x7f6360010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae49e04faf264572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.442+0000 7f635dffb640 1 -- 172.21.15.29:0/2407969614 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360011700 con 0x55fd1dbe24d0 2024-09-21T01:54:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.442+0000 7f635dffb640 1 -- 172.21.15.29:0/2407969614 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360011d50 con 0x55fd1dbe24d0 2024-09-21T01:54:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.443+0000 7f6375099740 1 -- 172.21.15.29:0/2407969614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 msgr2=0x55fd1dfac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.443+0000 7f6375099740 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63600103d0 tx=0x7f6360010890 comp rx=0 tx=0).stop 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6375099740 1 -- 172.21.15.29:0/2407969614 shutdown_connections 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6375099740 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfac010 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6375099740 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6375099740 1 --2- 172.21.15.29:0/2407969614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dc031b0 0x55fd1dfaae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6375099740 1 -- 172.21.15.29:0/2407969614 >> 172.21.15.29:0/2407969614 conn(0x55fd1dbf7300 msgr2=0x55fd1dfa68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6364ff9640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 0x7f63540b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.445+0000 7f6364ff9640 1 -- 172.21.15.29:0/3489666477 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6358107b50 con 0x7f636806c070 2024-09-21T01:54:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.448+0000 7f6366ffd640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 0x7f63540b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.448+0000 7f6375099740 1 -- 172.21.15.29:0/2407969614 shutdown_connections 2024-09-21T01:54:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.448+0000 7f6375099740 1 -- 172.21.15.29:0/2407969614 wait complete. 2024-09-21T01:54:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.449+0000 7f6375099740 1 -- start start 2024-09-21T01:54:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.449+0000 7f6375099740 1 Processor -- start 2024-09-21T01:54:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6375099740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfcdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6375099740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6375099740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6375099740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd1de44ab0 con 0x55fd1dbe24d0 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6375099740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fd1deb2730 con 0x55fd1dbda2f0 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.450+0000 7f6367fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.451+0000 7f6367fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52336/0 (socket says 172.21.15.29:52336) 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.451+0000 7f6367fff640 1 -- 172.21.15.29:0/2778536364 learned_addr learned my addr 172.21.15.29:0/2778536364 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.451+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd1dfacbd0 con 0x55fd1dc031b0 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.451+0000 7f6375099740 10 client.0 _ll_register_callbacks cb 0x55fd1df066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 -- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbda2f0 msgr2=0x55fd1dfcdeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfcdeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 -- 172.21.15.29:0/2778536364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 msgr2=0x55fd1dfce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 -- 172.21.15.29:0/2778536364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1dc6e790 con 0x55fd1dc031b0 2024-09-21T01:54:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6366ffd640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 0x7f63540b8c30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6360005c10 tx=0x7f6360005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6367fff640 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f635000bc70 tx=0x7f635000d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3112f085608e8cba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.452+0000 7f6346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350004230 con 0x55fd1dc031b0 2024-09-21T01:54:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f63500043d0 con 0x55fd1dc031b0 2024-09-21T01:54:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360011d50 con 0x55fd1dc031b0 2024-09-21T01:54:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f63500035f0 con 0x55fd1dc031b0 2024-09-21T01:54:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6375099740 10 client.26491 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd1dce7620 con 0x55fd1dc031b0 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.453+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360011d50 con 0x55fd1dc031b0 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f63500106d0 con 0x55fd1dc031b0 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 10 client.26491 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 10 client.26491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 20 client.26491 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dbc1d80 0x55fd1dfdb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 4 client.26491 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd1df1f640 con 0x55fd1dbc1d80 2024-09-21T01:54:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.454+0000 7f63677fe640 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dbc1d80 0x55fd1dfdb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.455+0000 7f6346ffd640 10 client.26491 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.455+0000 7f63677fe640 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dbc1d80 0x55fd1dfdb710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6636db639e220d0 server_cookie=a1d819df5db74aa9 in_seq=0 out_seq=0 2024-09-21T01:54:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.456+0000 7f6346ffd640 1 -- 172.21.15.29:0/2778536364 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd1df1f640 con 0x55fd1dbc1d80 2024-09-21T01:54:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.456+0000 7f6346ffd640 10 client.26491 handle_command_reply: tid=0 2024-09-21T01:54:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.456+0000 7f6375099740 1 client.26491 shutdown 2024-09-21T01:54:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.457+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 msgr2=0x55fd1dfcf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.457+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f635000bc70 tx=0x7f635000d6b0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 shutdown_connections 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dbc1d80 0x55fd1dfdb710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dc031b0 0x55fd1dfcf0e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6346ffd640 0 client.26491 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 --2- 172.21.15.29:0/2778536364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfcdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.458+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 >> 172.21.15.29:0/2778536364 conn(0x55fd1dbf7300 msgr2=0x55fd1dfa4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.459+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 shutdown_connections 2024-09-21T01:54:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.459+0000 7f6375099740 1 -- 172.21.15.29:0/2778536364 wait complete. 2024-09-21T01:54:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.459+0000 7f6375099740 20 client.26491 trim_cache size 0 max 16384 2024-09-21T01:54:34.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.471+0000 7f31c759e640 1 -- 172.21.15.29:0/1640673790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810a490 msgr2=0x7f31c810a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.471+0000 7f31c759e640 1 --2- 172.21.15.29:0/1640673790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810a490 0x7f31c810a8a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009930 tx=0x7f31b802a440 comp rx=0 tx=0).stop 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 -- 172.21.15.29:0/1640673790 shutdown_connections 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 --2- 172.21.15.29:0/1640673790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810d000 0x7f31c8116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 --2- 172.21.15.29:0/1640673790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c810c6b0 0x7f31c810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 --2- 172.21.15.29:0/1640673790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810a490 0x7f31c810a8a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 -- 172.21.15.29:0/1640673790 >> 172.21.15.29:0/1640673790 conn(0x7f31c8079420 msgr2=0x7f31c807b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 -- 172.21.15.29:0/1640673790 shutdown_connections 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.472+0000 7f31c759e640 1 -- 172.21.15.29:0/1640673790 wait complete. 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.473+0000 7f31c759e640 1 Processor -- start 2024-09-21T01:54:34.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.473+0000 7f31c759e640 1 -- start start 2024-09-21T01:54:34.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 0x7f31c81a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c810c6b0 0x7f31c81a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8119ed0 con 0x7f31c810c6b0 2024-09-21T01:54:34.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8119d50 con 0x7f31c810d000 2024-09-21T01:54:34.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31c811a050 con 0x7f31c810a490 2024-09-21T01:54:34.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 0x7f31c81a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58340/0 (socket says 172.21.15.29:58340) 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 0x7f31c81a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52360/0 (socket says 172.21.15.29:52360) 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.474+0000 7f31c659c640 1 -- 172.21.15.29:0/3002692688 learned_addr learned my addr 172.21.15.29:0/3002692688 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.475+0000 7f31c6d9d640 1 -- 172.21.15.29:0/3002692688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 msgr2=0x7f31c81a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.475+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 0x7f31c81a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.475+0000 7f31c6d9d640 1 -- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c810c6b0 msgr2=0x7f31c81a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.475+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c810c6b0 0x7f31c81a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.475+0000 7f31c6d9d640 1 -- 172.21.15.29:0/3002692688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c81a6870 con 0x7f31c810d000 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009fd0 tx=0x7f31c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31b77fe640 1 -- 172.21.15.29:0/3002692688 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000ce20 con 0x7f31c810d000 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31b77fe640 1 -- 172.21.15.29:0/3002692688 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c0005020 con 0x7f31c810d000 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31b77fe640 1 -- 172.21.15.29:0/3002692688 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0010690 con 0x7f31c810d000 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c81a6b60 con 0x7f31c810d000 2024-09-21T01:54:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.476+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c81ad2c0 con 0x7f31c810d000 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.480+0000 7f31d3393740 1 Processor -- start 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.483+0000 7f31d3393740 1 -- start start 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.483+0000 7f31d3393740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 0x55eb92b93610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.483+0000 7f31b77fe640 1 -- 172.21.15.29:0/3002692688 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f31c00054d0 con 0x7f31c810d000 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.483+0000 7f31c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 0x55eb92b93610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 0x55eb92b93610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58348/0 (socket says 172.21.15.29:58348) 2024-09-21T01:54:34.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31c5d9b640 1 -- 172.21.15.29:0/1990713778 learned_addr learned my addr 172.21.15.29:0/1990713778 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927cb670 0x55eb92b94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55eb92b09e60 con 0x55eb927c3490 2024-09-21T01:54:34.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb92af9290 con 0x55eb927ec350 2024-09-21T01:54:34.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb92afd0a0 con 0x55eb927cb670 2024-09-21T01:54:34.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.484+0000 7f31c659c640 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.485+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927cb670 0x55eb92b94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 -- 172.21.15.29:0/1990713778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927cb670 msgr2=0x55eb92b94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927cb670 0x55eb92b94090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 -- 172.21.15.29:0/1990713778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 msgr2=0x55eb92b93610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 0x55eb92b93610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 -- 172.21.15.29:0/1990713778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb92857930 con 0x55eb927c3490 2024-09-21T01:54:34.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.486+0000 7f31c659c640 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b93b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009900 tx=0x7f31b8031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b224524c14c044f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.487+0000 7f31b77fe640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 0x7f31b00b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.487+0000 7f31b77fe640 1 -- 172.21.15.29:0/3002692688 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f31c00e7a10 con 0x7f31c810d000 2024-09-21T01:54:34.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.487+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 0x7f31b00b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.488+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 0x7f31b00b8bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009d70 tx=0x7f31bc009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.489+0000 7f31b4ff9640 1 -- 172.21.15.29:0/1990713778 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8040070 con 0x55eb927c3490 2024-09-21T01:54:34.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.489+0000 7f31b4ff9640 1 -- 172.21.15.29:0/1990713778 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b8004030 con 0x55eb927c3490 2024-09-21T01:54:34.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.491+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 msgr2=0x55eb92b93b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.491+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b93b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009900 tx=0x7f31b8031b10 comp rx=0 tx=0).stop 2024-09-21T01:54:34.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.491+0000 7f31b4ff9640 1 -- 172.21.15.29:0/1990713778 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8035040 con 0x55eb927c3490 2024-09-21T01:54:34.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.491+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 shutdown_connections 2024-09-21T01:54:34.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.491+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927cb670 0x55eb92b94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.492+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b93b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.492+0000 7f31d3393740 1 --2- 172.21.15.29:0/1990713778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927ec350 0x55eb92b93610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.492+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 >> 172.21.15.29:0/1990713778 conn(0x55eb927e0740 msgr2=0x55eb92b91500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.493+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 shutdown_connections 2024-09-21T01:54:34.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.494+0000 7f31d3393740 1 -- 172.21.15.29:0/1990713778 wait complete. 2024-09-21T01:54:34.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.495+0000 7f31d3393740 1 -- start start 2024-09-21T01:54:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.495+0000 7f6375099740 1 Processor -- start 2024-09-21T01:54:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.495+0000 7f6375099740 1 -- start start 2024-09-21T01:54:34.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.495+0000 7f31d3393740 1 Processor -- start 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfa7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f634800e540 con 0x55fd1dbe24d0 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63480100b0 con 0x55fd1dbc1d80 2024-09-21T01:54:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6375099740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f634800ec10 con 0x55fd1dbda2f0 2024-09-21T01:54:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927c3490 0x55eb92b9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6367fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6367fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38496/0 (socket says 172.21.15.29:38496) 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f6367fff640 1 -- 172.21.15.29:0/2830036267 learned_addr learned my addr 172.21.15.29:0/2830036267 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 -- 172.21.15.29:0/2830036267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 msgr2=0x55fd1dfd5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 -- 172.21.15.29:0/2830036267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbc1d80 msgr2=0x55fd1dfa7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927ec350 0x55eb92bab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfa7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 -- 172.21.15.29:0/2830036267 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1dfce8e0 con 0x55fd1dbe24d0 2024-09-21T01:54:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55eb9295a890 con 0x55eb927ec350 2024-09-21T01:54:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f6367fff640 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f635001a480 tx=0x7f6350010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e0694414bc8fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f635d7fa640 1 -- 172.21.15.29:0/2830036267 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350010870 con 0x55fd1dbe24d0 2024-09-21T01:54:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f635d7fa640 1 -- 172.21.15.29:0/2830036267 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63500054d0 con 0x55fd1dbe24d0 2024-09-21T01:54:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f635d7fa640 1 -- 172.21.15.29:0/2830036267 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635001a5a0 con 0x55fd1dbe24d0 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb92a93d40 con 0x55eb927cb670 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.496+0000 7f31d3393740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb92ba1de0 con 0x55eb927c3490 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58364/0 (socket says 172.21.15.29:58364) 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 -- 172.21.15.29:0/766578407 learned_addr learned my addr 172.21.15.29:0/766578407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c659c640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927c3490 0x55eb92b9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31d3393740 10 client.0 _ll_register_callbacks cb 0x55eb92af7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 -- 172.21.15.29:0/766578407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927c3490 msgr2=0x55eb92b9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927c3490 0x55eb92b9f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 -- 172.21.15.29:0/766578407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927ec350 msgr2=0x55eb92bab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927ec350 0x55eb92bab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927ec350 0x55eb92bab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.497+0000 7f31c5d9b640 1 -- 172.21.15.29:0/766578407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb92bab770 con 0x55eb927cb670 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005d80 tx=0x7f31bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e8b15776067092 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f31adffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00f940 con 0x55eb927cb670 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f31bc00fae0 con 0x55eb927cb670 2024-09-21T01:54:34.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.498+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b8004030 con 0x55eb927cb670 2024-09-21T01:54:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f6375099740 1 -- 172.21.15.29:0/2830036267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 msgr2=0x55fd1dfa6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f6375099740 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f635001a480 tx=0x7f6350010f30 comp rx=0 tx=0).stop 2024-09-21T01:54:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.499+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31bc00fce0 con 0x55eb927cb670 2024-09-21T01:54:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.499+0000 7f31d3393740 10 client.26486 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 -- 172.21.15.29:0/2830036267 shutdown_connections 2024-09-21T01:54:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfa6220 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 --2- 172.21.15.29:0/2830036267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfa7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.499+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55eb92bae130 con 0x55eb927cb670 2024-09-21T01:54:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.499+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31bc00fe80 con 0x55eb927cb670 2024-09-21T01:54:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 -- 172.21.15.29:0/2830036267 >> 172.21.15.29:0/2830036267 conn(0x55fd1dbf7300 msgr2=0x55fd1dfa1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.499+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f31bc01f620 con 0x55eb927cb670 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 10 client.26486 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 20 client.26486 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb927aacd0 0x55eb92bb15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 4 client.26486 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.500+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31b8004030 con 0x55eb927aacd0 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f31c659c640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb927aacd0 0x55eb92bb15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 -- 172.21.15.29:0/2830036267 shutdown_connections 2024-09-21T01:54:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.501+0000 7f6375099740 1 -- 172.21.15.29:0/2830036267 wait complete. 2024-09-21T01:54:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f31c659c640 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb927aacd0 0x55eb92bb15a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3cd9a30af7b096 server_cookie=7bde9cbeb894290 in_seq=0 out_seq=0 2024-09-21T01:54:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f31adffb640 10 client.26486 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 -- start start 2024-09-21T01:54:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 Processor -- start 2024-09-21T01:54:34.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f31adffb640 1 -- 172.21.15.29:0/766578407 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f31b8004030 con 0x55eb927aacd0 2024-09-21T01:54:34.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f31adffb640 10 client.26486 handle_command_reply: tid=0 2024-09-21T01:54:34.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f31d3393740 1 client.26486 shutdown 2024-09-21T01:54:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f634800f9e0 con 0x55fd1dbda2f0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f634800f310 con 0x55fd1dbe24d0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.502+0000 7f6375099740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f634800c520 con 0x55fd1dbc1d80 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f6375099740 10 client.0 _ll_register_callbacks cb 0x7f635000b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f6367fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f63677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f63677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52398/0 (socket says 172.21.15.29:52398) 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f6367fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfd4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58370/0 (socket says 172.21.15.29:58370) 2024-09-21T01:54:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f63677fe640 1 -- 172.21.15.29:0/2851305935 learned_addr learned my addr 172.21.15.29:0/2851305935 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 msgr2=0x55eb92b9f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 -- 172.21.15.29:0/2851305935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 msgr2=0x55fd1dfd5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfd5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 -- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 msgr2=0x55fd1dfd4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfd4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 -- 172.21.15.29:0/2851305935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1dfd4da0 con 0x55fd1dbda2f0 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6366ffd640 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5f00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6360010d90 tx=0x7f6360010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdcdaaa5bbcd40b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f6346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005d80 tx=0x7f31bc005f90 comp rx=0 tx=0).stop 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 shutdown_connections 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb927aacd0 0x55eb92bb15a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31adffb640 0 client.26486 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927ec350 0x55eb92bab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92b9f770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 --2- 172.21.15.29:0/766578407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927c3490 0x55eb92b9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.503+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 >> 172.21.15.29:0/766578407 conn(0x55eb927e0740 msgr2=0x55eb92b967f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 shutdown_connections 2024-09-21T01:54:34.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.504+0000 7f31d3393740 1 -- 172.21.15.29:0/766578407 wait complete. 2024-09-21T01:54:34.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.505+0000 7f31d3393740 20 client.26486 trim_cache size 0 max 16384 2024-09-21T01:54:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.505+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360038060 con 0x55fd1dbda2f0 2024-09-21T01:54:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.505+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63500054d0 con 0x55fd1dbda2f0 2024-09-21T01:54:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.506+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6360004980 con 0x55fd1dbda2f0 2024-09-21T01:54:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.518+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6360004b80 con 0x55fd1dbda2f0 2024-09-21T01:54:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.518+0000 7f6375099740 10 client.16728 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.518+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd1dfd3d40 con 0x55fd1dbda2f0 2024-09-21T01:54:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.519+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63500054d0 con 0x55fd1dbda2f0 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.519+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f6360032070 con 0x55fd1dbda2f0 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 10 client.16728 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 20 client.16728 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dc031b0 0x55fd1dfc6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 4 client.16728 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:54:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd1df1b0f0 con 0x55fd1dc031b0 2024-09-21T01:54:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.520+0000 7f63677fe640 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dc031b0 0x55fd1dfc6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.521+0000 7f6346ffd640 10 client.16728 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.521+0000 7f63677fe640 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dc031b0 0x55fd1dfc6440 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c8372522199dad server_cookie=199faa241c66e5 in_seq=0 out_seq=0 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6346ffd640 1 -- 172.21.15.29:0/2851305935 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd1df1b0f0 con 0x55fd1dc031b0 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6346ffd640 10 client.16728 handle_command_reply: tid=0 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 client.16728 shutdown 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 msgr2=0x55fd1dfd5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5f00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6360010d90 tx=0x7f6360010890 comp rx=0 tx=0).stop 2024-09-21T01:54:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 shutdown_connections 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd1dc031b0 0x55fd1dfc6440 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd1dbe24d0 0x55fd1dfd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd1dbda2f0 0x55fd1dfd5f00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 --2- 172.21.15.29:0/2851305935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd1dbc1d80 0x55fd1dfd5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 >> 172.21.15.29:0/2851305935 conn(0x55fd1dbf7300 msgr2=0x55fd1dfa6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6346ffd640 0 client.16728 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.533+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 shutdown_connections 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f6375099740 1 -- 172.21.15.29:0/2851305935 wait complete. 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f6375099740 20 client.16728 trim_cache size 0 max 16384 2024-09-21T01:54:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:54:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:54:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:54:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:54:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 msgr2=0x7f63540b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 0x7f63540b8c30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6360005c10 tx=0x7f6360005c70 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 msgr2=0x7f636819c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6358009980 tx=0x7f6358032cd0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 shutdown_connections 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f63540b6720 0x7f63540b8c30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63681093f0 0x7f636819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6368106890 0x7f636819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 --2- 172.21.15.29:0/3489666477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636806c070 0x7f636819c1f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 >> 172.21.15.29:0/3489666477 conn(0x7f63680fdf30 msgr2=0x7f6368104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.534+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 shutdown_connections 2024-09-21T01:54:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:34.535+0000 7f636e12b640 1 -- 172.21.15.29:0/3489666477 wait complete. 2024-09-21T01:54:34.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.538+0000 7f31d3393740 1 Processor -- start 2024-09-21T01:54:34.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.538+0000 7f31d3393740 1 -- start start 2024-09-21T01:54:34.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.538+0000 7f31d3393740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38522/0 (socket says 172.21.15.29:38522) 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 -- 172.21.15.29:0/829188813 learned_addr learned my addr 172.21.15.29:0/829188813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92bab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92b91570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8073e70 con 0x55eb927c3490 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81bfa70 con 0x55eb927cb670 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8074540 con 0x55eb927ec350 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 -- 172.21.15.29:0/829188813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 msgr2=0x55eb92b91570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92b91570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 -- 172.21.15.29:0/829188813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 msgr2=0x55eb92bab080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92bab080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.539+0000 7f31c659c640 1 -- 172.21.15.29:0/829188813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb92b95900 con 0x55eb927c3490 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.540+0000 7f31c659c640 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009230 tx=0x7f31b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e825ce56f387ea69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.541+0000 7f31affff640 1 -- 172.21.15.29:0/829188813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8040070 con 0x55eb927c3490 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.541+0000 7f31affff640 1 -- 172.21.15.29:0/829188813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b80055f0 con 0x55eb927c3490 2024-09-21T01:54:34.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.541+0000 7f31affff640 1 -- 172.21.15.29:0/829188813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8035040 con 0x55eb927c3490 2024-09-21T01:54:34.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.542+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 msgr2=0x55eb92b9df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.542+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009230 tx=0x7f31b8009260 comp rx=0 tx=0).stop 2024-09-21T01:54:34.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 shutdown_connections 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92b91570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927cb670 0x55eb92bab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 --2- 172.21.15.29:0/829188813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927c3490 0x55eb92b9df20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 >> 172.21.15.29:0/829188813 conn(0x55eb927e0740 msgr2=0x55eb92b90e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.543+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 shutdown_connections 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.544+0000 7f31d3393740 1 -- 172.21.15.29:0/829188813 wait complete. 2024-09-21T01:54:34.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.544+0000 7f31d3393740 1 -- start start 2024-09-21T01:54:34.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.544+0000 7f31d3393740 1 Processor -- start 2024-09-21T01:54:34.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927cb670 0x55eb92ba4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92c15100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31c659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8075310 con 0x55eb927cb670 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8074c40 con 0x55eb927c3490 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8071e50 con 0x55eb927ec350 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31c659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58384/0 (socket says 172.21.15.29:58384) 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31c659c640 1 -- 172.21.15.29:0/873884668 learned_addr learned my addr 172.21.15.29:0/873884668 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.545+0000 7f31d3393740 10 client.0 _ll_register_callbacks cb 0x7f31bc009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:34.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927cb670 0x55eb92ba4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92c15100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 -- 172.21.15.29:0/873884668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 msgr2=0x55eb92c15100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92c15100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 -- 172.21.15.29:0/873884668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927cb670 msgr2=0x55eb92ba4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927cb670 0x55eb92ba4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 -- 172.21.15.29:0/873884668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb92c15770 con 0x55eb927c3490 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c6d9d640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92c15100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31c659c640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009a50 tx=0x7f31b804f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731b4cad3e22b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31adffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b805f060 con 0x55eb927c3490 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f31b804fc90 con 0x55eb927c3490 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.546+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b80055f0 con 0x55eb927c3490 2024-09-21T01:54:34.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.547+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31b8040070 con 0x55eb927c3490 2024-09-21T01:54:34.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.547+0000 7f31d3393740 10 client.26492 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:34.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.547+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55eb92c1cb80 con 0x55eb927c3490 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b80318e0 con 0x55eb927c3490 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f31b803c5a0 con 0x55eb927c3490 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 10 client.26492 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 20 client.26492 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb92780080 0x55eb92c1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 4 client.26492 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f31b80055f0 con 0x55eb92780080 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.548+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb92780080 0x55eb92c1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.549+0000 7f31adffb640 10 client.26492 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.549+0000 7f31c5d9b640 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb92780080 0x55eb92c1ed60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362493745e09405 server_cookie=1ff3c20a78c1d8f6 in_seq=0 out_seq=0 2024-09-21T01:54:34.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.550+0000 7f31adffb640 1 -- 172.21.15.29:0/873884668 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f31b80055f0 con 0x55eb92780080 2024-09-21T01:54:34.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.550+0000 7f31adffb640 10 client.26492 handle_command_reply: tid=0 2024-09-21T01:54:34.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 client.26492 shutdown 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 msgr2=0x55eb92ba3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009a50 tx=0x7f31b804f660 comp rx=0 tx=0).stop 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 shutdown_connections 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55eb92780080 0x55eb92c1ed60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb927ec350 0x55eb92c15100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31adffb640 0 client.26492 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb927cb670 0x55eb92ba4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 --2- 172.21.15.29:0/873884668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb927c3490 0x55eb92ba3c20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 >> 172.21.15.29:0/873884668 conn(0x55eb927e0740 msgr2=0x55eb92b94640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 shutdown_connections 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.551+0000 7f31d3393740 1 -- 172.21.15.29:0/873884668 wait complete. 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31d3393740 20 client.26492 trim_cache size 0 max 16384 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "3657cfac-3db2-43e1-a67e-58a77fb4d032", 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:54:34.553 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 msgr2=0x7f31b00b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 0x7f31b00b8bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009d70 tx=0x7f31bc009880 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 msgr2=0x7f31c81a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009fd0 tx=0x7f31c000ef10 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 shutdown_connections 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f31b00b66e0 0x7f31b00b8bf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31c810d000 0x7f31c81a6140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c810c6b0 0x7f31c81a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 --2- 172.21.15.29:0/3002692688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c810a490 0x7f31c81a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.552+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 >> 172.21.15.29:0/3002692688 conn(0x7f31c8079420 msgr2=0x7f31c807b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.553+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 shutdown_connections 2024-09-21T01:54:34.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:34.553+0000 7f31c759e640 1 -- 172.21.15.29:0/3002692688 wait complete. 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi029 to 177.0M 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: Unable to set osd_memory_target on smithi029 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:54:34.567 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:54:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi029 to 177.0M 2024-09-21T01:54:34.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: Unable to set osd_memory_target on smithi029 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T01:54:34.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:54:34.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:54:34.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi029 to 177.0M 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: Unable to set osd_memory_target on smithi029 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T01:54:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:54:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:54:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:54:35.405 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:35 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:54:35.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-09-21T01:54:35.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:54:35.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:35 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-09-21T01:54:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:54:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:35 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:54:36.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 -- 172.21.15.29:0/2183069749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be01083e0 msgr2=0x7f0be01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 --2- 172.21.15.29:0/2183069749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be01083e0 0x7f0be01087f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009900 tx=0x7f0bd402a410 comp rx=0 tx=0).stop 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 -- 172.21.15.29:0/2183069749 shutdown_connections 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 --2- 172.21.15.29:0/2183069749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be0108d30 0x7f0be010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 --2- 172.21.15.29:0/2183069749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be01083e0 0x7f0be01087f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 --2- 172.21.15.29:0/2183069749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 0x7f0be0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 -- 172.21.15.29:0/2183069749 >> 172.21.15.29:0/2183069749 conn(0x7f0be0079750 msgr2=0x7f0be0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 -- 172.21.15.29:0/2183069749 shutdown_connections 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.501+0000 7f0be634c640 1 -- 172.21.15.29:0/2183069749 wait complete. 2024-09-21T01:54:36.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.502+0000 7f0be634c640 1 Processor -- start 2024-09-21T01:54:36.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.503+0000 7f0be634c640 1 -- start start 2024-09-21T01:54:36.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.504+0000 7f0be634c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 0x7f0be01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.504+0000 7f0be634c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.504+0000 7f0be634c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 0x7f0be019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.504+0000 7f0be634c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111930 con 0x7f0be0108d30 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0be634c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01117b0 con 0x7f0be01083e0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0be634c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111ab0 con 0x7f0be0102aa0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 0x7f0be01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0be48e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 0x7f0be019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0be48e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 0x7f0be019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38538/0 (socket says 172.21.15.29:38538) 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58396/0 (socket says 172.21.15.29:58396) 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0be48e9640 1 -- 172.21.15.29:0/1622889411 learned_addr learned my addr 172.21.15.29:0/1622889411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/1622889411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 msgr2=0x7f0be01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 0x7f0be01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 msgr2=0x7f0be019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 0x7f0be019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.505+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/1622889411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be019e210 con 0x7f0be01083e0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009230 tx=0x7f0bd4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0bdd7fa640 1 -- 172.21.15.29:0/1622889411 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd403f070 con 0x7f0be01083e0 2024-09-21T01:54:36.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be019e4a0 con 0x7f0be01083e0 2024-09-21T01:54:36.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a4c60 con 0x7f0be01083e0 2024-09-21T01:54:36.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0bdd7fa640 1 -- 172.21.15.29:0/1622889411 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd4031b50 con 0x7f0be01083e0 2024-09-21T01:54:36.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.506+0000 7f0bdd7fa640 1 -- 172.21.15.29:0/1622889411 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4035030 con 0x7f0be01083e0 2024-09-21T01:54:36.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.509+0000 7f0bec8c0740 1 Processor -- start 2024-09-21T01:54:36.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 -- start start 2024-09-21T01:54:36.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b041b0 0x562440ed4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ed50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562440e4ae50 con 0x562440b041b0 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562440e3a280 con 0x562440b0c390 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.510+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562440e3e090 con 0x562440b2d070 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b041b0 0x562440ed4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52438/0 (socket says 172.21.15.29:52438) 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 -- 172.21.15.29:0/4248340062 learned_addr learned my addr 172.21.15.29:0/4248340062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0bdd7fa640 1 -- 172.21.15.29:0/1622889411 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0bd4042070 con 0x7f0be01083e0 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ed50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 -- 172.21.15.29:0/4248340062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 msgr2=0x562440ed50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ed50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 -- 172.21.15.29:0/4248340062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b041b0 msgr2=0x562440ed4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b041b0 0x562440ed4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 -- 172.21.15.29:0/4248340062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562440b98650 con 0x562440b2d070 2024-09-21T01:54:36.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ed50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:54:36.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.511+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8010810 tx=0x7f0bd8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46e0dbc98dac7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.512+0000 7f0bc67fc640 1 -- 172.21.15.29:0/4248340062 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800b9a0 con 0x562440b2d070 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.512+0000 7f0bc67fc640 1 -- 172.21.15.29:0/4248340062 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd801a400 con 0x562440b2d070 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.512+0000 7f0bc67fc640 1 -- 172.21.15.29:0/4248340062 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80195c0 con 0x562440b2d070 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.512+0000 7f0bdd7fa640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 0x7f0bc80b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.513+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4248340062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 msgr2=0x562440ed4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.513+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8010810 tx=0x7f0bd8009500 comp rx=0 tx=0).stop 2024-09-21T01:54:36.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.514+0000 7f0bdffff640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 0x7f0bc80b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4248340062 shutdown_connections 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ed50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b041b0 0x562440ed4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4248340062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b2d070 0x562440ed4650 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4248340062 >> 172.21.15.29:0/4248340062 conn(0x562440b21460 msgr2=0x562440ed2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bdd7fa640 1 -- 172.21.15.29:0/1622889411 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bd41076f0 con 0x7f0be01083e0 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4248340062 shutdown_connections 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4248340062 wait complete. 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.515+0000 7f0bdffff640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 0x7f0bc80b8b10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x562440ed32d0 tx=0x7f0bd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.516+0000 7f0bec8c0740 1 -- start start 2024-09-21T01:54:36.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.516+0000 7f0bec8c0740 1 Processor -- start 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440edfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440ee0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562440c9b700 con 0x562440b2d070 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562440dd5110 con 0x562440b0c390 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562440ee2e30 con 0x562440b041b0 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bec8c0740 10 client.0 _ll_register_callbacks cb 0x562440dee990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58410/0 (socket says 172.21.15.29:58410) 2024-09-21T01:54:36.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/16100987 learned_addr learned my addr 172.21.15.29:0/16100987 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdffff640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440edfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0be48e9640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440ee0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/16100987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 msgr2=0x562440edfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440edfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/16100987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 msgr2=0x562440ee0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440ee0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.517+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/16100987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562440e02c40 con 0x562440b0c390 2024-09-21T01:54:36.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.518+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005ab0 tx=0x7f0bd40c9840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b682f883ca6cef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.518+0000 7f0bbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:54:36.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.518+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4049030 con 0x562440b0c390 2024-09-21T01:54:36.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.518+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bd40d6dc0 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd801a400 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0bd40cd050 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bec8c0740 10 client.26501 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562440eeea80 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd40cd3f0 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.519+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f0bd403bb40 con 0x562440b0c390 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 10 client.26501 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 20 client.26501 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440aeb9f0 0x562440ef1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 4 client.26501 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bd801a400 con 0x562440aeb9f0 2024-09-21T01:54:36.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.520+0000 7f0bdffff640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440aeb9f0 0x562440ef1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.521+0000 7f0bbf7fe640 10 client.26501 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:36.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.521+0000 7f0bdffff640 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440aeb9f0 0x562440ef1fc0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b457e9cd21445b server_cookie=da9f4ec4375ec30b in_seq=0 out_seq=0 2024-09-21T01:54:36.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.521+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/16100987 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0bd801a400 con 0x562440aeb9f0 2024-09-21T01:54:36.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.521+0000 7f0bbf7fe640 10 client.26501 handle_command_reply: tid=0 2024-09-21T01:54:36.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 client.26501 shutdown 2024-09-21T01:54:36.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 msgr2=0x562440ee01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005ab0 tx=0x7f0bd40c9840 comp rx=0 tx=0).stop 2024-09-21T01:54:36.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 shutdown_connections 2024-09-21T01:54:36.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440aeb9f0 0x562440ef1fc0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440ee0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b0c390 0x562440ee01e0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005ab0 tx=0x7f0bd40c9840 comp rx=0 tx=0).stop 2024-09-21T01:54:36.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/16100987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440edfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 >> 172.21.15.29:0/16100987 conn(0x562440b21460 msgr2=0x562440ed77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.522+0000 7f0bbf7fe640 0 client.26501 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:36.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.524+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 shutdown_connections 2024-09-21T01:54:36.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.524+0000 7f0bec8c0740 1 -- 172.21.15.29:0/16100987 wait complete. 2024-09-21T01:54:36.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.525+0000 7f0bec8c0740 20 client.26501 trim_cache size 0 max 16384 2024-09-21T01:54:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.554+0000 7f0bec8c0740 1 Processor -- start 2024-09-21T01:54:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.554+0000 7f0bec8c0740 1 -- start start 2024-09-21T01:54:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.554+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440ee1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.554+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 0x562440ed1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 0x562440ed20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00e3f0 con 0x562440ac0da0 2024-09-21T01:54:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00ff60 con 0x562440b2d070 2024-09-21T01:54:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bec8c0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00eac0 con 0x562440b041b0 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440ee1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440ee1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52464/0 (socket says 172.21.15.29:52464) 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdffff640 1 -- 172.21.15.29:0/2010390686 learned_addr learned my addr 172.21.15.29:0/2010390686 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 0x562440ed1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0be48e9640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 0x562440ed20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/2010390686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 msgr2=0x562440ee1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440ee1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/2010390686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 msgr2=0x562440ed20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 0x562440ed20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0bdf7fe640 1 -- 172.21.15.29:0/2010390686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562440ed74b0 con 0x562440b2d070 2024-09-21T01:54:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.555+0000 7f0be48e9640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 0x562440ed20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:54:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.556+0000 7f0bdf7fe640 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 0x562440ed1b70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402a930 tx=0x7f0bd4031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2dbc2e528455071 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.557+0000 7f0bc5ffb640 1 -- 172.21.15.29:0/2010390686 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4049070 con 0x562440b2d070 2024-09-21T01:54:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.557+0000 7f0bc5ffb640 1 -- 172.21.15.29:0/2010390686 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd40cd3f0 con 0x562440b2d070 2024-09-21T01:54:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.557+0000 7f0bc5ffb640 1 -- 172.21.15.29:0/2010390686 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40d7250 con 0x562440b2d070 2024-09-21T01:54:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.558+0000 7f0bec8c0740 1 -- 172.21.15.29:0/2010390686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 msgr2=0x562440ed1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.558+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 0x562440ed1b70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402a930 tx=0x7f0bd4031400 comp rx=0 tx=0).stop 2024-09-21T01:54:36.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.568+0000 7f0bec8c0740 1 -- 172.21.15.29:0/2010390686 shutdown_connections 2024-09-21T01:54:36.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.568+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440ac0da0 0x562440ed20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.568+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b2d070 0x562440ed1b70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.568+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/2010390686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440b041b0 0x562440ee1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.568+0000 7f0bec8c0740 1 -- 172.21.15.29:0/2010390686 >> 172.21.15.29:0/2010390686 conn(0x562440b21460 msgr2=0x562440edeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.576+0000 7f0bec8c0740 1 -- 172.21.15.29:0/2010390686 shutdown_connections 2024-09-21T01:54:36.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.576+0000 7f0bec8c0740 1 -- 172.21.15.29:0/2010390686 wait complete. 2024-09-21T01:54:36.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.576+0000 7f0bec8c0740 1 -- start start 2024-09-21T01:54:36.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.577+0000 7f0bec8c0740 1 Processor -- start 2024-09-21T01:54:36.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 0x562440ee43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bec8c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 0x562440ee48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 0x562440ee43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bdffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 0x562440ee43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52476/0 (socket says 172.21.15.29:52476) 2024-09-21T01:54:36.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 0x562440ee48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.578+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 0x562440ee48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58434/0 (socket says 172.21.15.29:58434) 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bdffff640 1 -- 172.21.15.29:0/4097113583 learned_addr learned my addr 172.21.15.29:0/4097113583 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440f641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00f890 con 0x562440b2d070 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00f1c0 con 0x562440b041b0 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00c3d0 con 0x562440ac0da0 2024-09-21T01:54:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.579+0000 7f0bec8c0740 10 client.0 _ll_register_callbacks cb 0x7f0bd4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.580+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440f641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.581+0000 7f0be48e9640 1 -- 172.21.15.29:0/4097113583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 msgr2=0x562440ee43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.581+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 0x562440ee43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.581+0000 7f0be48e9640 1 -- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 msgr2=0x562440ee48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.581+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 0x562440ee48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.581+0000 7f0be48e9640 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562440ee5180 con 0x562440b2d070 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.582+0000 7f0be48e9640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440f641d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000f80 tx=0x7f0bd8010600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726fc2d545034fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:36.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.582+0000 7f0bbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:54:36.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.585+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8029070 con 0x562440b2d070 2024-09-21T01:54:36.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.585+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bd800b9a0 con 0x562440b2d070 2024-09-21T01:54:36.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.586+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd40cd3f0 con 0x562440b2d070 2024-09-21T01:54:36.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.586+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0bd800bc00 con 0x562440b2d070 2024-09-21T01:54:36.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.587+0000 7f0bec8c0740 10 client.16752 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:36.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.587+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562440f65710 con 0x562440b2d070 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.587+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd40cd3f0 con 0x562440b2d070 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f0bd80385a0 con 0x562440b2d070 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 10 client.16752 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 20 client.16752 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440f65a60 0x562440f67e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 4 client.16752 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:54:36.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.588+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562440e4cda0 con 0x562440f65a60 2024-09-21T01:54:36.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.591+0000 7f0bdffff640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440f65a60 0x562440f67e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:36.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.592+0000 7f0bbf7fe640 10 client.16752 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:36.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.592+0000 7f0bdffff640 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440f65a60 0x562440f67e80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1789f7a869423d server_cookie=754319153b3d013e in_seq=0 out_seq=0 2024-09-21T01:54:36.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.592+0000 7f0bbf7fe640 1 -- 172.21.15.29:0/4097113583 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x562440e4cda0 con 0x562440f65a60 2024-09-21T01:54:36.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.592+0000 7f0bbf7fe640 10 client.16752 handle_command_reply: tid=0 2024-09-21T01:54:36.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.592+0000 7f0bec8c0740 1 client.16752 shutdown 2024-09-21T01:54:36.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 msgr2=0x562440f641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440f641d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000f80 tx=0x7f0bd8010600 comp rx=0 tx=0).stop 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 shutdown_connections 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562440f65a60 0x562440f67e80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562440b2d070 0x562440f641d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bbf7fe640 0 client.16752 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562440b041b0 0x562440ee48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.593+0000 7f0bec8c0740 1 --2- 172.21.15.29:0/4097113583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562440ac0da0 0x562440ee43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.594+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 >> 172.21.15.29:0/4097113583 conn(0x562440b21460 msgr2=0x562440ed5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.594+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 shutdown_connections 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.594+0000 7f0bec8c0740 1 -- 172.21.15.29:0/4097113583 wait complete. 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.594+0000 7f0bec8c0740 20 client.16752 trim_cache size 0 max 16384 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:54:36.595 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "3657cfac-3db2-43e1-a67e-58a77fb4d032", 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.597 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.598 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:54:36.599 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:54:36.600 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:54:36.601 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:54:36.601 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:54:36.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 msgr2=0x7f0bc80b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 0x7f0bc80b8b10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x562440ed32d0 tx=0x7f0bd0005f90 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 msgr2=0x7f0be0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009230 tx=0x7f0bd4003b90 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 shutdown_connections 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bc80b6600 0x7f0bc80b8b10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0be0108d30 0x7f0be019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0be01083e0 0x7f0be0198840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 --2- 172.21.15.29:0/1622889411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0102aa0 0x7f0be01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.595+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 >> 172.21.15.29:0/1622889411 conn(0x7f0be0079750 msgr2=0x7f0be01043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.596+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 shutdown_connections 2024-09-21T01:54:36.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:54:36.596+0000 7f0be634c640 1 -- 172.21.15.29:0/1622889411 wait complete. 2024-09-21T01:54:37.495 INFO:tasks.cephfs.filesystem:scrub status for tag:3657cfac-3db2-43e1-a67e-58a77fb4d032 - {'path': '/', 'tag': '3657cfac-3db2-43e1-a67e-58a77fb4d032', 'options': 'recursive,force'} 2024-09-21T01:54:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:37 smithi066 ceph-mon[35001]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 811 op/s 2024-09-21T01:54:37.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:37 smithi029 ceph-mon[31228]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 811 op/s 2024-09-21T01:54:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 360.0 s, 3140.1 tps, lat 32.069 ms stddev 37.672 2024-09-21T01:54:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:37 smithi190 ceph-mon[34368]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 811 op/s 2024-09-21T01:54:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:39 smithi066 ceph-mon[35001]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-09-21T01:54:39.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:39 smithi029 ceph-mon[31228]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-09-21T01:54:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:39 smithi190 ceph-mon[34368]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-09-21T01:54:40.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:54:40.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:54:40.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:54:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:54:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:54:40.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:54:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:54:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:54:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:54:41.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:41 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:54:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:41 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:54:41.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:41 smithi029 ceph-mon[31228]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.6 MiB/s wr, 840 op/s 2024-09-21T01:54:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:41 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:54:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:41 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:54:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:41 smithi190 ceph-mon[34368]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.6 MiB/s wr, 840 op/s 2024-09-21T01:54:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:41 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:54:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:41 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:54:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:41 smithi066 ceph-mon[35001]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.6 MiB/s wr, 840 op/s 2024-09-21T01:54:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 365.0 s, 3102.4 tps, lat 32.221 ms stddev 38.358 2024-09-21T01:54:43.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:43 smithi029 ceph-mon[31228]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T01:54:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:43 smithi190 ceph-mon[34368]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T01:54:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:43 smithi066 ceph-mon[35001]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T01:54:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:45 smithi029 ceph-mon[31228]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.5 MiB/s wr, 775 op/s 2024-09-21T01:54:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:45 smithi190 ceph-mon[34368]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.5 MiB/s wr, 775 op/s 2024-09-21T01:54:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:45 smithi066 ceph-mon[35001]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.5 MiB/s wr, 775 op/s 2024-09-21T01:54:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.681+0000 7fb2440ec640 1 -- 172.21.15.29:0/2890340474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 msgr2=0x7fb23c1076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.681+0000 7fb2440ec640 1 --2- 172.21.15.29:0/2890340474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c1076d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb23401aca0 tx=0x7fb23403b890 comp rx=0 tx=0).stop 2024-09-21T01:54:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.682+0000 7fb2440ec640 1 -- 172.21.15.29:0/2890340474 shutdown_connections 2024-09-21T01:54:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.682+0000 7fb2440ec640 1 --2- 172.21.15.29:0/2890340474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c1076d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.682+0000 7fb2440ec640 1 --2- 172.21.15.29:0/2890340474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.682+0000 7fb2440ec640 1 --2- 172.21.15.29:0/2890340474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 0x7fb23c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.682+0000 7fb2440ec640 1 -- 172.21.15.29:0/2890340474 >> 172.21.15.29:0/2890340474 conn(0x7fb23c0fdee0 msgr2=0x7fb23c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.689+0000 7fb2440ec640 1 -- 172.21.15.29:0/2890340474 shutdown_connections 2024-09-21T01:54:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.689+0000 7fb2440ec640 1 -- 172.21.15.29:0/2890340474 wait complete. 2024-09-21T01:54:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.690+0000 7fb2440ec640 1 Processor -- start 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 -- start start 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 0x7fb23c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10b190 con 0x7fb23c06b710 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10b010 con 0x7fb23c104610 2024-09-21T01:54:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.699+0000 7fb2440ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10b310 con 0x7fb23c1071d0 2024-09-21T01:54:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.701+0000 7fb241687640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.701+0000 7fb241687640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45934/0 (socket says 172.21.15.29:45934) 2024-09-21T01:54:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.701+0000 7fb241687640 1 -- 172.21.15.29:0/3814006618 learned_addr learned my addr 172.21.15.29:0/3814006618 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241e88640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 0x7fb23c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb242689640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241687640 1 -- 172.21.15.29:0/3814006618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 msgr2=0x7fb23c198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241687640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241687640 1 -- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 msgr2=0x7fb23c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241687640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 0x7fb23c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.702+0000 7fb241687640 1 -- 172.21.15.29:0/3814006618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c199810 con 0x7fb23c104610 2024-09-21T01:54:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.703+0000 7fb241687640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b450 tx=0x7fb22c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.705+0000 7fb232ffd640 1 -- 172.21.15.29:0/3814006618 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c015020 con 0x7fb23c104610 2024-09-21T01:54:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.705+0000 7fb232ffd640 1 -- 172.21.15.29:0/3814006618 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c011d40 con 0x7fb23c104610 2024-09-21T01:54:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.705+0000 7fb232ffd640 1 -- 172.21.15.29:0/3814006618 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00f340 con 0x7fb23c104610 2024-09-21T01:54:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.706+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c199aa0 con 0x7fb23c104610 2024-09-21T01:54:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.706+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a0090 con 0x7fb23c104610 2024-09-21T01:54:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.710+0000 7fb24b05c740 1 Processor -- start 2024-09-21T01:54:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 -- start start 2024-09-21T01:54:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617a47126f0 con 0x5617a43d74d0 2024-09-21T01:54:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617a4701b20 con 0x5617a43cf2f0 2024-09-21T01:54:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.711+0000 7fb24b05c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617a4705930 con 0x5617a43f81b0 2024-09-21T01:54:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.714+0000 7fb241e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.714+0000 7fb232ffd640 1 -- 172.21.15.29:0/3814006618 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb22c018020 con 0x7fb23c104610 2024-09-21T01:54:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.714+0000 7fb241e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45946/0 (socket says 172.21.15.29:45946) 2024-09-21T01:54:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.714+0000 7fb241e88640 1 -- 172.21.15.29:0/1726312990 learned_addr learned my addr 172.21.15.29:0/1726312990 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.715+0000 7fb232ffd640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 0x7fb2280b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.715+0000 7fb232ffd640 1 -- 172.21.15.29:0/3814006618 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb22c0e6d70 con 0x7fb23c104610 2024-09-21T01:54:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.717+0000 7fb242689640 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.718+0000 7fb241e88640 1 -- 172.21.15.29:0/1726312990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 msgr2=0x5617a479fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.718+0000 7fb241e88640 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.718+0000 7fb241e88640 1 -- 172.21.15.29:0/1726312990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 msgr2=0x5617a47a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.718+0000 7fb241e88640 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.718+0000 7fb241e88640 1 -- 172.21.15.29:0/1726312990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a46c7530 con 0x5617a43cf2f0 2024-09-21T01:54:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.721+0000 7fb241e88640 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c80 tx=0x7fb2380117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f423ea4773481bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.721+0000 7fb242689640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 0x7fb2280b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.722+0000 7fb227fff640 1 -- 172.21.15.29:0/1726312990 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238012450 con 0x5617a43cf2f0 2024-09-21T01:54:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.722+0000 7fb227fff640 1 -- 172.21.15.29:0/1726312990 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238012aa0 con 0x5617a43cf2f0 2024-09-21T01:54:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.722+0000 7fb227fff640 1 -- 172.21.15.29:0/1726312990 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23800fb80 con 0x5617a43cf2f0 2024-09-21T01:54:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 -- 172.21.15.29:0/1726312990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 msgr2=0x5617a47a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c80 tx=0x7fb2380117b0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb242689640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 0x7fb2280b8b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb234005e20 tx=0x7fb23401a5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 -- 172.21.15.29:0/1726312990 shutdown_connections 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a04c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.723+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1726312990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.724+0000 7fb24b05c740 1 -- 172.21.15.29:0/1726312990 >> 172.21.15.29:0/1726312990 conn(0x5617a43ec300 msgr2=0x5617a479b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.724+0000 7fb24b05c740 1 -- 172.21.15.29:0/1726312990 shutdown_connections 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.724+0000 7fb24b05c740 1 -- 172.21.15.29:0/1726312990 wait complete. 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.725+0000 7fb24b05c740 1 -- start start 2024-09-21T01:54:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.725+0000 7fb24b05c740 1 Processor -- start 2024-09-21T01:54:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43cf2f0 0x5617a47a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617a4639ab0 con 0x5617a43cf2f0 2024-09-21T01:54:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617a46a7730 con 0x5617a43d74d0 2024-09-21T01:54:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617a47a1b30 con 0x5617a43f81b0 2024-09-21T01:54:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.726+0000 7fb24b05c740 10 client.0 _ll_register_callbacks cb 0x5617a46fb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.727+0000 7fb242689640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.727+0000 7fb242689640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40658/0 (socket says 172.21.15.29:40658) 2024-09-21T01:54:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.727+0000 7fb242689640 1 -- 172.21.15.29:0/1568956064 learned_addr learned my addr 172.21.15.29:0/1568956064 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.728+0000 7fb241687640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.728+0000 7fb242689640 1 -- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 msgr2=0x5617a47a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.728+0000 7fb242689640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.728+0000 7fb242689640 1 -- 172.21.15.29:0/1568956064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43cf2f0 msgr2=0x5617a47a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.728+0000 7fb242689640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43cf2f0 0x5617a47a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.729+0000 7fb242689640 1 -- 172.21.15.29:0/1568956064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a4463790 con 0x5617a43f81b0 2024-09-21T01:54:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.729+0000 7fb242689640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb234046000 tx=0x7fb2340428a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686e511da3673e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.729+0000 7fb2257fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23405d070 con 0x5617a43f81b0 2024-09-21T01:54:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb238012aa0 con 0x5617a43f81b0 2024-09-21T01:54:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb2340044c0 con 0x5617a43f81b0 2024-09-21T01:54:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb234069050 con 0x5617a43f81b0 2024-09-21T01:54:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb24b05c740 10 client.26530 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.730+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617a44dc620 con 0x5617a43f81b0 2024-09-21T01:54:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.732+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238012aa0 con 0x5617a43f81b0 2024-09-21T01:54:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.732+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fb23401d040 con 0x5617a43f81b0 2024-09-21T01:54:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 10 client.26530 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 10 client.26530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 20 client.26530 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a43b6d80 0x5617a47b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 4 client.26530 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:54:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.733+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a4714640 con 0x5617a43b6d80 2024-09-21T01:54:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.734+0000 7fb241e88640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a43b6d80 0x5617a47b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.734+0000 7fb2257fa640 10 client.26530 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.734+0000 7fb241e88640 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a43b6d80 0x5617a47b7390 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29619c05e90ed97b server_cookie=837efc6cfde310ef in_seq=0 out_seq=0 2024-09-21T01:54:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.735+0000 7fb2257fa640 1 -- 172.21.15.29:0/1568956064 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617a4714640 con 0x5617a43b6d80 2024-09-21T01:54:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.735+0000 7fb2257fa640 10 client.26530 handle_command_reply: tid=0 2024-09-21T01:54:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 client.26530 shutdown 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 msgr2=0x5617a47aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb234046000 tx=0x7fb2340428a0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 shutdown_connections 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a43b6d80 0x5617a47b7390 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a47aa8e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 --2- 172.21.15.29:0/1568956064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43cf2f0 0x5617a47a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 >> 172.21.15.29:0/1568956064 conn(0x5617a43ec300 msgr2=0x5617a4799ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb2257fa640 0 client.26530 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 shutdown_connections 2024-09-21T01:54:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.736+0000 7fb24b05c740 1 -- 172.21.15.29:0/1568956064 wait complete. 2024-09-21T01:54:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.737+0000 7fb24b05c740 20 client.26530 trim_cache size 0 max 16384 2024-09-21T01:54:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.763+0000 7fb24b05c740 1 Processor -- start 2024-09-21T01:54:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.764+0000 7fb24b05c740 1 -- start start 2024-09-21T01:54:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb241e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb241e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45964/0 (socket says 172.21.15.29:45964) 2024-09-21T01:54:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1a8ea0 con 0x5617a43d74d0 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1aaa10 con 0x5617a43cf2f0 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1a9570 con 0x5617a43f81b0 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.765+0000 7fb241e88640 1 -- 172.21.15.29:0/3682803988 learned_addr learned my addr 172.21.15.29:0/3682803988 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241687640 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb242689640 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 -- 172.21.15.29:0/3682803988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 msgr2=0x5617a479bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 -- 172.21.15.29:0/3682803988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 msgr2=0x5617a47b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 -- 172.21.15.29:0/3682803988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a479c260 con 0x5617a43cf2f0 2024-09-21T01:54:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.766+0000 7fb241e88640 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c50 tx=0x7fb238009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5afaea28b25e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.767+0000 7fb2277fe640 1 -- 172.21.15.29:0/3682803988 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238033070 con 0x5617a43cf2f0 2024-09-21T01:54:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.767+0000 7fb2277fe640 1 -- 172.21.15.29:0/3682803988 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238005570 con 0x5617a43cf2f0 2024-09-21T01:54:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.767+0000 7fb2277fe640 1 -- 172.21.15.29:0/3682803988 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23802a070 con 0x5617a43cf2f0 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 msgr2=0x5617a47a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c50 tx=0x7fb238009530 comp rx=0 tx=0).stop 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 shutdown_connections 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43f81b0 0x5617a479bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43d74d0 0x5617a47b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3682803988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43cf2f0 0x5617a47a4a30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 >> 172.21.15.29:0/3682803988 conn(0x5617a43ec300 msgr2=0x5617a479ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 shutdown_connections 2024-09-21T01:54:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.768+0000 7fb24b05c740 1 -- 172.21.15.29:0/3682803988 wait complete. 2024-09-21T01:54:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.769+0000 7fb24b05c740 1 -- start start 2024-09-21T01:54:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.769+0000 7fb24b05c740 1 Processor -- start 2024-09-21T01:54:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43f81b0 0x5617a47af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1aa340 con 0x5617a43f81b0 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1a9c70 con 0x5617a43d74d0 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1a6e80 con 0x5617a43cf2f0 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb24b05c740 10 client.0 _ll_register_callbacks cb 0x7fb23401aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb241e88640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb241e88640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40682/0 (socket says 172.21.15.29:40682) 2024-09-21T01:54:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb241e88640 1 -- 172.21.15.29:0/3960406803 learned_addr learned my addr 172.21.15.29:0/3960406803 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:54:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb242689640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43f81b0 0x5617a47af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.771+0000 7fb241687640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 -- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 msgr2=0x5617a47aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 -- 172.21.15.29:0/3960406803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43f81b0 msgr2=0x5617a47af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:54:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43f81b0 0x5617a47af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 -- 172.21.15.29:0/3960406803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a47afeb0 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb241e88640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb23801c040 tx=0x7fb238009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f4eb0aa5126b28d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.772+0000 7fb2257fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23803c060 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617a47100f0 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb238005250 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb23802a070 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb24b05c740 10 client.26536 fetch_fsmap learned FSMap version 40 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617a47b11d0 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.773+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5617a47100f0 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fb23802c040 con 0x5617a43cf2f0 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 10 client.26536 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 20 client.26536 populate_metadata read hostname 'smithi029' 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a438c1c0 0x5617a4823da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:54:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 4 client.26536 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:54:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617a470d520 con 0x5617a438c1c0 2024-09-21T01:54:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.774+0000 7fb241687640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a438c1c0 0x5617a4823da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:54:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.776+0000 7fb241687640 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a438c1c0 0x5617a4823da0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8dbf73d3f1d5f36 server_cookie=d006ffc9b6ed729e in_seq=0 out_seq=0 2024-09-21T01:54:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.776+0000 7fb2257fa640 10 client.26536 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.788+0000 7fb2257fa640 1 -- 172.21.15.29:0/3960406803 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617a470d520 con 0x5617a438c1c0 2024-09-21T01:54:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.788+0000 7fb2257fa640 10 client.26536 handle_command_reply: tid=0 2024-09-21T01:54:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.788+0000 7fb24b05c740 1 client.26536 shutdown 2024-09-21T01:54:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.788+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 msgr2=0x5617a47ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb23801c040 tx=0x7fb238009720 comp rx=0 tx=0).stop 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 shutdown_connections 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617a438c1c0 0x5617a4823da0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a43f81b0 0x5617a47af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617a43d74d0 0x5617a47aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 --2- 172.21.15.29:0/3960406803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617a43cf2f0 0x5617a47ae520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb2257fa640 0 client.26536 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 >> 172.21.15.29:0/3960406803 conn(0x5617a43ec300 msgr2=0x5617a479ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 shutdown_connections 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 1 -- 172.21.15.29:0/3960406803 wait complete. 2024-09-21T01:54:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.789+0000 7fb24b05c740 20 client.26536 trim_cache size 0 max 16384 2024-09-21T01:54:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:54:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:54:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:54:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 msgr2=0x7fb2280b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 0x7fb2280b8b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb234005e20 tx=0x7fb23401a5b0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 msgr2=0x7fb23c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b450 tx=0x7fb22c00b910 comp rx=0 tx=0).stop 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 shutdown_connections 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2280b6650 0x7fb2280b8b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c1071d0 0x7fb23c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb23c104610 0x7fb23c198440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 --2- 172.21.15.29:0/3814006618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb23c06b710 0x7fb23c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:54:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 >> 172.21.15.29:0/3814006618 conn(0x7fb23c0fdee0 msgr2=0x7fb23c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:54:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 shutdown_connections 2024-09-21T01:54:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:54:47.790+0000 7fb2440ec640 1 -- 172.21.15.29:0/3814006618 wait complete. 2024-09-21T01:54:47.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:47 smithi029 ceph-mon[31228]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 911 op/s 2024-09-21T01:54:47.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 370.0 s, 2369.2 tps, lat 42.242 ms stddev 46.500 2024-09-21T01:54:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:47 smithi190 ceph-mon[34368]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 911 op/s 2024-09-21T01:54:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:47 smithi066 ceph-mon[35001]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 911 op/s 2024-09-21T01:54:49.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:49 smithi029 ceph-mon[31228]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 644 op/s 2024-09-21T01:54:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:49 smithi190 ceph-mon[34368]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 644 op/s 2024-09-21T01:54:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:49 smithi066 ceph-mon[35001]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 644 op/s 2024-09-21T01:54:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:51 smithi029 ceph-mon[31228]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 927 op/s 2024-09-21T01:54:51.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:51 smithi066 ceph-mon[35001]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 927 op/s 2024-09-21T01:54:51.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:51 smithi190 ceph-mon[34368]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 927 op/s 2024-09-21T01:54:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 375.0 s, 3194.6 tps, lat 31.272 ms stddev 38.584 2024-09-21T01:54:53.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:53 smithi029 ceph-mon[31228]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 755 op/s 2024-09-21T01:54:53.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:53 smithi190 ceph-mon[34368]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 755 op/s 2024-09-21T01:54:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:53 smithi066 ceph-mon[35001]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 755 op/s 2024-09-21T01:54:55.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:55 smithi029 ceph-mon[31228]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 709 op/s 2024-09-21T01:54:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:55 smithi190 ceph-mon[34368]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 709 op/s 2024-09-21T01:54:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:55 smithi066 ceph-mon[35001]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 709 op/s 2024-09-21T01:54:57.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:57 smithi029 ceph-mon[31228]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-21T01:54:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 380.0 s, 1989.0 tps, lat 49.861 ms stddev 57.386 2024-09-21T01:54:57.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:57 smithi190 ceph-mon[34368]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-21T01:54:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:57 smithi066 ceph-mon[35001]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-21T01:54:59.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:54:59 smithi029 ceph-mon[31228]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-21T01:54:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:54:59 smithi190 ceph-mon[34368]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-21T01:55:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:54:59 smithi066 ceph-mon[35001]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-21T01:55:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 -- 172.21.15.29:0/3037229567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57101025a0 msgr2=0x7f5710102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 --2- 172.21.15.29:0/3037229567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57101025a0 0x7f5710102a10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 -- 172.21.15.29:0/3037229567 shutdown_connections 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 --2- 172.21.15.29:0/3037229567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710103fc0 0x7f571010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 --2- 172.21.15.29:0/3037229567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103620 0x7f5710103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 --2- 172.21.15.29:0/3037229567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57101025a0 0x7f5710102a10 secure :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 -- 172.21.15.29:0/3037229567 >> 172.21.15.29:0/3037229567 conn(0x7f57100fdee0 msgr2=0x7f5710100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 -- 172.21.15.29:0/3037229567 shutdown_connections 2024-09-21T01:55:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.933+0000 7f5716b2b640 1 -- 172.21.15.29:0/3037229567 wait complete. 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.934+0000 7f5716b2b640 1 Processor -- start 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.934+0000 7f5716b2b640 1 -- start start 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5710103620 0x7f5710106250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 0x7f57101053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e1f0 con 0x7f5710103620 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e070 con 0x7f5710103fc0 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f57150c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 0x7f57101053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50588/0 (socket says 172.21.15.29:50588) 2024-09-21T01:55:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f57150c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 0x7f57101053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59680/0 (socket says 172.21.15.29:59680) 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 -- 172.21.15.29:0/90421130 learned_addr learned my addr 172.21.15.29:0/90421130 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e370 con 0x7f5710104f90 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f57148c7640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5710103620 0x7f5710106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 -- 172.21.15.29:0/90421130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 msgr2=0x7f57101053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 0x7f57101053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 -- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5710103620 msgr2=0x7f5710106250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5710103620 0x7f5710106250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.935+0000 7f570ffff640 1 -- 172.21.15.29:0/90421130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101a41d0 con 0x7f5710103fc0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f570ffff640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00e9f0 tx=0x7f56fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f570dffb640 1 -- 172.21.15.29:0/90421130 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00cd80 con 0x7f5710103fc0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a44c0 con 0x7f5710103fc0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f570dffb640 1 -- 172.21.15.29:0/90421130 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc0108e0 con 0x7f5710103fc0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f570dffb640 1 -- 172.21.15.29:0/90421130 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc018980 con 0x7f5710103fc0 2024-09-21T01:55:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.936+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a4a60 con 0x7f5710103fc0 2024-09-21T01:55:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f570dffb640 1 -- 172.21.15.29:0/90421130 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f56fc018b20 con 0x7f5710103fc0 2024-09-21T01:55:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 Processor -- start 2024-09-21T01:55:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 -- start start 2024-09-21T01:55:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b6a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1adb6f0 con 0x55a6d17a04d0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1acab20 con 0x55a6d17982f0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f571da95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1ace930 con 0x55a6d17c11b0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f57150c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f57150c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59686/0 (socket says 172.21.15.29:59686) 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f57150c8640 1 -- 172.21.15.29:0/2040326010 learned_addr learned my addr 172.21.15.29:0/2040326010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.940+0000 7f570ffff640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b6a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f570dffb640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 0x7f57000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57148c7640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 -- 172.21.15.29:0/2040326010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 msgr2=0x55a6d1b694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 -- 172.21.15.29:0/2040326010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 msgr2=0x55a6d1b6a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b6a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 -- 172.21.15.29:0/2040326010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d1a90530 con 0x55a6d17c11b0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f570ffff640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b6a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f570dffb640 1 -- 172.21.15.29:0/90421130 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f56fc0e7660 con 0x7f5710103fc0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 0x7f57000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.941+0000 7f57150c8640 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5708011590 tx=0x7f5708011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d02684f3e144be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.942+0000 7f56faffd640 1 -- 172.21.15.29:0/2040326010 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708017050 con 0x55a6d17c11b0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.942+0000 7f56faffd640 1 -- 172.21.15.29:0/2040326010 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570801bc50 con 0x55a6d17c11b0 2024-09-21T01:55:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.942+0000 7f56faffd640 1 -- 172.21.15.29:0/2040326010 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570800e3b0 con 0x55a6d17c11b0 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.943+0000 7f57150c8640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 0x7f57000b8a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5708022000 tx=0x7f5708023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- 172.21.15.29:0/2040326010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 msgr2=0x55a6d1b68e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5708011590 tx=0x7f5708011a50 comp rx=0 tx=0).stop 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- 172.21.15.29:0/2040326010 shutdown_connections 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b6a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 --2- 172.21.15.29:0/2040326010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b68e50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- 172.21.15.29:0/2040326010 >> 172.21.15.29:0/2040326010 conn(0x55a6d17b5300 msgr2=0x55a6d1b648e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- 172.21.15.29:0/2040326010 shutdown_connections 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- 172.21.15.29:0/2040326010 wait complete. 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.944+0000 7f571da95740 1 -- start start 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 Processor -- start 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b72660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17c11b0 0x55a6d1b73890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1a02ab0 con 0x55a6d17c11b0 2024-09-21T01:55:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1a70730 con 0x55a6d17a04d0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a6d1b6ab30 con 0x55a6d17982f0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f571da95740 10 client.0 _ll_register_callbacks cb 0x55a6d1ac46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f570ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f57150c8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17c11b0 0x55a6d1b73890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f570ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50596/0 (socket says 172.21.15.29:50596) 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f570ffff640 1 -- 172.21.15.29:0/1937332235 learned_addr learned my addr 172.21.15.29:0/1937332235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.945+0000 7f57148c7640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b72660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 -- 172.21.15.29:0/1937332235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 msgr2=0x55a6d1b72660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b72660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 -- 172.21.15.29:0/1937332235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17c11b0 msgr2=0x55a6d1b73890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17c11b0 0x55a6d1b73890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 -- 172.21.15.29:0/1937332235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d182c790 con 0x55a6d17a04d0 2024-09-21T01:55:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f57148c7640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b72660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f570ffff640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0e5110 tx=0x7f56fc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63a46a77f03f19f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f56effff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0ad620 con 0x55a6d17a04d0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570801bc50 con 0x55a6d17a04d0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f56fc0e9b20 con 0x55a6d17a04d0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f56fc0acfc0 con 0x55a6d17a04d0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f571da95740 10 client.26534 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.946+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6d18a5620 con 0x55a6d17a04d0 2024-09-21T01:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570801bc50 con 0x55a6d17a04d0 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f56fc0e70d0 con 0x55a6d17a04d0 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 10 client.26534 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 10 client.26534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 20 client.26534 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d177fd80 0x55a6d1b7fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 4 client.26534 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6d1add640 con 0x55a6d177fd80 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.947+0000 7f57148c7640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d177fd80 0x55a6d1b7fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.948+0000 7f57148c7640 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d177fd80 0x55a6d1b7fdc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc1755f8efccf5f server_cookie=339ad6d126afc3a5 in_seq=0 out_seq=0 2024-09-21T01:55:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.948+0000 7f56effff640 10 client.26534 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f56effff640 1 -- 172.21.15.29:0/1937332235 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6d1add640 con 0x55a6d177fd80 2024-09-21T01:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f56effff640 10 client.26534 handle_command_reply: tid=0 2024-09-21T01:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 client.26534 shutdown 2024-09-21T01:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 msgr2=0x55a6d1b72d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0e5110 tx=0x7f56fc014000 comp rx=0 tx=0).stop 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 shutdown_connections 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d177fd80 0x55a6d1b7fdc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17c11b0 0x55a6d1b73890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17a04d0 0x55a6d1b72d20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 --2- 172.21.15.29:0/1937332235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b72660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f56effff640 0 client.26534 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.949+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 >> 172.21.15.29:0/1937332235 conn(0x55a6d17b5300 msgr2=0x55a6d1b62ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.950+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 shutdown_connections 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.950+0000 7f571da95740 1 -- 172.21.15.29:0/1937332235 wait complete. 2024-09-21T01:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.950+0000 7f571da95740 20 client.26534 trim_cache size 0 max 16384 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 Processor -- start 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 -- start start 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ad780 con 0x55a6d17a04d0 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57101af2f0 con 0x55a6d17982f0 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f571da95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ade50 con 0x55a6d17c11b0 2024-09-21T01:55:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f570ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f570ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47410/0 (socket says 172.21.15.29:47410) 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f57150c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f570ffff640 1 -- 172.21.15.29:0/2870673632 learned_addr learned my addr 172.21.15.29:0/2870673632 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.973+0000 7f57148c7640 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 -- 172.21.15.29:0/2870673632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 msgr2=0x55a6d1b710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 -- 172.21.15.29:0/2870673632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 msgr2=0x55a6d1b5f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b5f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 -- 172.21.15.29:0/2870673632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d1b73c00 con 0x55a6d17a04d0 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.974+0000 7f570ffff640 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0aaac0 tx=0x7f56fc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ae782efff227e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.975+0000 7f56fa7fc640 1 -- 172.21.15.29:0/2870673632 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b2030 con 0x55a6d17a04d0 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.975+0000 7f56fa7fc640 1 -- 172.21.15.29:0/2870673632 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc0ee070 con 0x55a6d17a04d0 2024-09-21T01:55:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.975+0000 7f56fa7fc640 1 -- 172.21.15.29:0/2870673632 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0ad1a0 con 0x55a6d17a04d0 2024-09-21T01:55:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- 172.21.15.29:0/2870673632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 msgr2=0x55a6d1b70bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0aaac0 tx=0x7f56fc018420 comp rx=0 tx=0).stop 2024-09-21T01:55:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- 172.21.15.29:0/2870673632 shutdown_connections 2024-09-21T01:55:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17c11b0 0x55a6d1b710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b70bb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 --2- 172.21.15.29:0/2870673632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17982f0 0x55a6d1b5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- 172.21.15.29:0/2870673632 >> 172.21.15.29:0/2870673632 conn(0x55a6d17b5300 msgr2=0x55a6d1b64220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- 172.21.15.29:0/2870673632 shutdown_connections 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- 172.21.15.29:0/2870673632 wait complete. 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 -- start start 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.976+0000 7f571da95740 1 Processor -- start 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 0x55a6d1b79850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57101aec20 con 0x55a6d17a04d0 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ae550 con 0x55a6d17c11b0 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ab760 con 0x55a6d17982f0 2024-09-21T01:55:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57148c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57148c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59722/0 (socket says 172.21.15.29:59722) 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57148c7640 1 -- 172.21.15.29:0/2770396537 learned_addr learned my addr 172.21.15.29:0/2770396537 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f570ffff640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f571da95740 10 client.0 _ll_register_callbacks cb 0x7f56fc0a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57150c8640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 0x55a6d1b79850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57148c7640 1 -- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 msgr2=0x55a6d1b79850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.977+0000 7f57148c7640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 0x55a6d1b79850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f57148c7640 1 -- 172.21.15.29:0/2770396537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 msgr2=0x55a6d1b79220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f57148c7640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b79220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f57148c7640 1 -- 172.21.15.29:0/2770396537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d1b79fe0 con 0x55a6d17982f0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f57150c8640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 0x55a6d1b79850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f57148c7640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5704005ab0 tx=0x7f5704009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941690f0dab39c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.978+0000 7f56effff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.979+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704035040 con 0x55a6d17982f0 2024-09-21T01:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.979+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5704004040 con 0x55a6d17982f0 2024-09-21T01:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.979+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56fc0ee070 con 0x55a6d17982f0 2024-09-21T01:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.980+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5704004240 con 0x55a6d17982f0 2024-09-21T01:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.980+0000 7f571da95740 10 client.26554 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.980+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6d1bea8b0 con 0x55a6d17982f0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.980+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc0ee070 con 0x55a6d17982f0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f570403b430 con 0x55a6d17982f0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 10 client.26554 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 10 client.26554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 20 client.26554 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d17551c0 0x55a6d1bed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 4 client.26554 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6d1ad90f0 con 0x55a6d17551c0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f570ffff640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d17551c0 0x55a6d1bed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f570ffff640 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d17551c0 0x55a6d1bed280 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9b0c60ba2a08c5 server_cookie=bdb15e289877029f in_seq=0 out_seq=0 2024-09-21T01:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.981+0000 7f56effff640 10 client.26554 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f56effff640 1 -- 172.21.15.29:0/2770396537 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6d1ad90f0 con 0x55a6d17551c0 2024-09-21T01:55:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f56effff640 10 client.26554 handle_command_reply: tid=0 2024-09-21T01:55:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 client.26554 shutdown 2024-09-21T01:55:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 msgr2=0x55a6d1b7a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5704005ab0 tx=0x7f5704009230 comp rx=0 tx=0).stop 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 shutdown_connections 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a6d17551c0 0x55a6d1bed280 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6d17c11b0 0x55a6d1b79850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f56effff640 0 client.26554 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6d17a04d0 0x55a6d1b79220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 --2- 172.21.15.29:0/2770396537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6d17982f0 0x55a6d1b7a560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.992+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 >> 172.21.15.29:0/2770396537 conn(0x55a6d17b5300 msgr2=0x55a6d1b64200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 shutdown_connections 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f571da95740 1 -- 172.21.15.29:0/2770396537 wait complete. 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f571da95740 20 client.26554 trim_cache size 0 max 16384 2024-09-21T01:55:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:55:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:55:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:55:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 msgr2=0x7f57000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 0x7f57000b8a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5708022000 tx=0x7f5708023000 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 msgr2=0x7f5710104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.993+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00e9f0 tx=0x7f56fc00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 shutdown_connections 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57000b6510 0x7f57000b8a20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5710104f90 0x7f57101053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5710103fc0 0x7f5710104a50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 --2- 172.21.15.29:0/90421130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5710103620 0x7f5710106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 >> 172.21.15.29:0/90421130 conn(0x7f57100fdee0 msgr2=0x7f57100ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 shutdown_connections 2024-09-21T01:55:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:00.994+0000 7f5716b2b640 1 -- 172.21.15.29:0/90421130 wait complete. 2024-09-21T01:55:01.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:01 smithi029 ceph-mon[31228]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 808 op/s 2024-09-21T01:55:01.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:01 smithi190 ceph-mon[34368]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 808 op/s 2024-09-21T01:55:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:01 smithi066 ceph-mon[35001]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 808 op/s 2024-09-21T01:55:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 385.0 s, 2259.8 tps, lat 44.285 ms stddev 48.270 2024-09-21T01:55:03.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:03 smithi029 ceph-mon[31228]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 631 op/s 2024-09-21T01:55:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:03 smithi190 ceph-mon[34368]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 631 op/s 2024-09-21T01:55:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:03 smithi066 ceph-mon[35001]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 631 op/s 2024-09-21T01:55:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:05 smithi029 ceph-mon[31228]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T01:55:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:05 smithi190 ceph-mon[34368]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T01:55:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:05 smithi066 ceph-mon[35001]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T01:55:07.496 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:55:07.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:07 smithi029 ceph-mon[31228]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 762 op/s 2024-09-21T01:55:07.923 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 390.0 s, 2015.2 tps, lat 50.122 ms stddev 55.217 2024-09-21T01:55:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:07 smithi190 ceph-mon[34368]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 762 op/s 2024-09-21T01:55:08.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:07 smithi066 ceph-mon[35001]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 762 op/s 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 -- 172.21.15.29:0/91585504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810b6e0 msgr2=0x7f010810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 --2- 172.21.15.29:0/91585504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810b6e0 0x7f010810baf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f802a400 comp rx=0 tx=0).stop 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 -- 172.21.15.29:0/91585504 shutdown_connections 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 --2- 172.21.15.29:0/91585504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f010810c030 0x7f0108112860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 --2- 172.21.15.29:0/91585504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810b6e0 0x7f010810baf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 --2- 172.21.15.29:0/91585504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0108105da0 0x7f01081061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.541+0000 7f010f2ee640 1 -- 172.21.15.29:0/91585504 >> 172.21.15.29:0/91585504 conn(0x7f01081017f0 msgr2=0x7f0108103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.542+0000 7f010f2ee640 1 -- 172.21.15.29:0/91585504 shutdown_connections 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.542+0000 7f010f2ee640 1 -- 172.21.15.29:0/91585504 wait complete. 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 Processor -- start 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 -- start start 2024-09-21T01:55:08.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010810b6e0 0x7f01081101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 0x7f0108110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0108115d60 con 0x7f0108105da0 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0108115be0 con 0x7f010810c030 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.543+0000 7f010f2ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0108115ee0 con 0x7f010810b6e0 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50306/0 (socket says 172.21.15.29:50306) 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 -- 172.21.15.29:0/4000936826 learned_addr learned my addr 172.21.15.29:0/4000936826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010c889640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010810b6e0 0x7f01081101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d88b640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 0x7f0108110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 -- 172.21.15.29:0/4000936826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010810b6e0 msgr2=0x7f01081101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010810b6e0 0x7f01081101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 -- 172.21.15.29:0/4000936826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 msgr2=0x7f0108110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 0x7f0108110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 -- 172.21.15.29:0/4000936826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01081a23b0 con 0x7f0108105da0 2024-09-21T01:55:08.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d88b640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 0x7f0108110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:08.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.544+0000 7f010d08a640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f010400b4a0 tx=0x7f010400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.545+0000 7f00fe7fc640 1 -- 172.21.15.29:0/4000936826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104014070 con 0x7f0108105da0 2024-09-21T01:55:08.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.545+0000 7f00fe7fc640 1 -- 172.21.15.29:0/4000936826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0104010ac0 con 0x7f0108105da0 2024-09-21T01:55:08.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.545+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01081a26a0 con 0x7f0108105da0 2024-09-21T01:55:08.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.546+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a2bd0 con 0x7f0108105da0 2024-09-21T01:55:08.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.546+0000 7f00fe7fc640 1 -- 172.21.15.29:0/4000936826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104017c30 con 0x7f0108105da0 2024-09-21T01:55:08.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.551+0000 7f0115862740 1 Processor -- start 2024-09-21T01:55:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 -- start start 2024-09-21T01:55:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f00fe7fc640 1 -- 172.21.15.29:0/4000936826 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0104017070 con 0x7f0108105da0 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035d31b0 0x55a9039a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a903919e50 con 0x55a9035fc070 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a903909280 con 0x55a9035db390 2024-09-21T01:55:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.552+0000 7f0115862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a90390d090 con 0x55a9035d31b0 2024-09-21T01:55:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45682/0 (socket says 172.21.15.29:45682) 2024-09-21T01:55:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 -- 172.21.15.29:0/157768849 learned_addr learned my addr 172.21.15.29:0/157768849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010d88b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 -- 172.21.15.29:0/157768849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035d31b0 msgr2=0x55a9039a37b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035d31b0 0x55a9039a37b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 -- 172.21.15.29:0/157768849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 msgr2=0x55a9039a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.553+0000 7f010c889640 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.554+0000 7f010c889640 1 -- 172.21.15.29:0/157768849 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a903667650 con 0x55a9035db390 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.554+0000 7f00fe7fc640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 0x7f00f40b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.554+0000 7f010c889640 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f8031440 tx=0x7f00f80042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b96317414b9a999 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.555+0000 7f00f37fe640 1 -- 172.21.15.29:0/157768849 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8035040 con 0x55a9035db390 2024-09-21T01:55:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.555+0000 7f00fe7fc640 1 -- 172.21.15.29:0/4000936826 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f01040e64c0 con 0x7f0108105da0 2024-09-21T01:55:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.555+0000 7f00f37fe640 1 -- 172.21.15.29:0/157768849 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f8004740 con 0x55a9035db390 2024-09-21T01:55:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.555+0000 7f00f37fe640 1 -- 172.21.15.29:0/157768849 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8003850 con 0x55a9035db390 2024-09-21T01:55:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.556+0000 7f010d88b640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 0x7f00f40b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 -- 172.21.15.29:0/157768849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 msgr2=0x55a9039a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f8031440 tx=0x7f00f80042e0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f010d88b640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 0x7f00f40b8c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0100005fd0 tx=0x7f0100009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 -- 172.21.15.29:0/157768849 shutdown_connections 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035db390 0x55a9039a3cf0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f8031440 tx=0x7f00f80042e0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035d31b0 0x55a9039a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 --2- 172.21.15.29:0/157768849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 -- 172.21.15.29:0/157768849 >> 172.21.15.29:0/157768849 conn(0x55a9035f0460 msgr2=0x55a9039a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.557+0000 7f0115862740 1 -- 172.21.15.29:0/157768849 shutdown_connections 2024-09-21T01:55:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.558+0000 7f0115862740 1 -- 172.21.15.29:0/157768849 wait complete. 2024-09-21T01:55:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.558+0000 7f0115862740 1 -- start start 2024-09-21T01:55:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.558+0000 7f0115862740 1 Processor -- start 2024-09-21T01:55:08.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035ba9f0 0x55a9039bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a90376a700 con 0x55a9035fc070 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9038a4110 con 0x55a9035d31b0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9039b19f0 con 0x55a9035ba9f0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f0115862740 10 client.0 _ll_register_callbacks cb 0x55a9038bd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f010d08a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f010d08a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45692/0 (socket says 172.21.15.29:45692) 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f010d08a640 1 -- 172.21.15.29:0/3011867369 learned_addr learned my addr 172.21.15.29:0/3011867369 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.559+0000 7f010d88b640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035ba9f0 0x55a9039bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010c889640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 -- 172.21.15.29:0/3011867369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035ba9f0 msgr2=0x55a9039bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035ba9f0 0x55a9039bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 -- 172.21.15.29:0/3011867369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 msgr2=0x55a9039aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 -- 172.21.15.29:0/3011867369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9038d1c40 con 0x55a9035d31b0 2024-09-21T01:55:08.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010c889640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f010d08a640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01040ac590 tx=0x7f01040a8560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f8c5216ec9ce89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.560+0000 7f00f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010401e070 con 0x55a9035d31b0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f8004740 con 0x55a9035d31b0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f01040ab860 con 0x55a9035d31b0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f01040abd70 con 0x55a9035d31b0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f0115862740 10 client.26552 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.561+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a9039ce7a0 con 0x55a9035d31b0 2024-09-21T01:55:08.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f8004740 con 0x55a9035d31b0 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f01040ab2d0 con 0x55a9035d31b0 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 10 client.26552 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 10 client.26552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 20 client.26552 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a90358fda0 0x55a9039d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 4 client.26552 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.562+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a90391bda0 con 0x55a90358fda0 2024-09-21T01:55:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.563+0000 7f010c889640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a90358fda0 0x55a9039d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.563+0000 7f010c889640 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a90358fda0 0x55a9039d1fb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438f84e0f87cd7f0 server_cookie=b81f4a7ebc2d4fc4 in_seq=0 out_seq=0 2024-09-21T01:55:08.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.563+0000 7f00f0ff9640 10 client.26552 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:08.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.564+0000 7f00f0ff9640 1 -- 172.21.15.29:0/3011867369 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a90391bda0 con 0x55a90358fda0 2024-09-21T01:55:08.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.564+0000 7f00f0ff9640 10 client.26552 handle_command_reply: tid=0 2024-09-21T01:55:08.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.564+0000 7f0115862740 1 client.26552 shutdown 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 msgr2=0x55a9039ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01040ac590 tx=0x7f01040a8560 comp rx=0 tx=0).stop 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 shutdown_connections 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a90358fda0 0x55a9039d1fb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035ba9f0 0x55a9039bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035fc070 0x55a9039aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 --2- 172.21.15.29:0/3011867369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035d31b0 0x55a9039ae9f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 >> 172.21.15.29:0/3011867369 conn(0x55a9035f0460 msgr2=0x55a9039a6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f00f0ff9640 0 client.26552 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 shutdown_connections 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.566+0000 7f0115862740 1 -- 172.21.15.29:0/3011867369 wait complete. 2024-09-21T01:55:08.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.567+0000 7f0115862740 20 client.26552 trim_cache size 0 max 16384 2024-09-21T01:55:08.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.596+0000 7f0115862740 1 Processor -- start 2024-09-21T01:55:08.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.596+0000 7f0115862740 1 -- start start 2024-09-21T01:55:08.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 0x55a9039bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9039a6370 0x55a9039a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01080745b0 con 0x55a9035ba9f0 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01081b7590 con 0x55a9039a6370 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f0115862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0108074c80 con 0x55a9035fc070 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010d08a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 0x55a9039bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010d88b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9039a6370 0x55a9039a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36984/0 (socket says 172.21.15.29:36984) 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010d08a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 0x55a9039bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50326/0 (socket says 172.21.15.29:50326) 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 -- 172.21.15.29:0/3735613837 learned_addr learned my addr 172.21.15.29:0/3735613837 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 -- 172.21.15.29:0/3735613837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9039a6370 msgr2=0x55a9039a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9039a6370 0x55a9039a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 -- 172.21.15.29:0/3735613837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 msgr2=0x55a9039bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 0x55a9039bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.597+0000 7f010c889640 1 -- 172.21.15.29:0/3735613837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9039acef0 con 0x55a9035fc070 2024-09-21T01:55:08.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.598+0000 7f010c889640 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f66229c173cdb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.598+0000 7f00f2ffd640 1 -- 172.21.15.29:0/3735613837 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8035040 con 0x55a9035fc070 2024-09-21T01:55:08.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.598+0000 7f00f2ffd640 1 -- 172.21.15.29:0/3735613837 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f8031e00 con 0x55a9035fc070 2024-09-21T01:55:08.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.598+0000 7f00f2ffd640 1 -- 172.21.15.29:0/3735613837 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004ac0 con 0x55a9035fc070 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 -- 172.21.15.29:0/3735613837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 msgr2=0x55a9039a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f8000bc0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 -- 172.21.15.29:0/3735613837 shutdown_connections 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9039a6370 0x55a9039a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039a1190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.600+0000 7f0115862740 1 --2- 172.21.15.29:0/3735613837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9035ba9f0 0x55a9039bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.601+0000 7f0115862740 1 -- 172.21.15.29:0/3735613837 >> 172.21.15.29:0/3735613837 conn(0x55a9035f0460 msgr2=0x55a9039a0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.601+0000 7f0115862740 1 -- 172.21.15.29:0/3735613837 shutdown_connections 2024-09-21T01:55:08.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.601+0000 7f0115862740 1 -- 172.21.15.29:0/3735613837 wait complete. 2024-09-21T01:55:08.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.602+0000 7f0115862740 1 -- start start 2024-09-21T01:55:08.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.602+0000 7f0115862740 1 Processor -- start 2024-09-21T01:55:08.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.603+0000 7f0115862740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.603+0000 7f0115862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.603+0000 7f0115862740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9039a6370 0x55a9039b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f0115862740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01081b6ec0 con 0x55a9039a6370 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010d08a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010d08a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:45710/0 (socket says 172.21.15.29:45710) 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010d08a640 1 -- 172.21.15.29:0/1220907814 learned_addr learned my addr 172.21.15.29:0/1220907814 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010d88b640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9039a6370 0x55a9039b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0108075380 con 0x55a9035ba9f0 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0108072590 con 0x55a9035fc070 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f0115862740 10 client.0 _ll_register_callbacks cb 0x7f01040a8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 -- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 msgr2=0x55a9039b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 -- 172.21.15.29:0/1220907814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9039a6370 msgr2=0x55a9039b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:08.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9039a6370 0x55a9039b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.604+0000 7f010c889640 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a903a3c9c0 con 0x55a9035fc070 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.605+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f8031e00 con 0x55a9035fc070 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.605+0000 7f010d08a640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.605+0000 7f010c889640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039b3ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55a90388fce0 tx=0x7f00f8031780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d7e6c4297f9d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.605+0000 7f00f0ff9640 10 client.26566 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.606+0000 7f00f0ff9640 1 -- 172.21.15.29:0/1220907814 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8035040 con 0x55a9035fc070 2024-09-21T01:55:08.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.606+0000 7f00f0ff9640 1 -- 172.21.15.29:0/1220907814 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f00f8031c00 con 0x55a9035fc070 2024-09-21T01:55:08.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.607+0000 7f00f0ff9640 1 -- 172.21.15.29:0/1220907814 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f00f8005240 con 0x55a9035fc070 2024-09-21T01:55:08.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.607+0000 7f0115862740 10 client.26566 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:08.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.607+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=40}) -- 0x55a903a3d990 con 0x55a9035fc070 2024-09-21T01:55:08.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f00f0ff9640 1 -- 172.21.15.29:0/1220907814 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f00f80053e0 con 0x55a9035fc070 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 10 client.26566 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 20 client.26566 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a903a3db60 0x55a903a3ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 4 client.26566 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.608+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a903917850 con 0x55a903a3db60 2024-09-21T01:55:08.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.609+0000 7f010d08a640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a903a3db60 0x55a903a3ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:08.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.611+0000 7f00f0ff9640 10 client.26566 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:08.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.611+0000 7f010d08a640 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a903a3db60 0x55a903a3ff80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8f4fb90d3c8aa6 server_cookie=8f529f0a35467572 in_seq=0 out_seq=0 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f00f0ff9640 1 -- 172.21.15.29:0/1220907814 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55a903917850 con 0x55a903a3db60 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f00f0ff9640 10 client.26566 handle_command_reply: tid=0 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 client.26566 shutdown 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 msgr2=0x55a9039b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039b3ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55a90388fce0 tx=0x7f00f8031780 comp rx=0 tx=0).stop 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 shutdown_connections 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a903a3db60 0x55a903a3ff80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9039a6370 0x55a9039b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9035fc070 0x55a9039b3ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 --2- 172.21.15.29:0/1220907814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9035ba9f0 0x55a9039b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 >> 172.21.15.29:0/1220907814 conn(0x55a9035f0460 msgr2=0x55a9039a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.612+0000 7f00f0ff9640 0 client.26566 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:08.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.613+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 shutdown_connections 2024-09-21T01:55:08.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.614+0000 7f0115862740 1 -- 172.21.15.29:0/1220907814 wait complete. 2024-09-21T01:55:08.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.614+0000 7f0115862740 20 client.26566 trim_cache size 0 max 16384 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.616 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.617 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.618 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:55:08.619 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:55:08.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:55:08.621 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:55:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.615+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 msgr2=0x7f00f40b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.615+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 0x7f00f40b8c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0100005fd0 tx=0x7f0100009580 comp rx=0 tx=0).stop 2024-09-21T01:55:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.615+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 msgr2=0x7f010810fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.615+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f010400b4a0 tx=0x7f010400b960 comp rx=0 tx=0).stop 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 shutdown_connections 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f00f40b6720 0x7f00f40b8c30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010810c030 0x7f0108110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010810b6e0 0x7f01081101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 --2- 172.21.15.29:0/4000936826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0108105da0 0x7f010810fca0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 >> 172.21.15.29:0/4000936826 conn(0x7f01081017f0 msgr2=0x7f0108103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.616+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 shutdown_connections 2024-09-21T01:55:08.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:08.617+0000 7f010f2ee640 1 -- 172.21.15.29:0/4000936826 wait complete. 2024-09-21T01:55:09.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:55:09.500 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:55:09.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:09 smithi029 ceph-mon[31228]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 MiB/s wr, 521 op/s 2024-09-21T01:55:09.959 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:09.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:09 smithi190 ceph-mon[34368]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 MiB/s wr, 521 op/s 2024-09-21T01:55:10.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:09 smithi066 ceph-mon[35001]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 MiB/s wr, 521 op/s 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/1442080008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac102aa0 msgr2=0x7ff9ac102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/1442080008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac102eb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009900 tx=0x7ff99c02a400 comp rx=0 tx=0).stop 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/1442080008 shutdown_connections 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/1442080008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac108d30 0x7ff9ac10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/1442080008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/1442080008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac102eb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/1442080008 >> 172.21.15.29:0/1442080008 conn(0x7ff9ac079750 msgr2=0x7ff9ac079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/1442080008 shutdown_connections 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.565+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/1442080008 wait complete. 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.566+0000 7ff9b1fc2640 1 Processor -- start 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.566+0000 7ff9b1fc2640 1 -- start start 2024-09-21T01:55:10.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1129f0 con 0x7ff9ac1083e0 2024-09-21T01:55:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac112870 con 0x7ff9ac102aa0 2024-09-21T01:55:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.567+0000 7ff9b1fc2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac112b70 con 0x7ff9ac108d30 2024-09-21T01:55:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37014/0 (socket says 172.21.15.29:37014) 2024-09-21T01:55:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 -- 172.21.15.29:0/3836734750 learned_addr learned my addr 172.21.15.29:0/3836734750 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 -- 172.21.15.29:0/3836734750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac102aa0 msgr2=0x7ff9ac1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 -- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 msgr2=0x7ff9ac1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.568+0000 7ff9abfff640 1 -- 172.21.15.29:0/3836734750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac1a6810 con 0x7ff9ac108d30 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9aaffd640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9abfff640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400eb10 tx=0x7ff9a400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4004b20 con 0x7ff9ac108d30 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac1a6b00 con 0x7ff9ac108d30 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1ad260 con 0x7ff9ac108d30 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a4004cc0 con 0x7ff9ac108d30 2024-09-21T01:55:10.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.569+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4010720 con 0x7ff9ac108d30 2024-09-21T01:55:10.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.574+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff9a4005020 con 0x7ff9ac108d30 2024-09-21T01:55:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.575+0000 7ff9a8ff9640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 0x7ff9940b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.575+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff9a40e76c0 con 0x7ff9ac108d30 2024-09-21T01:55:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.581+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980052c0 con 0x7ff9ac108d30 2024-09-21T01:55:10.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.587+0000 7ff9ab7fe640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 0x7ff9940b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:10.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.588+0000 7ff9ab7fe640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 0x7ff9940b8b20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b230 tx=0x7ff99c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:10.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.593+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9a40adf00 con 0x7ff9ac108d30 2024-09-21T01:55:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.981+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9980055b0 con 0x7ff9ac108d30 2024-09-21T01:55:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.982+0000 7ff9a8ff9640 1 -- 172.21.15.29:0/3836734750 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11971 (secure 0 0 0) 0x7ff9a40b22e0 con 0x7ff9ac108d30 2024-09-21T01:55:10.984 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:55:10.984 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":40,"btime":"2024-09-21T01:53:56:421645+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:53:56.253083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:55:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 40 2024-09-21T01:55:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 msgr2=0x7ff9940b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 0x7ff9940b8b20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b230 tx=0x7ff99c00b040 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 msgr2=0x7ff9ac1a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400eb10 tx=0x7ff9a400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 shutdown_connections 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff9940b6610 0x7ff9940b8b20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9ac108d30 0x7ff9ac1a60e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ac1083e0 0x7ff9ac1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 --2- 172.21.15.29:0/3836734750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9ac102aa0 0x7ff9ac1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 >> 172.21.15.29:0/3836734750 conn(0x7ff9ac079750 msgr2=0x7ff9ac0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 shutdown_connections 2024-09-21T01:55:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:10.986+0000 7ff9b1fc2640 1 -- 172.21.15.29:0/3836734750 wait complete. 2024-09-21T01:55:11.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:55:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:11 smithi190 ceph-mon[34368]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 731 op/s 2024-09-21T01:55:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:11 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3836734750' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:55:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:11 smithi066 ceph-mon[35001]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 731 op/s 2024-09-21T01:55:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:11 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3836734750' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:55:12.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:11 smithi029 ceph-mon[31228]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 731 op/s 2024-09-21T01:55:12.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:11 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3836734750' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:55:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 395.0 s, 2543.9 tps, lat 36.710 ms stddev 43.605 2024-09-21T01:55:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:13 smithi190 ceph-mon[34368]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T01:55:14.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:13 smithi066 ceph-mon[35001]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T01:55:14.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:13 smithi029 ceph-mon[31228]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T01:55:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.146+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1887837922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103fd0 msgr2=0x7f2d3410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.146+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1887837922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103fd0 0x7f2d3410a970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009be0 tx=0x7f2d2c02a6d0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1887837922 shutdown_connections 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1887837922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103fd0 0x7f2d3410a970 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1887837922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103630 0x7f2d34103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1887837922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 0x7f2d34102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1887837922 >> 172.21.15.29:0/1887837922 conn(0x7f2d340fdf30 msgr2=0x7f2d341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.147+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1887837922 shutdown_connections 2024-09-21T01:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.148+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1887837922 wait complete. 2024-09-21T01:55:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d3c49d640 1 Processor -- start 2024-09-21T01:55:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d3c49d640 1 -- start start 2024-09-21T01:55:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d3c49d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 0x7f2d34198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d3c49d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39984/0 (socket says 172.21.15.29:39984) 2024-09-21T01:55:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.149+0000 7f2d3c49d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103fd0 0x7f2d3419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d39a38640 1 -- 172.21.15.29:0/1607947255 learned_addr learned my addr 172.21.15.29:0/1607947255 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3410e1f0 con 0x7f2d34103fd0 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3410e070 con 0x7f2d34103630 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3410e370 con 0x7f2d341025b0 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103fd0 0x7f2d3419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.150+0000 7f2d3a239640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 0x7f2d34198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 -- 172.21.15.29:0/1607947255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 msgr2=0x7f2d34198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 0x7f2d34198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 -- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103fd0 msgr2=0x7f2d3419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103fd0 0x7f2d3419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3419a850 con 0x7f2d34103630 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.151+0000 7f2d39a38640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d2400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.152+0000 7f2d2b7fe640 1 -- 172.21.15.29:0/1607947255 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240159e0 con 0x7f2d34103630 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.152+0000 7f2d2b7fe640 1 -- 172.21.15.29:0/1607947255 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2400cd60 con 0x7f2d34103630 2024-09-21T01:55:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.152+0000 7f2d2b7fe640 1 -- 172.21.15.29:0/1607947255 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240108f0 con 0x7f2d34103630 2024-09-21T01:55:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.152+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3419ab40 con 0x7f2d34103630 2024-09-21T01:55:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.152+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3406b780 con 0x7f2d34103630 2024-09-21T01:55:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.156+0000 7f2d43408740 1 Processor -- start 2024-09-21T01:55:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 -- start start 2024-09-21T01:55:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d2b7fe640 1 -- 172.21.15.29:0/1607947255 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2d24010a90 con 0x7f2d34103630 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 0x559dd9b39660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b39cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97684d0 0x559dd9b3a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559dd9aa36f0 con 0x559dd97602f0 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559dd9a92b20 con 0x559dd97891b0 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d43408740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dd9a96930 con 0x559dd97684d0 2024-09-21T01:55:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b39cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b39cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54034/0 (socket says 172.21.15.29:54034) 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d3a239640 1 -- 172.21.15.29:0/1772513740 learned_addr learned my addr 172.21.15.29:0/1772513740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.157+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 0x559dd9b39660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.158+0000 7f2d39a38640 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97684d0 0x559dd9b3a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/1772513740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97684d0 msgr2=0x559dd9b3a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97684d0 0x559dd9b3a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/1772513740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 msgr2=0x559dd9b39cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b39cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/1772513740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd9a58530 con 0x559dd97891b0 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d2b7fe640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 0x7f2d200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 0x559dd9b39660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009b30 tx=0x7f2d2c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c577e5ad6f46c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.159+0000 7f2d2b7fe640 1 -- 172.21.15.29:0/1607947255 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d240e7380 con 0x7f2d34103630 2024-09-21T01:55:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.160+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 0x7f2d200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.160+0000 7f2d28bfe640 1 -- 172.21.15.29:0/1772513740 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c041030 con 0x559dd97891b0 2024-09-21T01:55:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.160+0000 7f2d28bfe640 1 -- 172.21.15.29:0/1772513740 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2c030530 con 0x559dd97891b0 2024-09-21T01:55:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.160+0000 7f2d28bfe640 1 -- 172.21.15.29:0/1772513740 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c036040 con 0x559dd97891b0 2024-09-21T01:55:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.161+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 0x7f2d200b8aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c003fd0 tx=0x7f2d2c0044f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 -- 172.21.15.29:0/1772513740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 msgr2=0x559dd9b39660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 0x559dd9b39660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009b30 tx=0x7f2d2c005fd0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 -- 172.21.15.29:0/1772513740 shutdown_connections 2024-09-21T01:55:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97684d0 0x559dd9b3a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b39cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 --2- 172.21.15.29:0/1772513740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97891b0 0x559dd9b39660 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 -- 172.21.15.29:0/1772513740 >> 172.21.15.29:0/1772513740 conn(0x559dd977d300 msgr2=0x559dd9b350f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.162+0000 7f2d43408740 1 -- 172.21.15.29:0/1772513740 shutdown_connections 2024-09-21T01:55:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.163+0000 7f2d43408740 1 -- 172.21.15.29:0/1772513740 wait complete. 2024-09-21T01:55:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 -- start start 2024-09-21T01:55:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 Processor -- start 2024-09-21T01:55:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b44010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559dd99caab0 con 0x559dd97684d0 2024-09-21T01:55:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559dd9a38730 con 0x559dd97602f0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dd9b3b2e0 con 0x559dd97891b0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.164+0000 7f2d43408740 10 client.0 _ll_register_callbacks cb 0x559dd9a8c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39994/0 (socket says 172.21.15.29:39994) 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54036/0 (socket says 172.21.15.29:54036) 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 -- 172.21.15.29:0/2354957878 learned_addr learned my addr 172.21.15.29:0/2354957878 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b44010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 -- 172.21.15.29:0/2354957878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 msgr2=0x559dd9b44010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b44010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 -- 172.21.15.29:0/2354957878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 msgr2=0x559dd9b434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 -- 172.21.15.29:0/2354957878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd97f4790 con 0x559dd97602f0 2024-09-21T01:55:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d39a38640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b44010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d3a239640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009cf0 tx=0x7f2d300039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2feba3f16f3de99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.165+0000 7f2d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.166+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30012860 con 0x559dd97602f0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.166+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c030530 con 0x559dd97602f0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.166+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d30012a00 con 0x559dd97602f0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.168+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d30012c00 con 0x559dd97602f0 2024-09-21T01:55:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.168+0000 7f2d43408740 10 client.26576 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.168+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559dd986d620 con 0x559dd97602f0 2024-09-21T01:55:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.168+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2c030530 con 0x559dd97602f0 2024-09-21T01:55:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.168+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2d3001a020 con 0x559dd97602f0 2024-09-21T01:55:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 10 client.26576 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 20 client.26576 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd9747d80 0x559dd9b50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 4 client.26576 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd9aa5640 con 0x559dd9747d80 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d39a38640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd9747d80 0x559dd9b50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.169+0000 7f2d1dffb640 10 client.26576 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.170+0000 7f2d39a38640 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd9747d80 0x559dd9b50630 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbb698ad3306e72 server_cookie=2ed805649534afe6 in_seq=0 out_seq=0 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.171+0000 7f2d1dffb640 1 -- 172.21.15.29:0/2354957878 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dd9aa5640 con 0x559dd9747d80 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.171+0000 7f2d1dffb640 10 client.26576 handle_command_reply: tid=0 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.171+0000 7f2d43408740 1 client.26576 shutdown 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.171+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 msgr2=0x559dd9b42de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.171+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009cf0 tx=0x7f2d300039c0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 shutdown_connections 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd9747d80 0x559dd9b50630 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b44010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 --2- 172.21.15.29:0/2354957878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b42de0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 >> 172.21.15.29:0/2354957878 conn(0x559dd977d300 msgr2=0x559dd9b2aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d1dffb640 0 client.26576 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 shutdown_connections 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 1 -- 172.21.15.29:0/2354957878 wait complete. 2024-09-21T01:55:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.172+0000 7f2d43408740 20 client.26576 trim_cache size 0 max 16384 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 Processor -- start 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 -- start start 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b27700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341a8c70 con 0x559dd97684d0 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341aa7e0 con 0x559dd97602f0 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d43408740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341a9340 con 0x559dd97891b0 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d39a38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b34a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54048/0 (socket says 172.21.15.29:54048) 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d39a38640 1 -- 172.21.15.29:0/1777776987 learned_addr learned my addr 172.21.15.29:0/1777776987 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d3a239640 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b27700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.198+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 -- 172.21.15.29:0/1777776987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 msgr2=0x559dd9b41fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b41fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 -- 172.21.15.29:0/1777776987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 msgr2=0x559dd9b34a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b34a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 -- 172.21.15.29:0/1777776987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd9b44380 con 0x559dd97602f0 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.199+0000 7f2d3a239640 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b27700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009cc0 tx=0x7f2d300040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=526f9686ee49f32e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.200+0000 7f2d1ffff640 1 -- 172.21.15.29:0/1777776987 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30033070 con 0x559dd97602f0 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.200+0000 7f2d1ffff640 1 -- 172.21.15.29:0/1777776987 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d30012860 con 0x559dd97602f0 2024-09-21T01:55:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.200+0000 7f2d1ffff640 1 -- 172.21.15.29:0/1777776987 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3001c680 con 0x559dd97602f0 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 -- 172.21.15.29:0/1777776987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 msgr2=0x559dd9b27700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b27700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009cc0 tx=0x7f2d300040f0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 -- 172.21.15.29:0/1777776987 shutdown_connections 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b41fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97684d0 0x559dd9b34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 --2- 172.21.15.29:0/1777776987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97602f0 0x559dd9b27700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 -- 172.21.15.29:0/1777776987 >> 172.21.15.29:0/1777776987 conn(0x559dd977d300 msgr2=0x559dd9b505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 -- 172.21.15.29:0/1777776987 shutdown_connections 2024-09-21T01:55:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.201+0000 7f2d43408740 1 -- 172.21.15.29:0/1777776987 wait complete. 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.202+0000 7f2d43408740 1 -- start start 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.202+0000 7f2d43408740 1 Processor -- start 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97684d0 0x559dd9b47f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b48a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341aa110 con 0x559dd97602f0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341a9a40 con 0x559dd97684d0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341a6c50 con 0x559dd97891b0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d43408740 10 client.0 _ll_register_callbacks cb 0x7f2d30004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d3a239640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b47880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54050/0 (socket says 172.21.15.29:54050) 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d3aa3a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b48a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d3a239640 1 -- 172.21.15.29:0/3262926311 learned_addr learned my addr 172.21.15.29:0/3262926311 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.203+0000 7f2d39a38640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97684d0 0x559dd9b47f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97684d0 msgr2=0x559dd9b47f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97684d0 0x559dd9b47f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/3262926311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 msgr2=0x559dd9b47880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b47880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 -- 172.21.15.29:0/3262926311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd9b491a0 con 0x559dd97891b0 2024-09-21T01:55:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d3aa3a640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b48a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c02acc0 tx=0x7f2d2c04d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3bf88457835e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c061060 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d2c030530 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.204+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30012860 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c036040 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 10 client.26590 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559dd9b4a470 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d30012860 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2d2c003ae0 con 0x559dd97891b0 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 10 client.26590 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 10 client.26590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 20 client.26590 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd971d1c0 0x559dd9b4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 4 client.26590 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:55:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.205+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dd9aa10f0 con 0x559dd971d1c0 2024-09-21T01:55:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.206+0000 7f2d3a239640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd971d1c0 0x559dd9b4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.207+0000 7f2d3a239640 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd971d1c0 0x559dd9b4aa00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a11817b1e541a4 server_cookie=e7b081284b208d5a in_seq=0 out_seq=0 2024-09-21T01:55:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.207+0000 7f2d1dffb640 10 client.26590 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d1dffb640 1 -- 172.21.15.29:0/3262926311 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dd9aa10f0 con 0x559dd971d1c0 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d1dffb640 10 client.26590 handle_command_reply: tid=0 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 client.26590 shutdown 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 msgr2=0x559dd9b48a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b48a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c02acc0 tx=0x7f2d2c04d000 comp rx=0 tx=0).stop 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 shutdown_connections 2024-09-21T01:55:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559dd971d1c0 0x559dd9b4aa00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dd97891b0 0x559dd9b48a40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559dd97684d0 0x559dd9b47f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 --2- 172.21.15.29:0/3262926311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559dd97602f0 0x559dd9b47880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d1dffb640 0 client.26590 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 >> 172.21.15.29:0/3262926311 conn(0x559dd977d300 msgr2=0x559dd9b505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 shutdown_connections 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.215+0000 7f2d43408740 1 -- 172.21.15.29:0/3262926311 wait complete. 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d43408740 20 client.26590 trim_cache size 0 max 16384 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:55:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 msgr2=0x7f2d200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 0x7f2d200b8aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c003fd0 tx=0x7f2d2c0044f0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 msgr2=0x7f2d34199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d2400edd0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 shutdown_connections 2024-09-21T01:55:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d200b6590 0x7f2d200b8aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d34103fd0 0x7f2d3419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d34103630 0x7f2d34199480 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 --2- 172.21.15.29:0/1607947255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d341025b0 0x7f2d34198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 >> 172.21.15.29:0/1607947255 conn(0x7f2d340fdf30 msgr2=0x7f2d34106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 shutdown_connections 2024-09-21T01:55:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:14.216+0000 7f2d3c49d640 1 -- 172.21.15.29:0/1607947255 wait complete. 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.689+0000 7f306710a640 1 -- 172.21.15.29:0/814605477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 msgr2=0x7f3060118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.689+0000 7f306710a640 1 --2- 172.21.15.29:0/814605477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 0x7f3060118b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3058009ca0 tx=0x7f305802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 -- 172.21.15.29:0/814605477 shutdown_connections 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 --2- 172.21.15.29:0/814605477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 0x7f3060118b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 --2- 172.21.15.29:0/814605477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 --2- 172.21.15.29:0/814605477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306010a590 0x7f306010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 -- 172.21.15.29:0/814605477 >> 172.21.15.29:0/814605477 conn(0x7f3060070ec0 msgr2=0x7f30600712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 -- 172.21.15.29:0/814605477 shutdown_connections 2024-09-21T01:55:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.690+0000 7f306710a640 1 -- 172.21.15.29:0/814605477 wait complete. 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.691+0000 7f306710a640 1 Processor -- start 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.691+0000 7f306710a640 1 -- start start 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306010a590 0x7f3060113410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 0x7f3060113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f306010d930 con 0x7f3060116640 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f306010d7b0 con 0x7f306010a590 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f306710a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f306010dab0 con 0x7f306010aee0 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f305ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f305ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36000/0 (socket says 172.21.15.29:36000) 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.692+0000 7f305ffff640 1 -- 172.21.15.29:0/332470002 learned_addr learned my addr 172.21.15.29:0/332470002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.693+0000 7f305ffff640 1 -- 172.21.15.29:0/332470002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306010a590 msgr2=0x7f3060113410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.693+0000 7f305ffff640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306010a590 0x7f3060113410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.693+0000 7f305ffff640 1 -- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 msgr2=0x7f3060113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.693+0000 7f305ffff640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 0x7f3060113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.693+0000 7f305ffff640 1 -- 172.21.15.29:0/332470002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601b5150 con 0x7f306010aee0 2024-09-21T01:55:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f305ffff640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9a0 tx=0x7f305400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f305dffb640 1 -- 172.21.15.29:0/332470002 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305400cd80 con 0x7f306010aee0 2024-09-21T01:55:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601b5440 con 0x7f306010aee0 2024-09-21T01:55:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f305dffb640 1 -- 172.21.15.29:0/332470002 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30540035f0 con 0x7f306010aee0 2024-09-21T01:55:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f305dffb640 1 -- 172.21.15.29:0/332470002 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054010690 con 0x7f306010aee0 2024-09-21T01:55:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.694+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601b59d0 con 0x7f306010aee0 2024-09-21T01:55:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.698+0000 7f305dffb640 1 -- 172.21.15.29:0/332470002 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3054010830 con 0x7f306010aee0 2024-09-21T01:55:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 Processor -- start 2024-09-21T01:55:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 -- start start 2024-09-21T01:55:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157239070 0x5561575e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556157556ea0 con 0x556157239070 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561575462d0 con 0x5561572101b0 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.699+0000 7f306d67e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55615754a0e0 con 0x556157218390 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.700+0000 7f305ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.700+0000 7f305ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36008/0 (socket says 172.21.15.29:36008) 2024-09-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.700+0000 7f305ffff640 1 -- 172.21.15.29:0/359414780 learned_addr learned my addr 172.21.15.29:0/359414780 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.700+0000 7f305dffb640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 0x7f30480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.700+0000 7f3064ea6640 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 -- 172.21.15.29:0/359414780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 msgr2=0x5561575e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 -- 172.21.15.29:0/359414780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157239070 msgr2=0x5561575e02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157239070 0x5561575e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 -- 172.21.15.29:0/359414780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561572a4650 con 0x556157218390 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305ffff640 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3054083c80 tx=0x7f30540727e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfda4f392ba8251b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f30656a7640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 0x7f30480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f3046ffd640 1 -- 172.21.15.29:0/359414780 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540ad5d0 con 0x556157218390 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.701+0000 7f305dffb640 1 -- 172.21.15.29:0/332470002 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30540e7660 con 0x7f306010aee0 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.702+0000 7f3046ffd640 1 -- 172.21.15.29:0/359414780 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30540ad310 con 0x556157218390 2024-09-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.702+0000 7f3046ffd640 1 -- 172.21.15.29:0/359414780 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540a13e0 con 0x556157218390 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.702+0000 7f30656a7640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 0x7f30480b8a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3058009620 tx=0x7f3058005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.702+0000 7f306d67e740 1 -- 172.21.15.29:0/359414780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 msgr2=0x5561575e0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.702+0000 7f306d67e740 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3054083c80 tx=0x7f30540727e0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 -- 172.21.15.29:0/359414780 shutdown_connections 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e0d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 --2- 172.21.15.29:0/359414780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157239070 0x5561575e02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 -- 172.21.15.29:0/359414780 >> 172.21.15.29:0/359414780 conn(0x55615722d460 msgr2=0x5561575de1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 -- 172.21.15.29:0/359414780 shutdown_connections 2024-09-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 -- 172.21.15.29:0/359414780 wait complete. 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 -- start start 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.703+0000 7f306d67e740 1 Processor -- start 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556157218390 0x5561575ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157239070 0x5561575ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5561573a7700 con 0x5561572101b0 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561574e1160 con 0x556157218390 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561575eea40 con 0x556157239070 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f3064ea6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f305ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556157218390 0x5561575ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f3064ea6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54108/0 (socket says 172.21.15.29:54108) 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f3064ea6640 1 -- 172.21.15.29:0/3502110830 learned_addr learned my addr 172.21.15.29:0/3502110830 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.704+0000 7f306d67e740 10 client.0 _ll_register_callbacks cb 0x5561574fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 -- 172.21.15.29:0/3502110830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157239070 msgr2=0x5561575ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157239070 0x5561575ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 -- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556157218390 msgr2=0x5561575ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556157218390 0x5561575ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 -- 172.21.15.29:0/3502110830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55615750ec90 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f3064ea6640 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3050011480 tx=0x7f3050011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924cde4c1ffb8794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.705+0000 7f303ffff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050012980 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3050012b20 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30540ad310 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3050012d20 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f306d67e740 10 client.16830 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.706+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5561575fa670 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30540ad310 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f305000f440 con 0x5561572101b0 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 10 client.16830 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 10 client.16830 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 20 client.16830 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5561571f79f0 0x5561575fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 4 client.16830 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556157558df0 con 0x5561571f79f0 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.707+0000 7f305ffff640 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5561571f79f0 0x5561575fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.708+0000 7f305ffff640 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5561571f79f0 0x5561575fdb90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=897b4e84d6d4d85d server_cookie=9766a4f04a1525dd in_seq=0 out_seq=0 2024-09-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.708+0000 7f303ffff640 10 client.16830 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.709+0000 7f303ffff640 1 -- 172.21.15.29:0/3502110830 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556157558df0 con 0x5561571f79f0 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.709+0000 7f303ffff640 10 client.16830 handle_command_reply: tid=0 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.709+0000 7f306d67e740 1 client.16830 shutdown 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 msgr2=0x5561575eb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3050011480 tx=0x7f3050011940 comp rx=0 tx=0).stop 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 shutdown_connections 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x5561571f79f0 0x5561575fdb90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f303ffff640 0 client.16830 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157239070 0x5561575ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556157218390 0x5561575ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 --2- 172.21.15.29:0/3502110830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575eb8e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 >> 172.21.15.29:0/3502110830 conn(0x55615722d460 msgr2=0x5561575e12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 shutdown_connections 2024-09-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.710+0000 7f306d67e740 1 -- 172.21.15.29:0/3502110830 wait complete. 2024-09-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.711+0000 7f306d67e740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 Processor -- start 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 -- start start 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 0x5561575e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30601be2e0 con 0x5561572101b0 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30601bfe50 con 0x5561571f79f0 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.733+0000 7f306d67e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30601be9b0 con 0x556157218390 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f305ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f3064ea6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 0x5561575e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f305ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54110/0 (socket says 172.21.15.29:54110) 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f305ffff640 1 -- 172.21.15.29:0/1617499062 learned_addr learned my addr 172.21.15.29:0/1617499062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 -- 172.21.15.29:0/1617499062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 msgr2=0x5561575e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 0x5561575e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 -- 172.21.15.29:0/1617499062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 msgr2=0x5561575e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f30656a7640 1 -- 172.21.15.29:0/1617499062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561575ecba0 con 0x556157218390 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f305ffff640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.734+0000 7f3064ea6640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 0x5561575e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:14.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.735+0000 7f30656a7640 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e9010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3058005cc0 tx=0x7f3058031ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab687ba0037b668 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.736+0000 7f30467fc640 1 -- 172.21.15.29:0/1617499062 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305804b070 con 0x556157218390 2024-09-21T01:55:14.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.736+0000 7f30467fc640 1 -- 172.21.15.29:0/1617499062 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3058036040 con 0x556157218390 2024-09-21T01:55:14.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.736+0000 7f30467fc640 1 -- 172.21.15.29:0/1617499062 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058030c90 con 0x556157218390 2024-09-21T01:55:14.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.737+0000 7f306d67e740 1 -- 172.21.15.29:0/1617499062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 msgr2=0x5561575e9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.737+0000 7f306d67e740 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e9010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3058005cc0 tx=0x7f3058031ec0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.737+0000 7f306d67e740 1 -- 172.21.15.29:0/1617499062 shutdown_connections 2024-09-21T01:55:14.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.737+0000 7f306d67e740 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556157218390 0x5561575e9010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561572101b0 0x5561575e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 --2- 172.21.15.29:0/1617499062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561571f79f0 0x5561575e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 -- 172.21.15.29:0/1617499062 >> 172.21.15.29:0/1617499062 conn(0x55615722d460 msgr2=0x5561575e12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 -- 172.21.15.29:0/1617499062 shutdown_connections 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 -- 172.21.15.29:0/1617499062 wait complete. 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 -- start start 2024-09-21T01:55:14.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.738+0000 7f306d67e740 1 Processor -- start 2024-09-21T01:55:14.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561571f79f0 0x5561575efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157218390 0x5561575f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30601bf780 con 0x556157218390 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30601bf0b0 con 0x5561572101b0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30601bc2c0 con 0x5561571f79f0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f306d67e740 10 client.0 _ll_register_callbacks cb 0x7f30500095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f305ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f305ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40074/0 (socket says 172.21.15.29:40074) 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f305ffff640 1 -- 172.21.15.29:0/2918520685 learned_addr learned my addr 172.21.15.29:0/2918520685 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f30656a7640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157218390 0x5561575f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.740+0000 7f3064ea6640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561571f79f0 0x5561575efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 -- 172.21.15.29:0/2918520685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561571f79f0 msgr2=0x5561575efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561571f79f0 0x5561575efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 -- 172.21.15.29:0/2918520685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157218390 msgr2=0x5561575f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:14.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157218390 0x5561575f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 -- 172.21.15.29:0/2918520685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561576785a0 con 0x5561572101b0 2024-09-21T01:55:14.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.741+0000 7f305ffff640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f30540ad790 tx=0x7f30540e7860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=372c9134c130d437 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:14.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.744+0000 7f303ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:14.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.744+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540a7040 con 0x5561572101b0 2024-09-21T01:55:14.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.744+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3058036040 con 0x5561572101b0 2024-09-21T01:55:14.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.744+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30540a5f70 con 0x5561572101b0 2024-09-21T01:55:14.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.753+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f305409fed0 con 0x5561572101b0 2024-09-21T01:55:14.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.754+0000 7f306d67e740 10 client.26597 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:14.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.754+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5561576797c0 con 0x5561572101b0 2024-09-21T01:55:14.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.755+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3058036040 con 0x5561572101b0 2024-09-21T01:55:14.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f30540a9a70 con 0x5561572101b0 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 10 client.26597 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 10 client.26597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 20 client.26597 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x556157239070 0x55615767bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 4 client.26597 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.756+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561575548a0 con 0x556157239070 2024-09-21T01:55:14.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.759+0000 7f3064ea6640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x556157239070 0x55615767bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:14.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.760+0000 7f3064ea6640 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x556157239070 0x55615767bf30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51844cadc37c8e83 server_cookie=29a7ffbd026e51a9 in_seq=0 out_seq=0 2024-09-21T01:55:14.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.760+0000 7f303ffff640 10 client.26597 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:55:14.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f303ffff640 1 -- 172.21.15.29:0/2918520685 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561575548a0 con 0x556157239070 2024-09-21T01:55:14.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f303ffff640 10 client.26597 handle_command_reply: tid=0 2024-09-21T01:55:14.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f306d67e740 1 client.26597 shutdown 2024-09-21T01:55:14.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 msgr2=0x5561575f01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f30540ad790 tx=0x7f30540e7860 comp rx=0 tx=0).stop 2024-09-21T01:55:14.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 shutdown_connections 2024-09-21T01:55:14.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.761+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x556157239070 0x55615767bf30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556157218390 0x5561575f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561572101b0 0x5561575f01a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 --2- 172.21.15.29:0/2918520685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561571f79f0 0x5561575efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 >> 172.21.15.29:0/2918520685 conn(0x55615722d460 msgr2=0x5561575e9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f303ffff640 0 client.26597 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 shutdown_connections 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 1 -- 172.21.15.29:0/2918520685 wait complete. 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.762+0000 7f306d67e740 20 client.26597 trim_cache size 0 max 16384 2024-09-21T01:55:14.763 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 msgr2=0x7f30480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 0x7f30480b8a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3058009620 tx=0x7f3058005c50 comp rx=0 tx=0).stop 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 msgr2=0x7f3060113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9a0 tx=0x7f305400ee60 comp rx=0 tx=0).stop 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 shutdown_connections 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30480b6510 0x7f30480b8a20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3060116640 0x7f3060113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306010aee0 0x7f3060113950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 --2- 172.21.15.29:0/332470002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306010a590 0x7f3060113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:14.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 >> 172.21.15.29:0/332470002 conn(0x7f3060070ec0 msgr2=0x7f3060073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:14.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 shutdown_connections 2024-09-21T01:55:14.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:14.763+0000 7f306710a640 1 -- 172.21.15.29:0/332470002 wait complete. 2024-09-21T01:55:15.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:55:15.890 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:15 smithi029 ceph-mon[31228]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-09-21T01:55:15.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:15 smithi190 ceph-mon[34368]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-09-21T01:55:16.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:15 smithi066 ceph-mon[35001]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-09-21T01:55:16.255 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2538906994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c01094f0 msgr2=0x7f66c0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2538906994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c01094f0 0x7f66c0109900 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b402a410 comp rx=0 tx=0).stop 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2538906994 shutdown_connections 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2538906994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c0109e40 0x7f66c0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2538906994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c01094f0 0x7f66c0109900 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2538906994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2538906994 >> 172.21.15.29:0/2538906994 conn(0x7f66c00ff600 msgr2=0x7f66c0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2538906994 shutdown_connections 2024-09-21T01:55:16.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.857+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2538906994 wait complete. 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 Processor -- start 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 -- start start 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c01094f0 0x7f66c019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0113b00 con 0x7f66c0109e40 2024-09-21T01:55:16.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.858+0000 7f66c4d2f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0113980 con 0x7f66c0103bb0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66c4d2f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0113c80 con 0x7f66c01094f0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54128/0 (socket says 172.21.15.29:54128) 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66bed9d640 1 -- 172.21.15.29:0/2680717449 learned_addr learned my addr 172.21.15.29:0/2680717449 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40086/0 (socket says 172.21.15.29:40086) 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c01094f0 0x7f66c019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 -- 172.21.15.29:0/2680717449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c01094f0 msgr2=0x7f66c019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c01094f0 0x7f66c019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 -- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 msgr2=0x7f66c019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 -- 172.21.15.29:0/2680717449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c01a04a0 con 0x7f66c0103bb0 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66bed9d640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:16.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.859+0000 7f66be59c640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d6d0 tx=0x7f66b000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.860+0000 7f66af7fe640 1 -- 172.21.15.29:0/2680717449 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0017070 con 0x7f66c0103bb0 2024-09-21T01:55:16.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.860+0000 7f66af7fe640 1 -- 172.21.15.29:0/2680717449 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b0009820 con 0x7f66c0103bb0 2024-09-21T01:55:16.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.860+0000 7f66af7fe640 1 -- 172.21.15.29:0/2680717449 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0004c40 con 0x7f66c0103bb0 2024-09-21T01:55:16.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.860+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c01a0790 con 0x7f66c0103bb0 2024-09-21T01:55:16.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.861+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c01a0b90 con 0x7f66c0103bb0 2024-09-21T01:55:16.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.864+0000 7f66cb2a3740 1 Processor -- start 2024-09-21T01:55:16.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 -- start start 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8188f070 0x561b81c34100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8186e390 0x561b81c46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561b81bacea0 con 0x561b8188f070 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561b81b9c2d0 con 0x561b818661b0 2024-09-21T01:55:16.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.865+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561b81ba00e0 con 0x561b8186e390 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.868+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.868+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40102/0 (socket says 172.21.15.29:40102) 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.868+0000 7f66be59c640 1 -- 172.21.15.29:0/3065942169 learned_addr learned my addr 172.21.15.29:0/3065942169 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66bed9d640 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8188f070 0x561b81c34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66be59c640 1 -- 172.21.15.29:0/3065942169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8186e390 msgr2=0x561b81c46a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66be59c640 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8186e390 0x561b81c46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66be59c640 1 -- 172.21.15.29:0/3065942169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8188f070 msgr2=0x561b81c34100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66be59c640 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8188f070 0x561b81c34100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.869+0000 7f66be59c640 1 -- 172.21.15.29:0/3065942169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b818fa650 con 0x561b818661b0 2024-09-21T01:55:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.874+0000 7f66be59c640 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f66b00ef3c0 tx=0x7f66b00eaf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d074ea03958ed5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.876+0000 7f66af7fe640 1 -- 172.21.15.29:0/2680717449 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f66b0005220 con 0x7f66c0103bb0 2024-09-21T01:55:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.877+0000 7f66af7fe640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 0x7f66a80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.877+0000 7f66af7fe640 1 -- 172.21.15.29:0/2680717449 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f66b00e8e80 con 0x7f66c0103bb0 2024-09-21T01:55:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.878+0000 7f66acff9640 1 -- 172.21.15.29:0/3065942169 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00ab6c0 con 0x561b818661b0 2024-09-21T01:55:16.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.878+0000 7f66acff9640 1 -- 172.21.15.29:0/3065942169 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b00aeb80 con 0x561b818661b0 2024-09-21T01:55:16.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.878+0000 7f66bed9d640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 0x7f66a80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.879+0000 7f66acff9640 1 -- 172.21.15.29:0/3065942169 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00ae7e0 con 0x561b818661b0 2024-09-21T01:55:16.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.879+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3065942169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 msgr2=0x561b81c34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.879+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f66b00ef3c0 tx=0x7f66b00eaf90 comp rx=0 tx=0).stop 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3065942169 shutdown_connections 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8186e390 0x561b81c46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b818661b0 0x561b81c34680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3065942169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8188f070 0x561b81c34100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3065942169 >> 172.21.15.29:0/3065942169 conn(0x561b81883460 msgr2=0x561b81c36620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.883+0000 7f66bed9d640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 0x7f66a80b8b20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009fd0 tx=0x7f66b8009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.884+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3065942169 shutdown_connections 2024-09-21T01:55:16.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.884+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3065942169 wait complete. 2024-09-21T01:55:16.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.885+0000 7f66cb2a3740 1 -- start start 2024-09-21T01:55:16.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.885+0000 7f66cb2a3740 1 Processor -- start 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c44720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8188f070 0x561b81c44c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561b819fd700 con 0x561b8186e390 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561b81b37160 con 0x561b8188f070 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561b81c475f0 con 0x561b818661b0 2024-09-21T01:55:16.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.886+0000 7f66cb2a3740 10 client.0 _ll_register_callbacks cb 0x561b81b509e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:16.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.887+0000 7f66be59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.887+0000 7f66be59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36054/0 (socket says 172.21.15.29:36054) 2024-09-21T01:55:16.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.887+0000 7f66be59c640 1 -- 172.21.15.29:0/3041186295 learned_addr learned my addr 172.21.15.29:0/3041186295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.888+0000 7f66be59c640 1 -- 172.21.15.29:0/3041186295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8188f070 msgr2=0x561b81c44c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:55:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.888+0000 7f66be59c640 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8188f070 0x561b81c44c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.888+0000 7f66be59c640 1 -- 172.21.15.29:0/3041186295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 msgr2=0x561b81c44720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.888+0000 7f66be59c640 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c44720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.888+0000 7f66be59c640 1 -- 172.21.15.29:0/3041186295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b81b64c90 con 0x561b818661b0 2024-09-21T01:55:16.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.889+0000 7f66be59c640 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66b00ac2e0 tx=0x7f66b00ab0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c279d0978cf72a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.889+0000 7f66a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:16.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.889+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0020070 con 0x561b818661b0 2024-09-21T01:55:16.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.889+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b00aeb80 con 0x561b818661b0 2024-09-21T01:55:16.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.889+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f66b00ea9a0 con 0x561b818661b0 2024-09-21T01:55:16.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.894+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f66b00ae9c0 con 0x561b818661b0 2024-09-21T01:55:16.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.899+0000 7f66cb2a3740 10 client.26614 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:16.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.899+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b81c6a3d0 con 0x561b818661b0 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b00aeb80 con 0x561b818661b0 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f66b00b1070 con 0x561b818661b0 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 10 client.26614 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 10 client.26614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 20 client.26614 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b8184d9f0 0x561b81c6d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 4 client.26614 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.900+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b81baedf0 con 0x561b8184d9f0 2024-09-21T01:55:16.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.904+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b8184d9f0 0x561b81c6d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.915+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b8184d9f0 0x561b81c6d9b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af0287bb6232ef68 server_cookie=614678dfae3afb21 in_seq=0 out_seq=0 2024-09-21T01:55:16.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.915+0000 7f66a5ffb640 10 client.26614 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:55:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.916+0000 7f66a5ffb640 1 -- 172.21.15.29:0/3041186295 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b81baedf0 con 0x561b8184d9f0 2024-09-21T01:55:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.916+0000 7f66a5ffb640 10 client.26614 handle_command_reply: tid=0 2024-09-21T01:55:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.917+0000 7f66cb2a3740 1 client.26614 shutdown 2024-09-21T01:55:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.918+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 msgr2=0x561b81c441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.918+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66b00ac2e0 tx=0x7f66b00ab0d0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 shutdown_connections 2024-09-21T01:55:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8188f070 0x561b81c44c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c44720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b8184d9f0 0x561b81c6d9b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/3041186295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b818661b0 0x561b81c441e0 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66b00ac2e0 tx=0x7f66b00ab0d0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 >> 172.21.15.29:0/3041186295 conn(0x561b81883460 msgr2=0x561b81c33570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66a5ffb640 0 client.26614 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.919+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 shutdown_connections 2024-09-21T01:55:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.920+0000 7f66cb2a3740 1 -- 172.21.15.29:0/3041186295 wait complete. 2024-09-21T01:55:16.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.920+0000 7f66cb2a3740 20 client.26614 trim_cache size 0 max 16384 2024-09-21T01:55:16.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.948+0000 7f66cb2a3740 1 Processor -- start 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.949+0000 7f66cb2a3740 1 -- start start 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b81822da0 0x561b81c43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0075380 con 0x561b81822da0 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0076ef0 con 0x561b8186e390 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0075a50 con 0x561b8188f070 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40112/0 (socket says 172.21.15.29:40112) 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66be59c640 1 -- 172.21.15.29:0/1669418843 learned_addr learned my addr 172.21.15.29:0/1669418843 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66bed9d640 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b81822da0 0x561b81c43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66be59c640 1 -- 172.21.15.29:0/1669418843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 msgr2=0x561b81c36570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.950+0000 7f66be59c640 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c36570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66be59c640 1 -- 172.21.15.29:0/1669418843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b81822da0 msgr2=0x561b81c43170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66be59c640 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b81822da0 0x561b81c43170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66be59c640 1 -- 172.21.15.29:0/1669418843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b81c34eb0 con 0x561b8186e390 2024-09-21T01:55:16.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66be59c640 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f66b00aed70 tx=0x7f66b0009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f036d1077cd30b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66a7fff640 1 -- 172.21.15.29:0/1669418843 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0020070 con 0x561b8186e390 2024-09-21T01:55:16.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66a7fff640 1 -- 172.21.15.29:0/1669418843 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b00b87d0 con 0x561b8186e390 2024-09-21T01:55:16.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.951+0000 7f66a7fff640 1 -- 172.21.15.29:0/1669418843 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00aee40 con 0x561b8186e390 2024-09-21T01:55:16.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.952+0000 7f66cb2a3740 1 -- 172.21.15.29:0/1669418843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 msgr2=0x561b81c67af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.952+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f66b00aed70 tx=0x7f66b0009f00 comp rx=0 tx=0).stop 2024-09-21T01:55:16.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 -- 172.21.15.29:0/1669418843 shutdown_connections 2024-09-21T01:55:16.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b81822da0 0x561b81c43170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c36570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/1669418843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b8186e390 0x561b81c67af0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 -- 172.21.15.29:0/1669418843 >> 172.21.15.29:0/1669418843 conn(0x561b81883460 msgr2=0x561b81c42a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 -- 172.21.15.29:0/1669418843 shutdown_connections 2024-09-21T01:55:16.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.953+0000 7f66cb2a3740 1 -- 172.21.15.29:0/1669418843 wait complete. 2024-09-21T01:55:16.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.954+0000 7f66cb2a3740 1 -- start start 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.954+0000 7f66cb2a3740 1 Processor -- start 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0076820 con 0x561b8186e390 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0076150 con 0x561b81822da0 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0073360 con 0x561b8188f070 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66cb2a3740 10 client.0 _ll_register_callbacks cb 0x7f66b00eae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40122/0 (socket says 172.21.15.29:40122) 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 -- 172.21.15.29:0/2549967974 learned_addr learned my addr 172.21.15.29:0/2549967974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66bed9d640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 -- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 msgr2=0x561b81c64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 -- 172.21.15.29:0/2549967974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 msgr2=0x561b81c63b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c63b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 -- 172.21.15.29:0/2549967974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b81c352a0 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66bed9d640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66be59c640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d6a0 tx=0x7f66b00ab4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570892e28ba7887 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.955+0000 7f66a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0020070 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f66b00ea200 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b00b87d0 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f66b00c6020 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 10 client.26612 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b81c352a0 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b00aeb80 con 0x561b81822da0 2024-09-21T01:55:16.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f66b00b1030 con 0x561b81822da0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 10 client.26612 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 10 client.26612 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 20 client.26612 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b81cd84a0 0x561b81cda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 4 client.26612 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.956+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f66b00b87d0 con 0x561b81cd84a0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b81cd84a0 0x561b81cda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66a5ffb640 10 client.26612 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66bdd9b640 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b81cd84a0 0x561b81cda8c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f2d2abc5fed9417 server_cookie=102c0583338e420 in_seq=0 out_seq=0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66a5ffb640 1 -- 172.21.15.29:0/2549967974 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f66b00b87d0 con 0x561b81cd84a0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66a5ffb640 10 client.26612 handle_command_reply: tid=0 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.957+0000 7f66cb2a3740 1 client.26612 shutdown 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 msgr2=0x561b81c73970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d6a0 tx=0x7f66b00ab4b0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 shutdown_connections 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x561b81cd84a0 0x561b81cda8c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b8188f070 0x561b81c64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66a5ffb640 0 client.26612 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561b8186e390 0x561b81c63b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 --2- 172.21.15.29:0/2549967974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b81822da0 0x561b81c73970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 >> 172.21.15.29:0/2549967974 conn(0x561b81883460 msgr2=0x561b81c44150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 shutdown_connections 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 1 -- 172.21.15.29:0/2549967974 wait complete. 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.958+0000 7f66cb2a3740 20 client.26612 trim_cache size 0 max 16384 2024-09-21T01:55:16.963 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 msgr2=0x7f66a80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 0x7f66a80b8b20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009fd0 tx=0x7f66b8009680 comp rx=0 tx=0).stop 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 msgr2=0x7f66c019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d6d0 tx=0x7f66b000db90 comp rx=0 tx=0).stop 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 shutdown_connections 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f66a80b6610 0x7f66a80b8b20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0109e40 0x7f66c019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66c01094f0 0x7f66c019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 --2- 172.21.15.29:0/2680717449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0103bb0 0x7f66c019a530 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:16.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 >> 172.21.15.29:0/2680717449 conn(0x7f66c00ff600 msgr2=0x7f66c0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:16.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 shutdown_connections 2024-09-21T01:55:16.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:16.959+0000 7f66c4d2f640 1 -- 172.21.15.29:0/2680717449 wait complete. 2024-09-21T01:55:17.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:55:17.762 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:17 smithi029 ceph-mon[31228]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-21T01:55:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 400.0 s, 2290.3 tps, lat 45.941 ms stddev 77.229 2024-09-21T01:55:17.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:17 smithi190 ceph-mon[34368]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-21T01:55:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:17 smithi066 ceph-mon[35001]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-21T01:55:18.119 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:19.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 -- 172.21.15.29:0/3664074753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac102630 msgr2=0x7f7eac106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/3664074753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac102630 0x7f7eac106190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009be0 tx=0x7f7ea402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 -- 172.21.15.29:0/3664074753 shutdown_connections 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/3664074753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac102630 0x7f7eac106190 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/3664074753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac101ce0 0x7f7eac1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/3664074753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac1053f0 0x7f7eac105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 -- 172.21.15.29:0/3664074753 >> 172.21.15.29:0/3664074753 conn(0x7f7eac077c60 msgr2=0x7f7eac0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 -- 172.21.15.29:0/3664074753 shutdown_connections 2024-09-21T01:55:19.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.502+0000 7f7eb22d4640 1 -- 172.21.15.29:0/3664074753 wait complete. 2024-09-21T01:55:19.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.503+0000 7f7eb22d4640 1 Processor -- start 2024-09-21T01:55:19.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.503+0000 7f7eb22d4640 1 -- start start 2024-09-21T01:55:19.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac102630 0x7f7eac1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 0x7f7eac1a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac11a090 con 0x7f7eac101ce0 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac119f10 con 0x7f7eac1053f0 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac11a210 con 0x7f7eac102630 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54158/0 (socket says 172.21.15.29:54158) 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 -- 172.21.15.29:0/520569824 learned_addr learned my addr 172.21.15.29:0/520569824 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:19.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eb0871640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 0x7f7eac1a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 -- 172.21.15.29:0/520569824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac102630 msgr2=0x7f7eac1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac102630 0x7f7eac1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac102630 0x7f7eac1a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 -- 172.21.15.29:0/520569824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 msgr2=0x7f7eac1a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.505+0000 7f7eabfff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 0x7f7eac1a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7eabfff640 1 -- 172.21.15.29:0/520569824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1a69d0 con 0x7f7eac101ce0 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7eb0871640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 0x7f7eac1a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7eabfff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9f0 tx=0x7f7e9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7ea9ffb640 1 -- 172.21.15.29:0/520569824 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400ccc0 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1a6cc0 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7ea9ffb640 1 -- 172.21.15.29:0/520569824 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e940054d0 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.506+0000 7f7ea9ffb640 1 -- 172.21.15.29:0/520569824 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94010430 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.507+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1ad420 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.509+0000 7f7ea9ffb640 1 -- 172.21.15.29:0/520569824 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7e940046c0 con 0x7f7eac101ce0 2024-09-21T01:55:19.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.511+0000 7f7ea9ffb640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 0x7f7e980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.511+0000 7f7ea9ffb640 1 -- 172.21.15.29:0/520569824 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7e940e76b0 con 0x7f7eac101ce0 2024-09-21T01:55:19.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.511+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 0x7f7e980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.512+0000 7f7eb8848740 1 Processor -- start 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.512+0000 7f7eb8848740 1 -- start start 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.512+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 0x7f7e980b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7eac108220 tx=0x7f7e9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5628a2318ea0 con 0x5628a1ffb070 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5628a23082d0 con 0x5628a1fda390 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb8848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5628a230c0e0 con 0x5628a1fd21b0 2024-09-21T01:55:19.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54164/0 (socket says 172.21.15.29:54164) 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 -- 172.21.15.29:0/3938868159 learned_addr learned my addr 172.21.15.29:0/3938868159 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eb0871640 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 -- 172.21.15.29:0/3938868159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 msgr2=0x5628a23a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 -- 172.21.15.29:0/3938868159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 msgr2=0x5628a23a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 -- 172.21.15.29:0/3938868159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a2066650 con 0x5628a1ffb070 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.513+0000 7f7eabfff640 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009230 tx=0x7f7e9400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1049d55a7db91b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.514+0000 7f7ea27fc640 1 -- 172.21.15.29:0/3938868159 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940adf50 con 0x5628a1ffb070 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.514+0000 7f7ea27fc640 1 -- 172.21.15.29:0/3938868159 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e940b2330 con 0x5628a1ffb070 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.514+0000 7f7ea27fc640 1 -- 172.21.15.29:0/3938868159 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940aeb80 con 0x5628a1ffb070 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- 172.21.15.29:0/3938868159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 msgr2=0x5628a23a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009230 tx=0x7f7e9400ce60 comp rx=0 tx=0).stop 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- 172.21.15.29:0/3938868159 shutdown_connections 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3938868159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23a22c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- 172.21.15.29:0/3938868159 >> 172.21.15.29:0/3938868159 conn(0x5628a1fef460 msgr2=0x5628a23a01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- 172.21.15.29:0/3938868159 shutdown_connections 2024-09-21T01:55:19.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- 172.21.15.29:0/3938868159 wait complete. 2024-09-21T01:55:19.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.515+0000 7f7eb8848740 1 -- start start 2024-09-21T01:55:19.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.516+0000 7f7eb8848740 1 Processor -- start 2024-09-21T01:55:19.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.516+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fd21b0 0x5628a23ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5628a2169700 con 0x5628a1ffb070 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5628a22a3160 con 0x5628a1fd21b0 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5628a23b0aa0 con 0x5628a1fda390 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb8848740 10 client.0 _ll_register_callbacks cb 0x5628a22bc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36110/0 (socket says 172.21.15.29:36110) 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7ea3fff640 1 -- 172.21.15.29:0/2261931261 learned_addr learned my addr 172.21.15.29:0/2261931261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eabfff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fd21b0 0x5628a23ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.517+0000 7f7eb0871640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 -- 172.21.15.29:0/2261931261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fd21b0 msgr2=0x5628a23ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fd21b0 0x5628a23ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 -- 172.21.15.29:0/2261931261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 msgr2=0x5628a23b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 -- 172.21.15.29:0/2261931261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a22d0c90 con 0x5628a1fda390 2024-09-21T01:55:19.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c012450 tx=0x7f7e9c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dedf09900e612a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c020840 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7e9c0209e0 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.518+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e940b2330 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e9c020be0 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 10 client.26626 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5628a23bc7e0 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e940b2330 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f7e9c00f040 con 0x5628a1fda390 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 10 client.26626 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 10 client.26626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 20 client.26626 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:19.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1fb99f0 0x5628a23bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 4 client.26626 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.519+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628a231adf0 con 0x5628a1fb99f0 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.520+0000 7f7eabfff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1fb99f0 0x5628a23bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.520+0000 7f7eabfff640 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1fb99f0 0x5628a23bfd30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9fc583b641abfaa server_cookie=7e6b4703f05b5d13 in_seq=0 out_seq=0 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.520+0000 7f7e8b7fe640 10 client.26626 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/2261931261 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628a231adf0 con 0x5628a1fb99f0 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7e8b7fe640 10 client.26626 handle_command_reply: tid=0 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 client.26626 shutdown 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 msgr2=0x5628a23ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c012450 tx=0x7f7e9c0046c0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 shutdown_connections 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1fb99f0 0x5628a23bfd30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fda390 0x5628a23ade90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 --2- 172.21.15.29:0/2261931261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fd21b0 0x5628a23ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 >> 172.21.15.29:0/2261931261 conn(0x5628a1fef460 msgr2=0x5628a23a54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7e8b7fe640 0 client.26626 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 shutdown_connections 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 1 -- 172.21.15.29:0/2261931261 wait complete. 2024-09-21T01:55:19.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.521+0000 7f7eb8848740 20 client.26626 trim_cache size 0 max 16384 2024-09-21T01:55:19.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 Processor -- start 2024-09-21T01:55:19.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 -- start start 2024-09-21T01:55:19.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1ffb070 0x5628a23aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b5950 con 0x5628a1fd21b0 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b74c0 con 0x5628a1fda390 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.541+0000 7f7eb8848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b6020 con 0x5628a1ffb070 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40170/0 (socket says 172.21.15.29:40170) 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54184/0 (socket says 172.21.15.29:54184) 2024-09-21T01:55:19.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 -- 172.21.15.29:0/3736003397 learned_addr learned my addr 172.21.15.29:0/3736003397 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 -- 172.21.15.29:0/3736003397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1ffb070 msgr2=0x5628a23aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1ffb070 0x5628a23aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 -- 172.21.15.29:0/3736003397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 msgr2=0x5628a23ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7ea3fff640 1 -- 172.21.15.29:0/3736003397 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a23c0260 con 0x5628a1fda390 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.542+0000 7f7eabfff640 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.543+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0107b0 tx=0x7f7e9c005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b685453af734ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.543+0000 7f7ea1ffb640 1 -- 172.21.15.29:0/3736003397 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0117f0 con 0x5628a1fda390 2024-09-21T01:55:19.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.543+0000 7f7ea1ffb640 1 -- 172.21.15.29:0/3736003397 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c024070 con 0x5628a1fda390 2024-09-21T01:55:19.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.543+0000 7f7ea1ffb640 1 -- 172.21.15.29:0/3736003397 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0053b0 con 0x5628a1fda390 2024-09-21T01:55:19.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.544+0000 7f7eb8848740 1 -- 172.21.15.29:0/3736003397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 msgr2=0x5628a23b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.544+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0107b0 tx=0x7f7e9c005cd0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 -- 172.21.15.29:0/3736003397 shutdown_connections 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1ffb070 0x5628a23aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b9c90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 --2- 172.21.15.29:0/3736003397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1fd21b0 0x5628a23ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 -- 172.21.15.29:0/3736003397 >> 172.21.15.29:0/3736003397 conn(0x5628a1fef460 msgr2=0x5628a23ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 -- 172.21.15.29:0/3736003397 shutdown_connections 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.545+0000 7f7eb8848740 1 -- 172.21.15.29:0/3736003397 wait complete. 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.546+0000 7f7eb8848740 1 -- start start 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.546+0000 7f7eb8848740 1 Processor -- start 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b6df0 con 0x5628a1ffb070 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b6720 con 0x5628a1fda390 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b3930 con 0x5628a1fd21b0 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb8848740 10 client.0 _ll_register_callbacks cb 0x7f7e9c009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40182/0 (socket says 172.21.15.29:40182) 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 -- 172.21.15.29:0/1919404168 learned_addr learned my addr 172.21.15.29:0/1919404168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eabfff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eb0871640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 -- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 msgr2=0x5628a23b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 -- 172.21.15.29:0/1919404168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 msgr2=0x5628a23b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7ea3fff640 1 -- 172.21.15.29:0/1919404168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a24297b0 con 0x5628a1fda390 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.547+0000 7f7eabfff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.548+0000 7f7ea3fff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011bf0 tx=0x7f7e9c011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=297a8ab23265db4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.548+0000 7f7e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.548+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c005450 con 0x5628a1fda390 2024-09-21T01:55:19.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.548+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c024070 con 0x5628a1fda390 2024-09-21T01:55:19.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.548+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7e9c0055f0 con 0x5628a1fda390 2024-09-21T01:55:19.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.550+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e9c012ce0 con 0x5628a1fda390 2024-09-21T01:55:19.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.550+0000 7f7eb8848740 10 client.26627 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:19.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.550+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5628a242aa10 con 0x5628a1fda390 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c024070 con 0x5628a1fda390 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f7e9c03c020 con 0x5628a1fda390 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 10 client.26627 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 10 client.26627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 20 client.26627 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1f8eda0 0x5628a242d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 4 client.26627 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.551+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628a23168a0 con 0x5628a1f8eda0 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7eabfff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1f8eda0 0x5628a242d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:19.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7e8b7fe640 10 client.26627 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7eabfff640 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1f8eda0 0x5628a242d060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab89a1c20391c539 server_cookie=d1d2efd797b80282 in_seq=0 out_seq=0 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7e8b7fe640 1 -- 172.21.15.29:0/1919404168 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628a23168a0 con 0x5628a1f8eda0 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7e8b7fe640 10 client.26627 handle_command_reply: tid=0 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.552+0000 7f7eb8848740 1 client.26627 shutdown 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 msgr2=0x5628a23b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011bf0 tx=0x7f7e9c011c20 comp rx=0 tx=0).stop 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 shutdown_connections 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628a1f8eda0 0x5628a242d060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628a1ffb070 0x5628a23b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628a1fda390 0x5628a23b22f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7e8b7fe640 0 client.26627 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 --2- 172.21.15.29:0/1919404168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628a1fd21b0 0x5628a23b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 >> 172.21.15.29:0/1919404168 conn(0x5628a1fef460 msgr2=0x5628a23a32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 shutdown_connections 2024-09-21T01:55:19.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 1 -- 172.21.15.29:0/1919404168 wait complete. 2024-09-21T01:55:19.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.553+0000 7f7eb8848740 20 client.26627 trim_cache size 0 max 16384 2024-09-21T01:55:19.557 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.554+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 msgr2=0x7f7e980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.554+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 0x7f7e980b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7eac108220 tx=0x7f7e9c005f90 comp rx=0 tx=0).stop 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.554+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 msgr2=0x7f7eac1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.554+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9f0 tx=0x7f7e9400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 shutdown_connections 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7e980b6510 0x7f7e980b8a20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac1053f0 0x7f7eac1a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eac102630 0x7f7eac1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 --2- 172.21.15.29:0/520569824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac101ce0 0x7f7eac1a0a60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:19.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 >> 172.21.15.29:0/520569824 conn(0x7f7eac077c60 msgr2=0x7f7eac100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:19.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 shutdown_connections 2024-09-21T01:55:19.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:19.555+0000 7f7eb22d4640 1 -- 172.21.15.29:0/520569824 wait complete. 2024-09-21T01:55:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:19 smithi190 ceph-mon[34368]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-21T01:55:20.047 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:19 smithi029 ceph-mon[31228]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-21T01:55:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:19 smithi066 ceph-mon[35001]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-21T01:55:20.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:55:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f4705ffb640 1 -- 172.21.15.29:0/874646788 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80318a0 con 0x7f4708102a60 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 -- 172.21.15.29:0/874646788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4708102a60 msgr2=0x7f4708102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 --2- 172.21.15.29:0/874646788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4708102a60 0x7f4708102e70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009900 tx=0x7f46f802a410 comp rx=0 tx=0).stop 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 -- 172.21.15.29:0/874646788 shutdown_connections 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 --2- 172.21.15.29:0/874646788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708108cf0 0x7f470810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 --2- 172.21.15.29:0/874646788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47081083a0 0x7f47081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 --2- 172.21.15.29:0/874646788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4708102a60 0x7f4708102e70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 -- 172.21.15.29:0/874646788 >> 172.21.15.29:0/874646788 conn(0x7f4708079730 msgr2=0x7f4708079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 -- 172.21.15.29:0/874646788 shutdown_connections 2024-09-21T01:55:21.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.561+0000 7f470d561640 1 -- 172.21.15.29:0/874646788 wait complete. 2024-09-21T01:55:21.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.562+0000 7f470d561640 1 Processor -- start 2024-09-21T01:55:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.563+0000 7f470d561640 1 -- start start 2024-09-21T01:55:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708102a60 0x7f47081115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708108cf0 0x7f47081100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113ac0 con 0x7f4708108cf0 2024-09-21T01:55:21.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113940 con 0x7f4708102a60 2024-09-21T01:55:21.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.564+0000 7f470d561640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113c40 con 0x7f47081083a0 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.565+0000 7f47067fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.565+0000 7f47067fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36140/0 (socket says 172.21.15.29:36140) 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.565+0000 7f47067fc640 1 -- 172.21.15.29:0/3788920654 learned_addr learned my addr 172.21.15.29:0/3788920654 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.565+0000 7f47077fe640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708108cf0 0x7f47081100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.565+0000 7f4706ffd640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708102a60 0x7f47081115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 -- 172.21.15.29:0/3788920654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708102a60 msgr2=0x7f47081115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708102a60 0x7f47081115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 -- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708108cf0 msgr2=0x7f47081100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708108cf0 0x7f47081100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 -- 172.21.15.29:0/3788920654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4708110910 con 0x7f47081083a0 2024-09-21T01:55:21.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f47067fc640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f470000eb90 tx=0x7f470000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f46effff640 1 -- 172.21.15.29:0/3788920654 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700010630 con 0x7f47081083a0 2024-09-21T01:55:21.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f46effff640 1 -- 172.21.15.29:0/3788920654 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470000ee90 con 0x7f47081083a0 2024-09-21T01:55:21.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47081ace30 con 0x7f47081083a0 2024-09-21T01:55:21.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f46effff640 1 -- 172.21.15.29:0/3788920654 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700018ac0 con 0x7f47081083a0 2024-09-21T01:55:21.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.566+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081ad2e0 con 0x7f47081083a0 2024-09-21T01:55:21.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.569+0000 7f46effff640 1 -- 172.21.15.29:0/3788920654 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4700018c60 con 0x7f47081083a0 2024-09-21T01:55:21.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.570+0000 7f4713ad5740 1 Processor -- start 2024-09-21T01:55:21.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 -- start start 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec83e390 0x5599ecc07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5599ecb7cea0 con 0x5599ec85f070 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ecb6c2d0 con 0x5599ec83e390 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.571+0000 7f4713ad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599ecb700e0 con 0x5599ec8361b0 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54220/0 (socket says 172.21.15.29:54220) 2024-09-21T01:55:21.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 -- 172.21.15.29:0/3378460464 learned_addr learned my addr 172.21.15.29:0/3378460464 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f46effff640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 0x7f46f00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f46effff640 1 -- 172.21.15.29:0/3788920654 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f47000e7930 con 0x7f47081083a0 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 -- 172.21.15.29:0/3378460464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 msgr2=0x5599ecc06be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc06be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 -- 172.21.15.29:0/3378460464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec83e390 msgr2=0x5599ecc07120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec83e390 0x5599ecc07120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 -- 172.21.15.29:0/3378460464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ec8ca650 con 0x5599ec85f070 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 0x7f46f00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.572+0000 7f47077fe640 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b850 tx=0x7f46fc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57810e22d58b5f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.573+0000 7f46ed7fa640 1 -- 172.21.15.29:0/3378460464 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc022070 con 0x5599ec85f070 2024-09-21T01:55:21.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.573+0000 7f46ed7fa640 1 -- 172.21.15.29:0/3378460464 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46fc010dc0 con 0x5599ec85f070 2024-09-21T01:55:21.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.573+0000 7f46ed7fa640 1 -- 172.21.15.29:0/3378460464 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc01c860 con 0x5599ec85f070 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- 172.21.15.29:0/3378460464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 msgr2=0x5599ecc066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f47077fe640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 0x7f46f00b8aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46fc01f000 tx=0x7f46fc01c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b850 tx=0x7f46fc004610 comp rx=0 tx=0).stop 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- 172.21.15.29:0/3378460464 shutdown_connections 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec83e390 0x5599ecc07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc06be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 --2- 172.21.15.29:0/3378460464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc066a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- 172.21.15.29:0/3378460464 >> 172.21.15.29:0/3378460464 conn(0x5599ec853460 msgr2=0x5599ecc04590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- 172.21.15.29:0/3378460464 shutdown_connections 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- 172.21.15.29:0/3378460464 wait complete. 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.574+0000 7f4713ad5740 1 -- start start 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 Processor -- start 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc11d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5599ec9cd700 con 0x5599ec83e390 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ecb07160 con 0x5599ec8361b0 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599ecc14e80 con 0x5599ec85f070 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4713ad5740 10 client.0 _ll_register_callbacks cb 0x5599ecb209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f47077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f47067fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f47077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36160/0 (socket says 172.21.15.29:36160) 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f47077fe640 1 -- 172.21.15.29:0/192890471 learned_addr learned my addr 172.21.15.29:0/192890471 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.575+0000 7f4706ffd640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc11d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.576+0000 7f47077fe640 1 -- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 msgr2=0x5599ecc11d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.576+0000 7f47077fe640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc11d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.576+0000 7f47077fe640 1 -- 172.21.15.29:0/192890471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 msgr2=0x5599ecc122b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.576+0000 7f47077fe640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc122b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.576+0000 7f47077fe640 1 -- 172.21.15.29:0/192890471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ecb34c90 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.577+0000 7f47077fe640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f46fc005bd0 tx=0x7f46fc013f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e57995cb90b6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.577+0000 7f46eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.577+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc022040 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.577+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f46fc013190 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.578+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46fc010dc0 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.578+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46fc013390 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.578+0000 7f4713ad5740 10 client.26635 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.578+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5599ecc20ba0 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.578+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46fc013530 con 0x5599ec85f070 2024-09-21T01:55:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f46fc02e020 con 0x5599ec85f070 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 10 client.26635 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 10 client.26635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 20 client.26635 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec81d9f0 0x5599ecc24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 4 client.26635 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.579+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f46fc010dc0 con 0x5599ec81d9f0 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.580+0000 7f4706ffd640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec81d9f0 0x5599ecc24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.580+0000 7f46eaff5640 10 client.26635 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.580+0000 7f4706ffd640 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec81d9f0 0x5599ecc24010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67ef67988d38ae70 server_cookie=7835d339caa78051 in_seq=0 out_seq=0 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f46eaff5640 1 -- 172.21.15.29:0/192890471 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f46fc010dc0 con 0x5599ec81d9f0 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f46eaff5640 10 client.26635 handle_command_reply: tid=0 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 client.26635 shutdown 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 msgr2=0x5599ecc1e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f46fc005bd0 tx=0x7f46fc013f00 comp rx=0 tx=0).stop 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 shutdown_connections 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec81d9f0 0x5599ecc24010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec85f070 0x5599ecc1e170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f46eaff5640 0 client.26635 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 --2- 172.21.15.29:0/192890471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc11d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 >> 172.21.15.29:0/192890471 conn(0x5599ec853460 msgr2=0x5599ecc09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 shutdown_connections 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 1 -- 172.21.15.29:0/192890471 wait complete. 2024-09-21T01:55:21.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.581+0000 7f4713ad5740 20 client.26635 trim_cache size 0 max 16384 2024-09-21T01:55:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.597+0000 7f4713ad5740 1 Processor -- start 2024-09-21T01:55:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.597+0000 7f4713ad5740 1 -- start start 2024-09-21T01:55:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.598+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.598+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.598+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec85f070 0x5599ecc0f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.598+0000 7f4713ad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b59f0 con 0x5599ec83e390 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.598+0000 7f4706ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:36164/0 (socket says 172.21.15.29:36164) 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 -- 172.21.15.29:0/2050148480 learned_addr learned my addr 172.21.15.29:0/2050148480 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b7560 con 0x5599ec85f070 2024-09-21T01:55:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b60c0 con 0x5599ec8361b0 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 -- 172.21.15.29:0/2050148480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec85f070 msgr2=0x5599ecc0f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec85f070 0x5599ecc0f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 -- 172.21.15.29:0/2050148480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 msgr2=0x5599ecc0ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.599+0000 7f4706ffd640 1 -- 172.21.15.29:0/2050148480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ecc13030 con 0x5599ec8361b0 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.600+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a6de9cae28c77a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.600+0000 7f46ecff9640 1 -- 172.21.15.29:0/2050148480 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035040 con 0x5599ec8361b0 2024-09-21T01:55:21.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.600+0000 7f46ecff9640 1 -- 172.21.15.29:0/2050148480 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80054f0 con 0x5599ec8361b0 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.600+0000 7f46ecff9640 1 -- 172.21.15.29:0/2050148480 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80314a0 con 0x5599ec8361b0 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 msgr2=0x5599ecc139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8005a20 comp rx=0 tx=0).stop 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 shutdown_connections 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec85f070 0x5599ecc0f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec83e390 0x5599ecc0ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2050148480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec8361b0 0x5599ecc139a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 >> 172.21.15.29:0/2050148480 conn(0x5599ec853460 msgr2=0x5599ecc13290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.601+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 shutdown_connections 2024-09-21T01:55:21.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.602+0000 7f4713ad5740 1 -- 172.21.15.29:0/2050148480 wait complete. 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.602+0000 7f4713ad5740 1 -- start start 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.602+0000 7f4713ad5740 1 Processor -- start 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec83e390 0x5599ecc17c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b6e90 con 0x5599ec85f070 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b67c0 con 0x5599ec8361b0 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47081b39d0 con 0x5599ec83e390 2024-09-21T01:55:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4713ad5740 10 client.0 _ll_register_callbacks cb 0x7f46fc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4706ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f47077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4706ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40240/0 (socket says 172.21.15.29:40240) 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f4706ffd640 1 -- 172.21.15.29:0/2720533209 learned_addr learned my addr 172.21.15.29:0/2720533209 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.603+0000 7f47067fc640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec83e390 0x5599ecc17c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 -- 172.21.15.29:0/2720533209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec83e390 msgr2=0x5599ecc17c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec83e390 0x5599ecc17c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 -- 172.21.15.29:0/2720533209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 msgr2=0x5599ecc181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 -- 172.21.15.29:0/2720533209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ecc18860 con 0x5599ec8361b0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f4706ffd640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46f8004650 tx=0x7f46f8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d60cc9406c93f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.604+0000 7f46eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:21.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.605+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8051070 con 0x5599ec8361b0 2024-09-21T01:55:21.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.605+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f46f8004ca0 con 0x5599ec8361b0 2024-09-21T01:55:21.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.605+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f80054f0 con 0x5599ec8361b0 2024-09-21T01:55:21.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.606+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46f8035040 con 0x5599ec8361b0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.606+0000 7f4713ad5740 10 client.26639 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.606+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5599ecc8e6d0 con 0x5599ec8361b0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.606+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80054f0 con 0x5599ec8361b0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f46f8031ad0 con 0x5599ec8361b0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 10 client.26639 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 10 client.26639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 20 client.26639 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec7f2da0 0x5599ecc90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 4 client.26639 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599ecb7edf0 con 0x5599ec7f2da0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f47067fc640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec7f2da0 0x5599ecc90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.607+0000 7f46eaff5640 10 client.26639 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:55:21.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f47067fc640 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec7f2da0 0x5599ecc90e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98105ab6532803c7 server_cookie=1aa111fd9ec48a64 in_seq=0 out_seq=0 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f46eaff5640 1 -- 172.21.15.29:0/2720533209 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599ecb7edf0 con 0x5599ec7f2da0 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f46eaff5640 10 client.26639 handle_command_reply: tid=0 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 client.26639 shutdown 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 msgr2=0x5599ecc1a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46f8004650 tx=0x7f46f8005770 comp rx=0 tx=0).stop 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 shutdown_connections 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5599ec7f2da0 0x5599ecc90e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ec85f070 0x5599ecc181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ec83e390 0x5599ecc17c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 --2- 172.21.15.29:0/2720533209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ec8361b0 0x5599ecc1a7b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 >> 172.21.15.29:0/2720533209 conn(0x5599ec853460 msgr2=0x5599ecc07680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f46eaff5640 0 client.26639 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 shutdown_connections 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.608+0000 7f4713ad5740 1 -- 172.21.15.29:0/2720533209 wait complete. 2024-09-21T01:55:21.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f4713ad5740 20 client.26639 trim_cache size 0 max 16384 2024-09-21T01:55:21.612 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:55:21.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 msgr2=0x7f46f00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 0x7f46f00b8aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46fc01f000 tx=0x7f46fc01c000 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 msgr2=0x7f470810fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f470000eb90 tx=0x7f470000c6a0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 shutdown_connections 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46f00b6590 0x7f46f00b8aa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4708108cf0 0x7f47081100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47081083a0 0x7f470810fbb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 --2- 172.21.15.29:0/3788920654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4708102a60 0x7f47081115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 >> 172.21.15.29:0/3788920654 conn(0x7f4708079730 msgr2=0x7f47081013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.609+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 shutdown_connections 2024-09-21T01:55:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:21.610+0000 7f470d561640 1 -- 172.21.15.29:0/3788920654 wait complete. 2024-09-21T01:55:21.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:21 smithi029 ceph-mon[31228]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 768 op/s 2024-09-21T01:55:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:21 smithi190 ceph-mon[34368]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 768 op/s 2024-09-21T01:55:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:21 smithi066 ceph-mon[35001]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 768 op/s 2024-09-21T01:55:22.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:55:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 405.0 s, 2386.2 tps, lat 42.488 ms stddev 52.811 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 -- 172.21.15.29:0/3912056999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 msgr2=0x7fd25c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 --2- 172.21.15.29:0/3912056999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c10f560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd25400bd70 tx=0x7fd25402c470 comp rx=0 tx=0).stop 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 -- 172.21.15.29:0/3912056999 shutdown_connections 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 --2- 172.21.15.29:0/3912056999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c10f560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 --2- 172.21.15.29:0/3912056999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 0x7fd25c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 --2- 172.21.15.29:0/3912056999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd25c102aa0 0x7fd25c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 -- 172.21.15.29:0/3912056999 >> 172.21.15.29:0/3912056999 conn(0x7fd25c079750 msgr2=0x7fd25c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.418+0000 7fd262ccc640 1 -- 172.21.15.29:0/3912056999 shutdown_connections 2024-09-21T01:55:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.419+0000 7fd262ccc640 1 -- 172.21.15.29:0/3912056999 wait complete. 2024-09-21T01:55:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.420+0000 7fd262ccc640 1 Processor -- start 2024-09-21T01:55:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.420+0000 7fd262ccc640 1 -- start start 2024-09-21T01:55:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd25c102aa0 0x7fd25c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 0x7fd25c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c111930 con 0x7fd25c102aa0 2024-09-21T01:55:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1117b0 con 0x7fd25c1083e0 2024-09-21T01:55:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.421+0000 7fd262ccc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c111ab0 con 0x7fd25c108d30 2024-09-21T01:55:23.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.422+0000 7fd261269640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.422+0000 7fd261269640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57130/0 (socket says 172.21.15.29:57130) 2024-09-21T01:55:23.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.422+0000 7fd261269640 1 -- 172.21.15.29:0/4000915014 learned_addr learned my addr 172.21.15.29:0/4000915014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:23.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.422+0000 7fd25bfff640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 0x7fd25c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 -- 172.21.15.29:0/4000915014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 msgr2=0x7fd25c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 0x7fd25c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 -- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd25c102aa0 msgr2=0x7fd25c19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd25c102aa0 0x7fd25c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 -- 172.21.15.29:0/4000915014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c1accc0 con 0x7fd25c108d30 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.423+0000 7fd261269640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd254006010 tx=0x7fd25400b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.424+0000 7fd259ffb640 1 -- 172.21.15.29:0/4000915014 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254038040 con 0x7fd25c108d30 2024-09-21T01:55:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.424+0000 7fd259ffb640 1 -- 172.21.15.29:0/4000915014 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd254003b70 con 0x7fd25c108d30 2024-09-21T01:55:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.424+0000 7fd259ffb640 1 -- 172.21.15.29:0/4000915014 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540099e0 con 0x7fd25c108d30 2024-09-21T01:55:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.424+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c1ace90 con 0x7fd25c108d30 2024-09-21T01:55:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.425+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c1ad230 con 0x7fd25c108d30 2024-09-21T01:55:23.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.429+0000 7fd269240740 1 Processor -- start 2024-09-21T01:55:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.429+0000 7fd259ffb640 1 -- 172.21.15.29:0/4000915014 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd254009b80 con 0x7fd25c108d30 2024-09-21T01:55:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.433+0000 7fd259ffb640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 0x7fd24c0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.434+0000 7fd259ffb640 1 -- 172.21.15.29:0/4000915014 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd254107c30 con 0x7fd25c108d30 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 -- start start 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b0239b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b0239c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b02311ea0 con 0x555b01ff4070 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b023012d0 con 0x555b01fd3390 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd269240740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b023050e0 con 0x555b01fcb1b0 2024-09-21T01:55:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.438+0000 7fd25bfff640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 0x7fd24c0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.439+0000 7fd260a68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b0239b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.439+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.439+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57146/0 (socket says 172.21.15.29:57146) 2024-09-21T01:55:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.439+0000 7fd25bfff640 1 -- 172.21.15.29:0/2714263198 learned_addr learned my addr 172.21.15.29:0/2714263198 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.441+0000 7fd25bfff640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 0x7fd24c0b8b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd248009fd0 tx=0x7fd248009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.442+0000 7fd25bfff640 1 -- 172.21.15.29:0/2714263198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 msgr2=0x555b0239c120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.442+0000 7fd25bfff640 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b0239c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.442+0000 7fd25bfff640 1 -- 172.21.15.29:0/2714263198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 msgr2=0x555b0239b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.442+0000 7fd25bfff640 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b0239b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.442+0000 7fd25bfff640 1 -- 172.21.15.29:0/2714263198 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0205f650 con 0x555b01fcb1b0 2024-09-21T01:55:23.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.446+0000 7fd25bfff640 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2480157c0 tx=0x7fd24801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05aed36e1af2d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.447+0000 7fd246ffd640 1 -- 172.21.15.29:0/2714263198 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248027070 con 0x555b01fcb1b0 2024-09-21T01:55:23.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.447+0000 7fd246ffd640 1 -- 172.21.15.29:0/2714263198 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd248018cb0 con 0x555b01fcb1b0 2024-09-21T01:55:23.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.447+0000 7fd246ffd640 1 -- 172.21.15.29:0/2714263198 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24801e070 con 0x555b01fcb1b0 2024-09-21T01:55:23.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 -- 172.21.15.29:0/2714263198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 msgr2=0x555b0239bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2480157c0 tx=0x7fd24801c000 comp rx=0 tx=0).stop 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 -- 172.21.15.29:0/2714263198 shutdown_connections 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b0239c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b0239bbe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 --2- 172.21.15.29:0/2714263198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b0239b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.449+0000 7fd269240740 1 -- 172.21.15.29:0/2714263198 >> 172.21.15.29:0/2714263198 conn(0x555b01fe8460 msgr2=0x555b02399590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.450+0000 7fd269240740 1 -- 172.21.15.29:0/2714263198 shutdown_connections 2024-09-21T01:55:23.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.450+0000 7fd269240740 1 -- 172.21.15.29:0/2714263198 wait complete. 2024-09-21T01:55:23.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.450+0000 7fd269240740 1 -- start start 2024-09-21T01:55:23.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.450+0000 7fd269240740 1 Processor -- start 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 0x555b023a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 0x555b023c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b02162700 con 0x555b01fcb1b0 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b0229c160 con 0x555b01ff4070 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b023bae30 con 0x555b01fd3390 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.451+0000 7fd269240740 10 client.0 _ll_register_callbacks cb 0x555b022b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:23.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd260a68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 0x555b023a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd260a68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 0x555b023a6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50300/0 (socket says 172.21.15.29:50300) 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd260a68640 1 -- 172.21.15.29:0/2056589282 learned_addr learned my addr 172.21.15.29:0/2056589282 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 0x555b023c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 -- 172.21.15.29:0/2056589282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 msgr2=0x555b023a73e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023a73e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 -- 172.21.15.29:0/2056589282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 msgr2=0x555b023a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 0x555b023a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 -- 172.21.15.29:0/2056589282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b022c9c90 con 0x555b01ff4070 2024-09-21T01:55:23.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.452+0000 7fd261269640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 0x555b023c4150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd254105790 tx=0x7fd2540cd720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b073ceb7c444ab60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd23bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540cd8e0 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd2540cd030 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248018cb0 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd2540ce4d0 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd269240740 10 client.26645 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.453+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555b023c6c60 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd248018cb0 con 0x555b01ff4070 2024-09-21T01:55:23.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd25403b030 con 0x555b01ff4070 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd269240740 10 client.26645 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd269240740 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd269240740 20 client.26645 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.454+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01fb29f0 0x555b023ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.455+0000 7fd269240740 4 client.26645 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.455+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b02313df0 con 0x555b01fb29f0 2024-09-21T01:55:23.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.455+0000 7fd260a68640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01fb29f0 0x555b023ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.456+0000 7fd260a68640 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01fb29f0 0x555b023ca2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53998eb43e6afa07 server_cookie=51ff69a8e6e7608f in_seq=0 out_seq=0 2024-09-21T01:55:23.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.457+0000 7fd23bfff640 10 client.26645 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:23.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.458+0000 7fd23bfff640 1 -- 172.21.15.29:0/2056589282 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b02313df0 con 0x555b01fb29f0 2024-09-21T01:55:23.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.458+0000 7fd23bfff640 10 client.26645 handle_command_reply: tid=0 2024-09-21T01:55:23.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.458+0000 7fd269240740 1 client.26645 shutdown 2024-09-21T01:55:23.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 msgr2=0x555b023c4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 0x555b023c4150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd254105790 tx=0x7fd2540cd720 comp rx=0 tx=0).stop 2024-09-21T01:55:23.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 shutdown_connections 2024-09-21T01:55:23.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01fb29f0 0x555b023ca2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01ff4070 0x555b023c4150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd23bfff640 0 client.26645 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 --2- 172.21.15.29:0/2056589282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01fcb1b0 0x555b023a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.459+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 >> 172.21.15.29:0/2056589282 conn(0x555b01fe8460 msgr2=0x555b0239e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.460+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 shutdown_connections 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.460+0000 7fd269240740 1 -- 172.21.15.29:0/2056589282 wait complete. 2024-09-21T01:55:23.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.460+0000 7fd269240740 20 client.26645 trim_cache size 0 max 16384 2024-09-21T01:55:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 Processor -- start 2024-09-21T01:55:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 -- start start 2024-09-21T01:55:23.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 0x555b0239eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1a4970 con 0x555b01ff4070 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1aeda0 con 0x555b01fcb1b0 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd269240740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1ad900 con 0x555b01fd3390 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd261269640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd261269640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50314/0 (socket says 172.21.15.29:50314) 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd261269640 1 -- 172.21.15.29:0/3574089093 learned_addr learned my addr 172.21.15.29:0/3574089093 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd260a68640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 0x555b0239eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.480+0000 7fd25bfff640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 -- 172.21.15.29:0/3574089093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 msgr2=0x555b023c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 -- 172.21.15.29:0/3574089093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 msgr2=0x555b0239eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 0x555b0239eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 -- 172.21.15.29:0/3574089093 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b02399e80 con 0x555b01ff4070 2024-09-21T01:55:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd260a68640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 0x555b0239eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd261269640 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd25403c040 tx=0x7fd2540cd850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790631f92760f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.481+0000 7fd2467fc640 1 -- 172.21.15.29:0/3574089093 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540ca420 con 0x555b01ff4070 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.482+0000 7fd2467fc640 1 -- 172.21.15.29:0/3574089093 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2540cd8e0 con 0x555b01ff4070 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.482+0000 7fd2467fc640 1 -- 172.21.15.29:0/3574089093 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540f5340 con 0x555b01ff4070 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 -- 172.21.15.29:0/3574089093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 msgr2=0x555b023c4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd25403c040 tx=0x7fd2540cd850 comp rx=0 tx=0).stop 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 -- 172.21.15.29:0/3574089093 shutdown_connections 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b023c4590 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fd3390 0x555b023c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 --2- 172.21.15.29:0/3574089093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fcb1b0 0x555b0239eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 -- 172.21.15.29:0/3574089093 >> 172.21.15.29:0/3574089093 conn(0x555b01fe8460 msgr2=0x555b0239e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 -- 172.21.15.29:0/3574089093 shutdown_connections 2024-09-21T01:55:23.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.483+0000 7fd269240740 1 -- 172.21.15.29:0/3574089093 wait complete. 2024-09-21T01:55:23.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.484+0000 7fd269240740 1 -- start start 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.484+0000 7fd269240740 1 Processor -- start 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b023bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b02433cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1ae6d0 con 0x555b01ff4070 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1ae000 con 0x555b01fd3390 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c1a2950 con 0x555b01fcb1b0 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd260a68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd269240740 10 client.0 _ll_register_callbacks cb 0x7fd25400b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd261269640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b02433cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd260a68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57162/0 (socket says 172.21.15.29:57162) 2024-09-21T01:55:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd260a68640 1 -- 172.21.15.29:0/3441890609 learned_addr learned my addr 172.21.15.29:0/3441890609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:23.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.485+0000 7fd25bfff640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b023bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 -- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 msgr2=0x555b023bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b023bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 -- 172.21.15.29:0/3441890609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 msgr2=0x555b02433cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b02433cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 -- 172.21.15.29:0/3441890609 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b02434210 con 0x555b01fcb1b0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd260a68640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd250009e00 tx=0x7fd2500039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a227a30727e4688b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd23bfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250028060 con 0x555b01fcb1b0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.486+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd250010640 con 0x555b01fcb1b0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.487+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2540cd8e0 con 0x555b01fcb1b0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.487+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd250010860 con 0x555b01fcb1b0 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.487+0000 7fd269240740 10 client.26656 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.487+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555b024353f0 con 0x555b01fcb1b0 2024-09-21T01:55:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd250023410 con 0x555b01fcb1b0 2024-09-21T01:55:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fd25001d380 con 0x555b01fcb1b0 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 10 client.26656 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 20 client.26656 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01f87da0 0x555b02437b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 4 client.26656 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.488+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd2540cd8e0 con 0x555b01f87da0 2024-09-21T01:55:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.489+0000 7fd25bfff640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01f87da0 0x555b02437b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.489+0000 7fd23bfff640 10 client.26656 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd25bfff640 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01f87da0 0x555b02437b40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680ae585a69b5c0e server_cookie=83b1af127df5657a in_seq=0 out_seq=0 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd23bfff640 1 -- 172.21.15.29:0/3441890609 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd2540cd8e0 con 0x555b01f87da0 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd23bfff640 10 client.26656 handle_command_reply: tid=0 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 client.26656 shutdown 2024-09-21T01:55:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 msgr2=0x555b023bc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd250009e00 tx=0x7fd2500039c0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 shutdown_connections 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555b01f87da0 0x555b02437b40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b01ff4070 0x555b02433cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b01fd3390 0x555b023bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 --2- 172.21.15.29:0/3441890609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b01fcb1b0 0x555b023bc4a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 >> 172.21.15.29:0/3441890609 conn(0x555b01fe8460 msgr2=0x555b0239c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd23bfff640 0 client.26656 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 shutdown_connections 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 1 -- 172.21.15.29:0/3441890609 wait complete. 2024-09-21T01:55:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.490+0000 7fd269240740 20 client.26656 trim_cache size 0 max 16384 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 msgr2=0x7fd24c0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 0x7fd24c0b8b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd248009fd0 tx=0x7fd248009580 comp rx=0 tx=0).stop 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 msgr2=0x7fd25c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd254006010 tx=0x7fd25400b3e0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 shutdown_connections 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd24c0b6660 0x7fd24c0b8b70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25c108d30 0x7fd25c198890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd25c1083e0 0x7fd25c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 --2- 172.21.15.29:0/4000915014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd25c102aa0 0x7fd25c19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 >> 172.21.15.29:0/4000915014 conn(0x7fd25c079750 msgr2=0x7fd25c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 shutdown_connections 2024-09-21T01:55:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:23.491+0000 7fd262ccc640 1 -- 172.21.15.29:0/4000915014 wait complete. 2024-09-21T01:55:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:23 smithi190 ceph-mon[34368]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 MiB/s wr, 646 op/s 2024-09-21T01:55:24.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:23 smithi066 ceph-mon[35001]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 MiB/s wr, 646 op/s 2024-09-21T01:55:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:23 smithi029 ceph-mon[31228]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 MiB/s wr, 646 op/s 2024-09-21T01:55:25.277 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:55:25.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:55:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:25.973 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:25 smithi029 ceph-mon[31228]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-09-21T01:55:25.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:25 smithi190 ceph-mon[34368]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-09-21T01:55:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:25 smithi066 ceph-mon[35001]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 -- 172.21.15.29:0/3267358621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09e5c0 msgr2=0x7f7b2c0a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 --2- 172.21.15.29:0/3267358621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c0a4df0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7b34066480 tx=0x7f7b34098170 comp rx=0 tx=0).stop 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 -- 172.21.15.29:0/3267358621 shutdown_connections 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 --2- 172.21.15.29:0/3267358621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c0a4df0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 --2- 172.21.15.29:0/3267358621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09dc70 0x7f7b2c09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 --2- 172.21.15.29:0/3267358621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c098740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 -- 172.21.15.29:0/3267358621 >> 172.21.15.29:0/3267358621 conn(0x7f7b2c010080 msgr2=0x7f7b2c010490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 -- 172.21.15.29:0/3267358621 shutdown_connections 2024-09-21T01:55:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.269+0000 7f7b38e71640 1 -- 172.21.15.29:0/3267358621 wait complete. 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.270+0000 7f7b38e71640 1 Processor -- start 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.270+0000 7f7b38e71640 1 -- start start 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.270+0000 7f7b38e71640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.270+0000 7f7b38e71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09dc70 0x7f7b2c12f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b38e71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c134440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b38e71640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0a81b0 con 0x7f7b2c09dc70 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b38e71640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0a8030 con 0x7f7b2c09e5c0 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b38e71640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0a8330 con 0x7f7b2c098330 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57176/0 (socket says 172.21.15.29:57176) 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 -- 172.21.15.29:0/29619231 learned_addr learned my addr 172.21.15.29:0/29619231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09dc70 0x7f7b2c12f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b33fff640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 -- 172.21.15.29:0/29619231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 msgr2=0x7f7b2c134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 -- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09dc70 msgr2=0x7f7b2c12f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09dc70 0x7f7b2c12f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 -- 172.21.15.29:0/29619231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c134b70 con 0x7f7b2c098330 2024-09-21T01:55:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b33fff640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.271+0000 7f7b337fe640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400e9f0 tx=0x7f7b2400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.272+0000 7f7b30ff9640 1 -- 172.21.15.29:0/29619231 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400cd80 con 0x7f7b2c098330 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.272+0000 7f7b30ff9640 1 -- 172.21.15.29:0/29619231 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b240054d0 con 0x7f7b2c098330 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.273+0000 7f7b30ff9640 1 -- 172.21.15.29:0/29619231 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240186d0 con 0x7f7b2c098330 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.274+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c134e60 con 0x7f7b2c098330 2024-09-21T01:55:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.274+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c13b5c0 con 0x7f7b2c098330 2024-09-21T01:55:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 Processor -- start 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 -- start start 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b30ff9640 1 -- 172.21.15.29:0/29619231 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7b24004680 con 0x7f7b2c098330 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd7237e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7238360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fd71a5e60 con 0x560fd6e5f490 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560fd7195290 con 0x560fd6e67670 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.277+0000 7f7b40647740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560fd71990a0 con 0x560fd6e88350 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7238360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57314/0 (socket says 172.21.15.29:57314) 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3751758362 learned_addr learned my addr 172.21.15.29:0/3751758362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b33fff640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd7237e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3751758362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 msgr2=0x560fd7237e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd7237e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3751758362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 msgr2=0x560fd7238360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7238360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3751758362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd6ef3930 con 0x560fd6e67670 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.278+0000 7f7b337fe640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7238360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b33fff640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd7237e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009c20 tx=0x7f7b28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7417bd5734c2411f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b30ff9640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 0x7f7b1c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b19ffb640 1 -- 172.21.15.29:0/3751758362 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004230 con 0x560fd6e67670 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b19ffb640 1 -- 172.21.15.29:0/3751758362 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b280043d0 con 0x560fd6e67670 2024-09-21T01:55:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b19ffb640 1 -- 172.21.15.29:0/3751758362 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004d80 con 0x560fd6e67670 2024-09-21T01:55:26.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b33fff640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 0x7f7b1c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.279+0000 7f7b30ff9640 1 -- 172.21.15.29:0/29619231 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7b240e71e0 con 0x7f7b2c098330 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b40647740 1 -- 172.21.15.29:0/3751758362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 msgr2=0x560fd72388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b33fff640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 0x7f7b1c0b8a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b34052910 tx=0x7f7b340a3040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b40647740 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009c20 tx=0x7f7b28011810 comp rx=0 tx=0).stop 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b40647740 1 -- 172.21.15.29:0/3751758362 shutdown_connections 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b40647740 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd72388a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.280+0000 7f7b40647740 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7238360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 --2- 172.21.15.29:0/3751758362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd7237e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 -- 172.21.15.29:0/3751758362 >> 172.21.15.29:0/3751758362 conn(0x560fd6e7c740 msgr2=0x560fd7235d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 -- 172.21.15.29:0/3751758362 shutdown_connections 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 -- 172.21.15.29:0/3751758362 wait complete. 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 -- start start 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.281+0000 7f7b40647740 1 Processor -- start 2024-09-21T01:55:26.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7243a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd7243fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd724f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fd6ff6890 con 0x560fd6e5f490 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560fd712fd40 con 0x560fd6e67670 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560fd7246600 con 0x560fd6e88350 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b40647740 10 client.0 _ll_register_callbacks cb 0x560fd7193c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b33fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd724f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b32ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd7243fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b33fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd724f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57204/0 (socket says 172.21.15.29:57204) 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.282+0000 7f7b33fff640 1 -- 172.21.15.29:0/1872772239 learned_addr learned my addr 172.21.15.29:0/1872772239 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.283+0000 7f7b32ffd640 1 -- 172.21.15.29:0/1872772239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 msgr2=0x560fd724f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.283+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd724f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.283+0000 7f7b32ffd640 1 -- 172.21.15.29:0/1872772239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 msgr2=0x560fd7243a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.283+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7243a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.283+0000 7f7b32ffd640 1 -- 172.21.15.29:0/1872772239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd724ff40 con 0x560fd6e67670 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.284+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd7243fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b28000c00 tx=0x7f7b28019c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b51fb443947c183 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.284+0000 7f7b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.285+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28028020 con 0x560fd6e67670 2024-09-21T01:55:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.285+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7b28024620 con 0x560fd6e67670 2024-09-21T01:55:26.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.285+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b280043d0 con 0x560fd6e67670 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.285+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7b28009e90 con 0x560fd6e67670 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.285+0000 7f7b40647740 10 client.26663 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.286+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560fd72527e0 con 0x560fd6e67670 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.286+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2801f040 con 0x560fd6e67670 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f7b28018070 con 0x560fd6e67670 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 10 client.26663 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 20 client.26663 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e46cd0 0x560fd7255b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 4 client.26663 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b280043d0 con 0x560fd6e46cd0 2024-09-21T01:55:26.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.287+0000 7f7b337fe640 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e46cd0 0x560fd7255b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.288+0000 7f7b337fe640 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e46cd0 0x560fd7255b40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6935ea7500c2d73 server_cookie=f7c6ed526af5a1b in_seq=0 out_seq=0 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.288+0000 7f7b12ffd640 10 client.26663 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.288+0000 7f7b12ffd640 1 -- 172.21.15.29:0/1872772239 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7b280043d0 con 0x560fd6e46cd0 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.288+0000 7f7b12ffd640 10 client.26663 handle_command_reply: tid=0 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.288+0000 7f7b40647740 1 client.26663 shutdown 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 msgr2=0x560fd7243fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd7243fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b28000c00 tx=0x7f7b28019c20 comp rx=0 tx=0).stop 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 shutdown_connections 2024-09-21T01:55:26.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e46cd0 0x560fd7255b40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e88350 0x560fd724f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e67670 0x560fd7243fc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b12ffd640 0 client.26663 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 --2- 172.21.15.29:0/1872772239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd7243a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 >> 172.21.15.29:0/1872772239 conn(0x560fd6e7c740 msgr2=0x560fd723afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 shutdown_connections 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 1 -- 172.21.15.29:0/1872772239 wait complete. 2024-09-21T01:55:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.289+0000 7f7b40647740 20 client.26663 trim_cache size 0 max 16384 2024-09-21T01:55:26.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 Processor -- start 2024-09-21T01:55:26.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 -- start start 2024-09-21T01:55:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e5f490 0x560fd724f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd72355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c13f4b0 con 0x560fd6e46cd0 2024-09-21T01:55:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c141020 con 0x560fd6e5f490 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b40647740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c13fb80 con 0x560fd6e67670 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50368/0 (socket says 172.21.15.29:50368) 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b337fe640 1 -- 172.21.15.29:0/1055392766 learned_addr learned my addr 172.21.15.29:0/1055392766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.314+0000 7f7b33fff640 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd72355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e5f490 0x560fd724f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 -- 172.21.15.29:0/1055392766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 msgr2=0x560fd72355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd72355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 -- 172.21.15.29:0/1055392766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e5f490 msgr2=0x560fd724f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e5f490 0x560fd724f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 -- 172.21.15.29:0/1055392766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd72427c0 con 0x560fd6e46cd0 2024-09-21T01:55:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b337fe640 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7b240aa6a0 tx=0x7f7b24004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5434d6116106c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b197fa640 1 -- 172.21.15.29:0/1055392766 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2401d040 con 0x560fd6e46cd0 2024-09-21T01:55:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b197fa640 1 -- 172.21.15.29:0/1055392766 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b24005680 con 0x560fd6e46cd0 2024-09-21T01:55:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.315+0000 7f7b197fa640 1 -- 172.21.15.29:0/1055392766 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240a8fe0 con 0x560fd6e46cd0 2024-09-21T01:55:26.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.316+0000 7f7b40647740 1 -- 172.21.15.29:0/1055392766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 msgr2=0x560fd723d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.316+0000 7f7b40647740 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7b240aa6a0 tx=0x7f7b24004a00 comp rx=0 tx=0).stop 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 -- 172.21.15.29:0/1055392766 shutdown_connections 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd72355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e5f490 0x560fd724f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 --2- 172.21.15.29:0/1055392766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e46cd0 0x560fd723d640 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 -- 172.21.15.29:0/1055392766 >> 172.21.15.29:0/1055392766 conn(0x560fd6e7c740 msgr2=0x560fd723cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.318+0000 7f7b40647740 1 -- 172.21.15.29:0/1055392766 shutdown_connections 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.319+0000 7f7b40647740 1 -- 172.21.15.29:0/1055392766 wait complete. 2024-09-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.319+0000 7f7b40647740 1 -- start start 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 Processor -- start 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e46cd0 0x560fd724bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd7246960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c140950 con 0x560fd6e5f490 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c140280 con 0x560fd6e46cd0 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c13d4f0 con 0x560fd6e67670 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b40647740 10 client.0 _ll_register_callbacks cb 0x7f7b28009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b32ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.320+0000 7f7b32ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50382/0 (socket says 172.21.15.29:50382) 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3091698048 learned_addr learned my addr 172.21.15.29:0/3091698048 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b337fe640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e46cd0 0x560fd724bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b33fff640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd7246960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3091698048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 msgr2=0x560fd7246960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd7246960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e46cd0 msgr2=0x560fd724bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e46cd0 0x560fd724bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 -- 172.21.15.29:0/3091698048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd72470f0 con 0x560fd6e5f490 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b32ffd640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x560fd723ad40 tx=0x7f7b28009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be32cb9278dda022 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.321+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28009e90 con 0x560fd6e5f490 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7b28019770 con 0x560fd6e5f490 2024-09-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b24005680 con 0x560fd6e5f490 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7b28005c50 con 0x560fd6e5f490 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 10 client.16911 fetch_fsmap learned FSMap version 40 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560fd72c0eb0 con 0x560fd6e5f490 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b28018360 con 0x560fd6e5f490 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f7b28018040 con 0x560fd6e5f490 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 10 client.16911 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.322+0000 7f7b40647740 20 client.16911 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e88350 0x560fd72c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b40647740 4 client.16911 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7b24005680 con 0x560fd6e88350 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b337fe640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e88350 0x560fd72c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b12ffd640 10 client.16911 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.323+0000 7f7b337fe640 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e88350 0x560fd72c3720 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4daa4c7d185f6965 server_cookie=114370e9004631b6 in_seq=0 out_seq=0 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.326+0000 7f7b12ffd640 1 -- 172.21.15.29:0/3091698048 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7b24005680 con 0x560fd6e88350 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.326+0000 7f7b12ffd640 10 client.16911 handle_command_reply: tid=0 2024-09-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.326+0000 7f7b40647740 1 client.16911 shutdown 2024-09-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.326+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 msgr2=0x560fd724c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.326+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x560fd723ad40 tx=0x7f7b28009640 comp rx=0 tx=0).stop 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 shutdown_connections 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560fd6e88350 0x560fd72c3720 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fd6e67670 0x560fd7246960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fd6e5f490 0x560fd724c210 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 --2- 172.21.15.29:0/3091698048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fd6e46cd0 0x560fd724bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 >> 172.21.15.29:0/3091698048 conn(0x560fd6e7c740 msgr2=0x560fd7235560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.327+0000 7f7b12ffd640 0 client.16911 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.328+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 shutdown_connections 2024-09-21T01:55:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.328+0000 7f7b40647740 1 -- 172.21.15.29:0/3091698048 wait complete. 2024-09-21T01:55:26.329 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:55:26.329 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:55:26.330 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "cfaef6f1-6b63-4cba-acca-61015f7b83fc", 2024-09-21T01:55:26.330 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:55:26.330 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.328+0000 7f7b40647740 20 client.16911 trim_cache size 0 max 16384 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 msgr2=0x7f7b1c0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 0x7f7b1c0b8a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b34052910 tx=0x7f7b340a3040 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 msgr2=0x7f7b2c12ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400e9f0 tx=0x7f7b2400edf0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 shutdown_connections 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7b1c0b6570 0x7f7b1c0b8a80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b2c09e5c0 0x7f7b2c134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b2c09dc70 0x7f7b2c12f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 --2- 172.21.15.29:0/29619231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c098330 0x7f7b2c12ec00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 >> 172.21.15.29:0/29619231 conn(0x7f7b2c010080 msgr2=0x7f7b2c09ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 shutdown_connections 2024-09-21T01:55:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:26.329+0000 7f7b38e71640 1 -- 172.21.15.29:0/29619231 wait complete. 2024-09-21T01:55:26.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:26 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:26 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:26 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 -- 172.21.15.29:0/612832254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5141050f0 msgr2=0x7fe51410ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/612832254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5141050f0 0x7fe51410ba90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009be0 tx=0x7fe50c02a6d0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 -- 172.21.15.29:0/612832254 shutdown_connections 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/612832254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5141050f0 0x7fe51410ba90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/612832254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe514104750 0x7fe514104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/612832254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141036d0 0x7fe514103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 -- 172.21.15.29:0/612832254 >> 172.21.15.29:0/612832254 conn(0x7fe5140ff030 msgr2=0x7fe5141014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 -- 172.21.15.29:0/612832254 shutdown_connections 2024-09-21T01:55:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.376+0000 7fe51b3e6640 1 -- 172.21.15.29:0/612832254 wait complete. 2024-09-21T01:55:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.377+0000 7fe51b3e6640 1 Processor -- start 2024-09-21T01:55:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 -- start start 2024-09-21T01:55:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5141036d0 0x7fe514197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 0x7fe514198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 0x7fe514198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410f300 con 0x7fe5141036d0 2024-09-21T01:55:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410f180 con 0x7fe5141050f0 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe51b3e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410f480 con 0x7fe514104750 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe519983640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 0x7fe514198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe518981640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 0x7fe514198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.378+0000 7fe519983640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 0x7fe514198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57356/0 (socket says 172.21.15.29:57356) 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe519983640 1 -- 172.21.15.29:0/957651409 learned_addr learned my addr 172.21.15.29:0/957651409 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe519182640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5141036d0 0x7fe514197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 -- 172.21.15.29:0/957651409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 msgr2=0x7fe514198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 0x7fe514198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 -- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5141036d0 msgr2=0x7fe514197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5141036d0 0x7fe514197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 -- 172.21.15.29:0/957651409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe514199710 con 0x7fe514104750 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.379+0000 7fe518981640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 0x7fe514198340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e790 tx=0x7fe50400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.380+0000 7fe50a7fc640 1 -- 172.21.15.29:0/957651409 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50400eea0 con 0x7fe514104750 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.380+0000 7fe50a7fc640 1 -- 172.21.15.29:0/957651409 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50400ccc0 con 0x7fe514104750 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.380+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe514199a00 con 0x7fe514104750 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.380+0000 7fe50a7fc640 1 -- 172.21.15.29:0/957651409 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504018680 con 0x7fe514104750 2024-09-21T01:55:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.380+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51419c0d0 con 0x7fe514104750 2024-09-21T01:55:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.384+0000 7fe50a7fc640 1 -- 172.21.15.29:0/957651409 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe504010430 con 0x7fe514104750 2024-09-21T01:55:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.384+0000 7fe52242e740 1 Processor -- start 2024-09-21T01:55:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.384+0000 7fe52242e740 1 -- start start 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 0x5561c2253e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 0x5561c22544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c2255010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5561c21c66f0 con 0x5561c1eac1b0 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561c21b5b20 con 0x5561c1e8b4d0 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe52242e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561c21b9930 con 0x5561c1e832f0 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe519182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 0x5561c22544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe519983640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 0x5561c2253e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe519983640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 0x5561c2253e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50408/0 (socket says 172.21.15.29:50408) 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe519983640 1 -- 172.21.15.29:0/1368014093 learned_addr learned my addr 172.21.15.29:0/1368014093 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.385+0000 7fe518981640 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c2255010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe518981640 1 -- 172.21.15.29:0/1368014093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 msgr2=0x5561c22544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe518981640 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 0x5561c22544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe518981640 1 -- 172.21.15.29:0/1368014093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 msgr2=0x5561c2253e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe518981640 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 0x5561c2253e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe518981640 1 -- 172.21.15.29:0/1368014093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c217b530 con 0x5561c1e8b4d0 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.386+0000 7fe519182640 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 0x5561c22544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.387+0000 7fe518981640 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c2255010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5040e4e20 tx=0x7fe5040ef9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c628dddd315b94a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.387+0000 7fe4ff7fe640 1 -- 172.21.15.29:0/1368014093 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040fb070 con 0x5561c1e8b4d0 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.387+0000 7fe4ff7fe640 1 -- 172.21.15.29:0/1368014093 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5040e94e0 con 0x5561c1e8b4d0 2024-09-21T01:55:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.387+0000 7fe4ff7fe640 1 -- 172.21.15.29:0/1368014093 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040f3a10 con 0x5561c1e8b4d0 2024-09-21T01:55:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.387+0000 7fe50a7fc640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 0x7fe5000b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.388+0000 7fe50a7fc640 1 -- 172.21.15.29:0/957651409 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe5040e73d0 con 0x7fe514104750 2024-09-21T01:55:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.388+0000 7fe52242e740 1 -- 172.21.15.29:0/1368014093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 msgr2=0x5561c2255010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.388+0000 7fe52242e740 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c2255010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5040e4e20 tx=0x7fe5040ef9b0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.388+0000 7fe519983640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 0x7fe5000b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 -- 172.21.15.29:0/1368014093 shutdown_connections 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c2255010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e832f0 0x5561c22544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 --2- 172.21.15.29:0/1368014093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1eac1b0 0x5561c2253e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 -- 172.21.15.29:0/1368014093 >> 172.21.15.29:0/1368014093 conn(0x5561c1ea0300 msgr2=0x5561c224f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 -- 172.21.15.29:0/1368014093 shutdown_connections 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.389+0000 7fe52242e740 1 -- 172.21.15.29:0/1368014093 wait complete. 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.390+0000 7fe519983640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 0x7fe5000b8b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0099a0 tx=0x7fe50c030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.390+0000 7fe52242e740 1 -- start start 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.390+0000 7fe52242e740 1 Processor -- start 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe52242e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe52242e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 0x5561c225dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe52242e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 0x5561c225e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe518981640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 0x5561c225dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe518981640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 0x5561c225dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57238/0 (socket says 172.21.15.29:57238) 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe518981640 1 -- 172.21.15.29:0/4241235581 learned_addr learned my addr 172.21.15.29:0/4241235581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5561c20edab0 con 0x5561c1e832f0 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe519983640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 0x5561c225e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.391+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561c215b730 con 0x5561c1eac1b0 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.392+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561c2255b30 con 0x5561c1e8b4d0 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.392+0000 7fe52242e740 10 client.0 _ll_register_callbacks cb 0x5561c21af6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.392+0000 7fe519182640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 -- 172.21.15.29:0/4241235581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 msgr2=0x5561c225dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 0x5561c225dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 -- 172.21.15.29:0/4241235581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 msgr2=0x5561c225d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c1f17790 con 0x5561c1eac1b0 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe519983640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 0x5561c225e830 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009420 tx=0x7fe50c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c27a6e66ddf9741 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe4fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c04f070 con 0x5561c1eac1b0 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.393+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe50c004040 con 0x5561c1eac1b0 2024-09-21T01:55:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.394+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5040e94e0 con 0x5561c1eac1b0 2024-09-21T01:55:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.394+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe50c059050 con 0x5561c1eac1b0 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.394+0000 7fe52242e740 10 client.26681 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.394+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5561c1f90620 con 0x5561c1eac1b0 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c0046f0 con 0x5561c1eac1b0 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fe50c04b940 con 0x5561c1eac1b0 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 10 client.26681 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 20 client.26681 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e6ad80 0x5561c226ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 4 client.26681 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.395+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5040e94e0 con 0x5561c1e6ad80 2024-09-21T01:55:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.396+0000 7fe519182640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e6ad80 0x5561c226ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.397+0000 7fe4fcff9640 10 client.26681 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.397+0000 7fe519182640 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e6ad80 0x5561c226ad00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c419e9adf60ae6 server_cookie=5fd72576cc61b97 in_seq=0 out_seq=0 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe4fcff9640 1 -- 172.21.15.29:0/4241235581 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe5040e94e0 con 0x5561c1e6ad80 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe4fcff9640 10 client.26681 handle_command_reply: tid=0 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 client.26681 shutdown 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 msgr2=0x5561c225e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 0x5561c225e830 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009420 tx=0x7fe50c009500 comp rx=0 tx=0).stop 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 shutdown_connections 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e6ad80 0x5561c226ad00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1eac1b0 0x5561c225e830 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1e8b4d0 0x5561c225dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 --2- 172.21.15.29:0/4241235581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 >> 172.21.15.29:0/4241235581 conn(0x5561c1ea0300 msgr2=0x5561c224dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.398+0000 7fe4fcff9640 0 client.26681 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.399+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 shutdown_connections 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.399+0000 7fe52242e740 1 -- 172.21.15.29:0/4241235581 wait complete. 2024-09-21T01:55:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.399+0000 7fe52242e740 20 client.26681 trim_cache size 0 max 16384 2024-09-21T01:55:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.422+0000 7fe52242e740 1 Processor -- start 2024-09-21T01:55:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.422+0000 7fe52242e740 1 -- start start 2024-09-21T01:55:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe52242e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe52242e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c224dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe52242e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c224f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe52242e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141b9dc0 con 0x5561c1e832f0 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe518981640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c224dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe518981640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c224dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57372/0 (socket says 172.21.15.29:57372) 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe518981640 1 -- 172.21.15.29:0/805060273 learned_addr learned my addr 172.21.15.29:0/805060273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe519983640 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c224f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.423+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141bb900 con 0x5561c1e8b4d0 2024-09-21T01:55:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141ba490 con 0x5561c1eac1b0 2024-09-21T01:55:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 -- 172.21.15.29:0/805060273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 msgr2=0x5561c224dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c224dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 -- 172.21.15.29:0/805060273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 msgr2=0x5561c225ee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:55:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 -- 172.21.15.29:0/805060273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c225e030 con 0x5561c1eac1b0 2024-09-21T01:55:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.424+0000 7fe519983640 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c224f290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe50c04bf00 tx=0x7fe50c030700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ebcfe8538e054af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.425+0000 7fe4feffd640 1 -- 172.21.15.29:0/805060273 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c04f040 con 0x5561c1eac1b0 2024-09-21T01:55:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.425+0000 7fe4feffd640 1 -- 172.21.15.29:0/805060273 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c062020 con 0x5561c1eac1b0 2024-09-21T01:55:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.425+0000 7fe4feffd640 1 -- 172.21.15.29:0/805060273 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c036040 con 0x5561c1eac1b0 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 msgr2=0x5561c224f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c224f290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe50c04bf00 tx=0x7fe50c030700 comp rx=0 tx=0).stop 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 shutdown_connections 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c224f290 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e8b4d0 0x5561c224dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 --2- 172.21.15.29:0/805060273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e832f0 0x5561c225ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.426+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 >> 172.21.15.29:0/805060273 conn(0x5561c1ea0300 msgr2=0x5561c226ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.427+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 shutdown_connections 2024-09-21T01:55:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.427+0000 7fe52242e740 1 -- 172.21.15.29:0/805060273 wait complete. 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.427+0000 7fe52242e740 1 -- start start 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.427+0000 7fe52242e740 1 Processor -- start 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 0x5561c2260b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c2261080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe518981640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 0x5561c2260b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57376/0 (socket says 172.21.15.29:57376) 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe518981640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 0x5561c2260b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50426/0 (socket says 172.21.15.29:50426) 2024-09-21T01:55:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 -- 172.21.15.29:0/3077246306 learned_addr learned my addr 172.21.15.29:0/3077246306 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141bb230 con 0x5561c1e8b4d0 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141bab60 con 0x5561c1e832f0 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141b7da0 con 0x5561c1eac1b0 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe52242e740 10 client.0 _ll_register_callbacks cb 0x7fe510005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519983640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c2261080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 -- 172.21.15.29:0/3077246306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 msgr2=0x5561c2261080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c2261080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 -- 172.21.15.29:0/3077246306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 msgr2=0x5561c2260b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 0x5561c2260b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.428+0000 7fe519182640 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c2261670 con 0x5561c1e832f0 2024-09-21T01:55:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.429+0000 7fe519182640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe510009fd0 tx=0x7fe510004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aef52bf2086fbac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.429+0000 7fe4fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.429+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510025440 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.429+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe510010430 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.429+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c062020 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe510010650 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 10 client.26687 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5561c22d5840 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510010ca0 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fe510023070 con 0x5561c1e832f0 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 10 client.26687 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 10 client.26687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 20 client.26687 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.430+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e401c0 0x5561c22d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.431+0000 7fe52242e740 4 client.26687 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.431+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe50c062020 con 0x5561c1e401c0 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.431+0000 7fe518981640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e401c0 0x5561c22d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.431+0000 7fe4fcff9640 10 client.26687 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.432+0000 7fe518981640 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e401c0 0x5561c22d8260 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501605536b189d62 server_cookie=9e17bac557717b4c in_seq=0 out_seq=0 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.444+0000 7fe4fcff9640 1 -- 172.21.15.29:0/3077246306 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe50c062020 con 0x5561c1e401c0 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.444+0000 7fe4fcff9640 10 client.26687 handle_command_reply: tid=0 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.444+0000 7fe52242e740 1 client.26687 shutdown 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 msgr2=0x5561c2264d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe510009fd0 tx=0x7fe510004410 comp rx=0 tx=0).stop 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 shutdown_connections 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561c1e401c0 0x5561c22d8260 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561c1eac1b0 0x5561c2261080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe4fcff9640 0 client.26687 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561c1e8b4d0 0x5561c2260b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 --2- 172.21.15.29:0/3077246306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561c1e832f0 0x5561c2264d30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 >> 172.21.15.29:0/3077246306 conn(0x5561c1ea0300 msgr2=0x5561c226ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 shutdown_connections 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 1 -- 172.21.15.29:0/3077246306 wait complete. 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe52242e740 20 client.26687 trim_cache size 0 max 16384 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:55:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 msgr2=0x7fe5000b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 0x7fe5000b8b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0099a0 tx=0x7fe50c030970 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 msgr2=0x7fe514198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.445+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 0x7fe514198340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e790 tx=0x7fe50400ec50 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 shutdown_connections 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe5000b6660 0x7fe5000b8b70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5141050f0 0x7fe514198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe514104750 0x7fe514198340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 --2- 172.21.15.29:0/957651409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5141036d0 0x7fe514197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 >> 172.21.15.29:0/957651409 conn(0x7fe5140ff030 msgr2=0x7fe514105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 shutdown_connections 2024-09-21T01:55:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:27.446+0000 7fe51b3e6640 1 -- 172.21.15.29:0/957651409 wait complete. 2024-09-21T01:55:27.640 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.965 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.966 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:27.966 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.966 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:27 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 410.0 s, 2837.0 tps, lat 34.883 ms stddev 41.044 2024-09-21T01:55:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-09-21T01:55:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:55:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:27.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:27 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-09-21T01:55:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:55:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:27 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:55:28.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 -- 172.21.15.29:0/1198741943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc1083a0 msgr2=0x7f9ddc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 --2- 172.21.15.29:0/1198741943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc1087b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009900 tx=0x7f9dcc02a420 comp rx=0 tx=0).stop 2024-09-21T01:55:28.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 -- 172.21.15.29:0/1198741943 shutdown_connections 2024-09-21T01:55:28.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 --2- 172.21.15.29:0/1198741943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 --2- 172.21.15.29:0/1198741943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc1087b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 --2- 172.21.15.29:0/1198741943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc102a60 0x7f9ddc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 -- 172.21.15.29:0/1198741943 >> 172.21.15.29:0/1198741943 conn(0x7f9ddc079730 msgr2=0x7f9ddc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 -- 172.21.15.29:0/1198741943 shutdown_connections 2024-09-21T01:55:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.268+0000 7f9de34e1640 1 -- 172.21.15.29:0/1198741943 wait complete. 2024-09-21T01:55:28.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.269+0000 7f9de34e1640 1 Processor -- start 2024-09-21T01:55:28.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.269+0000 7f9de34e1640 1 -- start start 2024-09-21T01:55:28.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1118c0 con 0x7f9ddc102a60 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc111740 con 0x7f9ddc108cf0 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de34e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc111a40 con 0x7f9ddc1083a0 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de127d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de127d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50452/0 (socket says 172.21.15.29:50452) 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de127d640 1 -- 172.21.15.29:0/2772806292 learned_addr learned my addr 172.21.15.29:0/2772806292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.270+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 -- 172.21.15.29:0/2772806292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc1083a0 msgr2=0x7f9ddc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 -- 172.21.15.29:0/2772806292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc108cf0 msgr2=0x7f9ddc1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 -- 172.21.15.29:0/2772806292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1a2670 con 0x7f9ddc102a60 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9de127d640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d950 tx=0x7f9dd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.271+0000 7f9dd27fc640 1 -- 172.21.15.29:0/2772806292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8017070 con 0x7f9ddc102a60 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.272+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1a2960 con 0x7f9ddc102a60 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.272+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a90c0 con 0x7f9ddc102a60 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.272+0000 7f9dd27fc640 1 -- 172.21.15.29:0/2772806292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd8009c30 con 0x7f9ddc102a60 2024-09-21T01:55:28.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.272+0000 7f9dd27fc640 1 -- 172.21.15.29:0/2772806292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800f620 con 0x7f9ddc102a60 2024-09-21T01:55:28.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.276+0000 7f9de9a55740 1 Processor -- start 2024-09-21T01:55:28.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.276+0000 7f9de9a55740 1 -- start start 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c61b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ade7bd7e50 con 0x55ade7899390 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ade7bc7280 con 0x55ade78ba070 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de9a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ade7bcb090 con 0x55ade78911b0 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de1a7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de1a7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c61650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57400/0 (socket says 172.21.15.29:57400) 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50468/0 (socket says 172.21.15.29:50468) 2024-09-21T01:55:28.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de1a7e640 1 -- 172.21.15.29:0/1240120503 learned_addr learned my addr 172.21.15.29:0/1240120503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 -- 172.21.15.29:0/1240120503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 msgr2=0x55ade7c61b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c61b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 -- 172.21.15.29:0/1240120503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 msgr2=0x55ade7c61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.277+0000 7f9de0a7c640 1 -- 172.21.15.29:0/1240120503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade7925650 con 0x55ade7899390 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.278+0000 7f9dd27fc640 1 -- 172.21.15.29:0/2772806292 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9dd800f7c0 con 0x7f9ddc102a60 2024-09-21T01:55:28.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.278+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006010 tx=0x7f9dcc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21716154afd2d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.278+0000 7f9dc77fe640 1 -- 172.21.15.29:0/1240120503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc03f070 con 0x55ade7899390 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.279+0000 7f9dc77fe640 1 -- 172.21.15.29:0/1240120503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc0316a0 con 0x55ade7899390 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.279+0000 7f9dc77fe640 1 -- 172.21.15.29:0/1240120503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc035070 con 0x55ade7899390 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.279+0000 7f9dd27fc640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 0x7f9dc80b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.279+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 0x7f9dc80b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.280+0000 7f9dd27fc640 1 -- 172.21.15.29:0/2772806292 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9dd80e6d40 con 0x7f9ddc102a60 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.280+0000 7f9de9a55740 1 -- 172.21.15.29:0/1240120503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 msgr2=0x55ade7c620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.280+0000 7f9de9a55740 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006010 tx=0x7f9dcc032ce0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.281+0000 7f9de9a55740 1 -- 172.21.15.29:0/1240120503 shutdown_connections 2024-09-21T01:55:28.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.281+0000 7f9de9a55740 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c620d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.281+0000 7f9de9a55740 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.281+0000 7f9de9a55740 1 --2- 172.21.15.29:0/1240120503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.282+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 0x7f9dc80b8a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009420 tx=0x7f9dd400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.282+0000 7f9de9a55740 1 -- 172.21.15.29:0/1240120503 >> 172.21.15.29:0/1240120503 conn(0x55ade78ae460 msgr2=0x55ade7c5f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.282+0000 7f9de9a55740 1 -- 172.21.15.29:0/1240120503 shutdown_connections 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.282+0000 7f9de9a55740 1 -- 172.21.15.29:0/1240120503 wait complete. 2024-09-21T01:55:28.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.282+0000 7f9de9a55740 1 -- start start 2024-09-21T01:55:28.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.283+0000 7f9de9a55740 1 Processor -- start 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 0x55ade7c6d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 0x55ade7c6d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de127d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de127d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57402/0 (socket says 172.21.15.29:57402) 2024-09-21T01:55:28.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de0a7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 0x55ade7c6d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de0a7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 0x55ade7c6d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57268/0 (socket says 172.21.15.29:57268) 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de1a7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 0x55ade7c6d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.284+0000 7f9de1a7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 0x55ade7c6d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50480/0 (socket says 172.21.15.29:50480) 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 -- 172.21.15.29:0/874267398 learned_addr learned my addr 172.21.15.29:0/874267398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ade7a28700 con 0x55ade78ba070 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ade7b62110 con 0x55ade78911b0 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ade7c6fdd0 con 0x55ade7899390 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de9a55740 10 client.0 _ll_register_callbacks cb 0x55ade7b7b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 -- 172.21.15.29:0/874267398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 msgr2=0x55ade7c6d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 0x55ade7c6d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 -- 172.21.15.29:0/874267398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 msgr2=0x55ade7c6d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 0x55ade7c6d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.285+0000 7f9de127d640 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade7b8fc40 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.286+0000 7f9de127d640 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ace10 tx=0x7f9dd80acd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7f6dc21e010107 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.286+0000 7f9dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8020070 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9dd80a87d0 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dcc0316a0 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9dd80ac990 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9de9a55740 10 client.26699 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ade7c7bb10 con 0x55ade78911b0 2024-09-21T01:55:28.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.287+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc0316a0 con 0x55ade78911b0 2024-09-21T01:55:28.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f9dd80b4ce0 con 0x55ade78911b0 2024-09-21T01:55:28.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 10 client.26699 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:28.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:28.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 20 client.26699 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:28.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade78789f0 0x55ade7c7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 4 client.26699 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:28.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.288+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ade7bd9da0 con 0x55ade78789f0 2024-09-21T01:55:28.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.289+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade78789f0 0x55ade7c7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.290+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade78789f0 0x55ade7c7efd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788046c9794f5698 server_cookie=f7458c0210c34e5a in_seq=0 out_seq=0 2024-09-21T01:55:28.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.290+0000 7f9dc4ff9640 10 client.26699 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:28.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/874267398 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ade7bd9da0 con 0x55ade78789f0 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9dc4ff9640 10 client.26699 handle_command_reply: tid=0 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 client.26699 shutdown 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 msgr2=0x55ade7c6ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ace10 tx=0x7f9dd80acd80 comp rx=0 tx=0).stop 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 shutdown_connections 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade78789f0 0x55ade7c7efd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade78ba070 0x55ade7c6d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade7899390 0x55ade7c6d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9dc4ff9640 0 client.26699 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 --2- 172.21.15.29:0/874267398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c6ccf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 >> 172.21.15.29:0/874267398 conn(0x55ade78ae460 msgr2=0x55ade7c647e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.291+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 shutdown_connections 2024-09-21T01:55:28.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.292+0000 7f9de9a55740 1 -- 172.21.15.29:0/874267398 wait complete. 2024-09-21T01:55:28.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.292+0000 7f9de9a55740 20 client.26699 trim_cache size 0 max 16384 2024-09-21T01:55:28.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.319+0000 7f9de9a55740 1 Processor -- start 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.319+0000 7f9de9a55740 1 -- start start 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c69ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000e4b0 con 0x55ade7899390 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000fff0 con 0x55ade78ba070 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de9a55740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000eb80 con 0x55ade78911b0 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de127d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de127d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c62d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57282/0 (socket says 172.21.15.29:57282) 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de1a7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de127d640 1 -- 172.21.15.29:0/3090642799 learned_addr learned my addr 172.21.15.29:0/3090642799 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c69ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de1a7e640 1 -- 172.21.15.29:0/3090642799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 msgr2=0x55ade7c62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de1a7e640 1 -- 172.21.15.29:0/3090642799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 msgr2=0x55ade7c69ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.320+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c69ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.321+0000 7f9de1a7e640 1 -- 172.21.15.29:0/3090642799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade7c6bc50 con 0x55ade78ba070 2024-09-21T01:55:28.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.321+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c6a420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bfd0 tx=0x7f9dd4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab65d4c12750ffca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.321+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/3090642799 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd401e040 con 0x55ade78ba070 2024-09-21T01:55:28.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.321+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/3090642799 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd400ae70 con 0x55ade78ba070 2024-09-21T01:55:28.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.321+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/3090642799 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4016430 con 0x55ade78ba070 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 -- 172.21.15.29:0/3090642799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 msgr2=0x55ade7c6a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c6a420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bfd0 tx=0x7f9dd4014000 comp rx=0 tx=0).stop 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 -- 172.21.15.29:0/3090642799 shutdown_connections 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78ba070 0x55ade7c6a420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3090642799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78911b0 0x55ade7c62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 -- 172.21.15.29:0/3090642799 >> 172.21.15.29:0/3090642799 conn(0x55ade78ae460 msgr2=0x55ade7c62680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.322+0000 7f9de9a55740 1 -- 172.21.15.29:0/3090642799 shutdown_connections 2024-09-21T01:55:28.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.323+0000 7f9de9a55740 1 -- 172.21.15.29:0/3090642799 wait complete. 2024-09-21T01:55:28.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.323+0000 7f9de9a55740 1 -- start start 2024-09-21T01:55:28.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.323+0000 7f9de9a55740 1 Processor -- start 2024-09-21T01:55:28.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de127d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de127d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57418/0 (socket says 172.21.15.29:57418) 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78ba070 0x55ade7cf11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de127d640 1 -- 172.21.15.29:0/3567696946 learned_addr learned my addr 172.21.15.29:0/3567696946 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:28.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000f920 con 0x55ade7899390 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000f250 con 0x55ade78911b0 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000c490 con 0x55ade78ba070 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de9a55740 10 client.0 _ll_register_callbacks cb 0x7f9dd80a8dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de1a7e640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78ba070 0x55ade7cf11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.324+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 -- 172.21.15.29:0/3567696946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78ba070 msgr2=0x55ade7cf11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78ba070 0x55ade7cf11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 -- 172.21.15.29:0/3567696946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 msgr2=0x55ade7c719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade7cf16e0 con 0x55ade78911b0 2024-09-21T01:55:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de127d640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80acb30 tx=0x7f9dd8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86a3daeac492bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8020070 con 0x55ade78911b0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9dd80d40c0 con 0x55ade78911b0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.325+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd400ae70 con 0x55ade78911b0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.326+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9dd80ad580 con 0x55ade78911b0 2024-09-21T01:55:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.326+0000 7f9de9a55740 10 client.26705 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.326+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ade7cf2820 con 0x55ade78911b0 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.326+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd8009de0 con 0x55ade78911b0 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f9dd80b4ce0 con 0x55ade78911b0 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 10 client.26705 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 20 client.26705 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade784dda0 0x55ade7cf4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 4 client.26705 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9dd400ae70 con 0x55ade784dda0 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.327+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade784dda0 0x55ade7cf4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:28.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.328+0000 7f9dc4ff9640 10 client.26705 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.328+0000 7f9de0a7c640 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade784dda0 0x55ade7cf4ec0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20febb10e35c2969 server_cookie=f83c38b2126f4a37 in_seq=0 out_seq=0 2024-09-21T01:55:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.328+0000 7f9dc4ff9640 1 -- 172.21.15.29:0/3567696946 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x7f9dd400ae70 con 0x55ade784dda0 2024-09-21T01:55:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.328+0000 7f9dc4ff9640 10 client.26705 handle_command_reply: tid=0 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.329+0000 7f9de9a55740 1 client.26705 shutdown 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 msgr2=0x55ade7c71450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80acb30 tx=0x7f9dd8005740 comp rx=0 tx=0).stop 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 shutdown_connections 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ade784dda0 0x55ade7cf4ec0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ade78ba070 0x55ade7cf11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ade7899390 0x55ade7c719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9dc4ff9640 0 client.26705 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 --2- 172.21.15.29:0/3567696946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ade78911b0 0x55ade7c71450 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 >> 172.21.15.29:0/3567696946 conn(0x55ade78ae460 msgr2=0x55ade7c6e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 shutdown_connections 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 1 -- 172.21.15.29:0/3567696946 wait complete. 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.330+0000 7f9de9a55740 20 client.26705 trim_cache size 0 max 16384 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:55:28.336 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "cfaef6f1-6b63-4cba-acca-61015f7b83fc", 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:55:28.337 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.339 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.340 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:55:28.342 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 msgr2=0x7f9dc80b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 0x7f9dc80b8a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009420 tx=0x7f9dd400d040 comp rx=0 tx=0).stop 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 msgr2=0x7f9ddc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d950 tx=0x7f9dd800de10 comp rx=0 tx=0).stop 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 shutdown_connections 2024-09-21T01:55:28.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dc80b6580 0x7f9dc80b8a90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 --2- 172.21.15.29:0/2772806292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ddc102a60 0x7f9ddc19c700 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 >> 172.21.15.29:0/2772806292 conn(0x7f9ddc079730 msgr2=0x7f9ddc105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 shutdown_connections 2024-09-21T01:55:28.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:55:28.331+0000 7f9de34e1640 1 -- 172.21.15.29:0/2772806292 wait complete. 2024-09-21T01:55:29.157 INFO:tasks.cephfs.filesystem:scrub status for tag:cfaef6f1-6b63-4cba-acca-61015f7b83fc - {'path': '/', 'tag': 'cfaef6f1-6b63-4cba-acca-61015f7b83fc', 'options': 'recursive,force'} 2024-09-21T01:55:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:29 smithi190 ceph-mon[34368]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-09-21T01:55:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:29 smithi066 ceph-mon[35001]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-09-21T01:55:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:29 smithi029 ceph-mon[31228]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-09-21T01:55:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:31 smithi190 ceph-mon[34368]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.2 MiB/s wr, 880 op/s 2024-09-21T01:55:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:31 smithi066 ceph-mon[35001]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.2 MiB/s wr, 880 op/s 2024-09-21T01:55:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:31 smithi029 ceph-mon[31228]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.2 MiB/s wr, 880 op/s 2024-09-21T01:55:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 415.0 s, 2324.8 tps, lat 43.085 ms stddev 46.903 2024-09-21T01:55:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:33 smithi190 ceph-mon[34368]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 767 op/s 2024-09-21T01:55:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:34.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:33 smithi066 ceph-mon[35001]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 767 op/s 2024-09-21T01:55:34.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:34.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:33 smithi029 ceph-mon[31228]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 767 op/s 2024-09-21T01:55:34.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:34.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:35.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:35.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:55:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:35.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:55:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-09-21T01:55:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: Adjusting osd_memory_target on smithi066 to 176.0M 2024-09-21T01:55:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-21T01:55:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: Unable to set osd_memory_target on smithi066 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T01:55:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.8 MiB/s wr, 759 op/s 2024-09-21T01:55:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:55:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:55:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: Adjusting osd_memory_target on smithi066 to 176.0M 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: Unable to set osd_memory_target on smithi066 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.8 MiB/s wr, 759 op/s 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:55:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:55:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-09-21T01:55:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: Adjusting osd_memory_target on smithi066 to 176.0M 2024-09-21T01:55:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-21T01:55:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: Unable to set osd_memory_target on smithi066 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T01:55:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.8 MiB/s wr, 759 op/s 2024-09-21T01:55:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:55:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:55:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:55:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:36 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:55:36.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:36 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:55:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:36 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:55:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:36 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:55:37.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:36 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:55:37.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:36 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:55:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 420.0 s, 1654.4 tps, lat 60.646 ms stddev 65.432 2024-09-21T01:55:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:37 smithi190 ceph-mon[34368]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.2 MiB/s wr, 815 op/s 2024-09-21T01:55:37.995 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:37 smithi029 ceph-mon[31228]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.2 MiB/s wr, 815 op/s 2024-09-21T01:55:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:37 smithi066 ceph-mon[35001]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.2 MiB/s wr, 815 op/s 2024-09-21T01:55:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:39 smithi190 ceph-mon[34368]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 681 op/s 2024-09-21T01:55:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:39 smithi066 ceph-mon[35001]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 681 op/s 2024-09-21T01:55:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:39 smithi029 ceph-mon[31228]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 681 op/s 2024-09-21T01:55:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.611+0000 7f0bca7fc640 1 -- 172.21.15.29:0/2417650491 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc402f310 con 0x7f0bcc10d8e0 2024-09-21T01:55:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.612+0000 7f0bd1430640 1 -- 172.21.15.29:0/2417650491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10d8e0 msgr2=0x7f0bcc114280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.612+0000 7f0bd1430640 1 --2- 172.21.15.29:0/2417650491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc114280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009ca0 tx=0x7f0bc402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 -- 172.21.15.29:0/2417650491 shutdown_connections 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 --2- 172.21.15.29:0/2417650491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc114280 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 --2- 172.21.15.29:0/2417650491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 0x7f0bcc10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 --2- 172.21.15.29:0/2417650491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 -- 172.21.15.29:0/2417650491 >> 172.21.15.29:0/2417650491 conn(0x7f0bcc072a40 msgr2=0x7f0bcc074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 -- 172.21.15.29:0/2417650491 shutdown_connections 2024-09-21T01:55:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 -- 172.21.15.29:0/2417650491 wait complete. 2024-09-21T01:55:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.613+0000 7f0bd1430640 1 Processor -- start 2024-09-21T01:55:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 -- start start 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 0x7f0bcc1a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc1a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36368/0 (socket says 172.21.15.29:36368) 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 -- 172.21.15.29:0/113193900 learned_addr learned my addr 172.21.15.29:0/113193900 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc117b90 con 0x7f0bcc10bec0 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc117a10 con 0x7f0bcc10cf40 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc117d10 con 0x7f0bcc10d8e0 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 0x7f0bcc1a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc1a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 -- 172.21.15.29:0/113193900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10d8e0 msgr2=0x7f0bcc1a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc1a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 -- 172.21.15.29:0/113193900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 msgr2=0x7f0bcc1a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 0x7f0bcc1a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.614+0000 7f0bcaffd640 1 -- 172.21.15.29:0/113193900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc1aa6f0 con 0x7f0bcc10bec0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.615+0000 7f0bcaffd640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00e9f0 tx=0x7f0bbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.615+0000 7f0bc8ff9640 1 -- 172.21.15.29:0/113193900 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00cd80 con 0x7f0bcc10bec0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.615+0000 7f0bc8ff9640 1 -- 172.21.15.29:0/113193900 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bbc0108e0 con 0x7f0bcc10bec0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.615+0000 7f0bc8ff9640 1 -- 172.21.15.29:0/113193900 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc018980 con 0x7f0bcc10bec0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.615+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1aa9e0 con 0x7f0bcc10bec0 2024-09-21T01:55:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.616+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1b10d0 con 0x7f0bcc10bec0 2024-09-21T01:55:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.619+0000 7f0bd847b740 1 Processor -- start 2024-09-21T01:55:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.620+0000 7f0bd847b740 1 -- start start 2024-09-21T01:55:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.620+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a547e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50b54d0 0x5558a547f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bd847b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558a53f06f0 con 0x5558a50b54d0 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bd847b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5558a53dfb20 con 0x5558a50ad2f0 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bd847b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5558a53e3930 con 0x5558a50d61b0 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50b54d0 0x5558a547f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41548/0 (socket says 172.21.15.29:41548) 2024-09-21T01:55:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.621+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/1923776716 learned_addr learned my addr 172.21.15.29:0/1923776716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/1923776716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 msgr2=0x5558a547e4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a547e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/1923776716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50b54d0 msgr2=0x5558a547f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50b54d0 0x5558a547f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/1923776716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a53a5530 con 0x5558a50d61b0 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bc8ff9640 1 -- 172.21.15.29:0/113193900 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0bbc010430 con 0x7f0bcc10bec0 2024-09-21T01:55:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.622+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098e0 tx=0x7f0bc4009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f92de046810ab99c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.624+0000 7f0bc8ff9640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 0x7f0bb80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.624+0000 7f0bc8ff9640 1 -- 172.21.15.29:0/113193900 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bbc0e7520 con 0x7f0bcc10bec0 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.625+0000 7f0bc17fa640 1 -- 172.21.15.29:0/1923776716 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc402f310 con 0x5558a50d61b0 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.625+0000 7f0bcaffd640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 0x7f0bb80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.625+0000 7f0bc17fa640 1 -- 172.21.15.29:0/1923776716 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc4036030 con 0x5558a50d61b0 2024-09-21T01:55:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.625+0000 7f0bc17fa640 1 -- 172.21.15.29:0/1923776716 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40044c0 con 0x5558a50d61b0 2024-09-21T01:55:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.626+0000 7f0bd847b740 1 -- 172.21.15.29:0/1923776716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 msgr2=0x5558a547de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.626+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098e0 tx=0x7f0bc4009910 comp rx=0 tx=0).stop 2024-09-21T01:55:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.626+0000 7f0bcaffd640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 0x7f0bb80b8bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ad410 tx=0x7f0bbc00cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 -- 172.21.15.29:0/1923776716 shutdown_connections 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50b54d0 0x5558a547f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a547e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1923776716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50d61b0 0x5558a547de50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 -- 172.21.15.29:0/1923776716 >> 172.21.15.29:0/1923776716 conn(0x5558a50ca300 msgr2=0x5558a54798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 -- 172.21.15.29:0/1923776716 shutdown_connections 2024-09-21T01:55:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 -- 172.21.15.29:0/1923776716 wait complete. 2024-09-21T01:55:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.627+0000 7f0bd847b740 1 -- start start 2024-09-21T01:55:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 Processor -- start 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a5487450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a5488680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558a5317ab0 con 0x5558a50d61b0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5558a5385730 con 0x5558a50ad2f0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5558a547f950 con 0x5558a50b54d0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41552/0 (socket says 172.21.15.29:41552) 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bc3fff640 1 -- 172.21.15.29:0/1681208109 learned_addr learned my addr 172.21.15.29:0/1681208109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a5488680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bd847b740 10 client.0 _ll_register_callbacks cb 0x5558a53d96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.628+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a5487450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bc3fff640 1 -- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 msgr2=0x5558a5487450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a5487450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bc3fff640 1 -- 172.21.15.29:0/1681208109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 msgr2=0x5558a5488680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a5488680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bc3fff640 1 -- 172.21.15.29:0/1681208109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a5141790 con 0x5558a50b54d0 2024-09-21T01:55:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.629+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a5488680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.630+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400ba80 tx=0x7f0bb400bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9a632965609e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.630+0000 7f0bb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.630+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4011440 con 0x5558a50b54d0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.630+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bb40115e0 con 0x5558a50b54d0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.630+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4036030 con 0x5558a50b54d0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bb4011820 con 0x5558a50b54d0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 10 client.26713 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5558a51ba620 con 0x5558a50b54d0 2024-09-21T01:55:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4021030 con 0x5558a50b54d0 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f0bb40109c0 con 0x5558a50b54d0 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 10 client.26713 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 20 client.26713 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a5094d80 0x5558a5494c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 4 client.26713 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.631+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bc4036030 con 0x5558a5094d80 2024-09-21T01:55:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.632+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a5094d80 0x5558a5494c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.633+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a5094d80 0x5558a5494c90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3228b61b568e73 server_cookie=77e53d61f6aea78e in_seq=0 out_seq=0 2024-09-21T01:55:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.633+0000 7f0bb27fc640 10 client.26713 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.636+0000 7f0bb27fc640 1 -- 172.21.15.29:0/1681208109 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0bc4036030 con 0x5558a5094d80 2024-09-21T01:55:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.636+0000 7f0bb27fc640 10 client.26713 handle_command_reply: tid=0 2024-09-21T01:55:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.636+0000 7f0bd847b740 1 client.26713 shutdown 2024-09-21T01:55:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.646+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 msgr2=0x5558a5487b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.646+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400ba80 tx=0x7f0bb400bf40 comp rx=0 tx=0).stop 2024-09-21T01:55:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 shutdown_connections 2024-09-21T01:55:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a5094d80 0x5558a5494c90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bb27fc640 0 client.26713 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a5488680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50b54d0 0x5558a5487b10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 --2- 172.21.15.29:0/1681208109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50ad2f0 0x5558a5487450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.650+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 >> 172.21.15.29:0/1681208109 conn(0x5558a50ca300 msgr2=0x5558a5477ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.655+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 shutdown_connections 2024-09-21T01:55:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.655+0000 7f0bd847b740 1 -- 172.21.15.29:0/1681208109 wait complete. 2024-09-21T01:55:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.657+0000 7f0bd847b740 20 client.26713 trim_cache size 0 max 16384 2024-09-21T01:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.694+0000 7f0bd847b740 1 Processor -- start 2024-09-21T01:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.694+0000 7f0bd847b740 1 -- start start 2024-09-21T01:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.695+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.695+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.695+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.697+0000 7f0bc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.697+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41556/0 (socket says 172.21.15.29:41556) 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.697+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/3411514617 learned_addr learned my addr 172.21.15.29:0/3411514617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.697+0000 7f0bd847b740 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a54745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.698+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1b9cf0 con 0x5558a50d61b0 2024-09-21T01:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.698+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bb830 con 0x5558a50b54d0 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.698+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1ba3c0 con 0x5558a50ad2f0 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.699+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/3411514617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 msgr2=0x5558a548f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.699+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.699+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/3411514617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 msgr2=0x5558a54745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.699+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a54745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.699+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/3411514617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a547fd50 con 0x5558a50ad2f0 2024-09-21T01:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.700+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009980 tx=0x7f0bc40095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6281d9ebec0e2ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.701+0000 7f0bc0ff9640 1 -- 172.21.15.29:0/3411514617 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc402f310 con 0x5558a50ad2f0 2024-09-21T01:55:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.701+0000 7f0bc0ff9640 1 -- 172.21.15.29:0/3411514617 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc4036040 con 0x5558a50ad2f0 2024-09-21T01:55:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.701+0000 7f0bc0ff9640 1 -- 172.21.15.29:0/3411514617 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40043b0 con 0x5558a50ad2f0 2024-09-21T01:55:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 msgr2=0x5558a547edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009980 tx=0x7f0bc40095d0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 shutdown_connections 2024-09-21T01:55:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a54745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 --2- 172.21.15.29:0/3411514617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a547edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.702+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 >> 172.21.15.29:0/3411514617 conn(0x5558a50ca300 msgr2=0x5558a5486650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.704+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 shutdown_connections 2024-09-21T01:55:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.704+0000 7f0bd847b740 1 -- 172.21.15.29:0/3411514617 wait complete. 2024-09-21T01:55:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.705+0000 7f0bd847b740 1 -- start start 2024-09-21T01:55:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.705+0000 7f0bd847b740 1 Processor -- start 2024-09-21T01:55:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bd847b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bcb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:41564/0 (socket says 172.21.15.29:41564) 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/133386928 learned_addr learned my addr 172.21.15.29:0/133386928 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.706+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a548e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bb160 con 0x5558a50d61b0 2024-09-21T01:55:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1baa90 con 0x5558a50b54d0 2024-09-21T01:55:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1b7cd0 con 0x5558a50ad2f0 2024-09-21T01:55:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bd847b740 10 client.0 _ll_register_callbacks cb 0x7f0bb4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 msgr2=0x5558a548dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/133386928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 msgr2=0x5558a548e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a548e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.707+0000 7f0bcb7fe640 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a548e620 con 0x5558a50ad2f0 2024-09-21T01:55:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.708+0000 7f0bcb7fe640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40095a0 tx=0x7f0bc4003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30136d9fd6ec188 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.708+0000 7f0bb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.708+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40316d0 con 0x5558a50ad2f0 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.708+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0bc40040e0 con 0x5558a50ad2f0 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.709+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558a547f490 con 0x5558a50ad2f0 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.709+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bc4004b40 con 0x5558a50ad2f0 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.709+0000 7f0bd847b740 10 client.26719 fetch_fsmap learned FSMap version 41 2024-09-21T01:55:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.709+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5558a548d390 con 0x5558a50ad2f0 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5558a547f490 con 0x5558a50ad2f0 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc4043070 con 0x5558a50ad2f0 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 10 client.26719 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 20 client.26719 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a506a1c0 0x5558a55018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 4 client.26719 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558a53f2640 con 0x5558a506a1c0 2024-09-21T01:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.710+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a506a1c0 0x5558a55018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.711+0000 7f0bb27fc640 10 client.26719 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.712+0000 7f0bc3fff640 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a506a1c0 0x5558a55018a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe088e854f9ee53 server_cookie=7368dec88652be15 in_seq=0 out_seq=0 2024-09-21T01:55:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.730+0000 7f0bb27fc640 1 -- 172.21.15.29:0/133386928 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558a53f2640 con 0x5558a506a1c0 2024-09-21T01:55:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.730+0000 7f0bb27fc640 10 client.26719 handle_command_reply: tid=0 2024-09-21T01:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.730+0000 7f0bd847b740 1 client.26719 shutdown 2024-09-21T01:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.730+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 msgr2=0x5558a548eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.730+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40095a0 tx=0x7f0bc4003f40 comp rx=0 tx=0).stop 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 shutdown_connections 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558a506a1c0 0x5558a55018a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558a50d61b0 0x5558a548e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558a50b54d0 0x5558a548dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 --2- 172.21.15.29:0/133386928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558a50ad2f0 0x5558a548eee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 >> 172.21.15.29:0/133386928 conn(0x5558a50ca300 msgr2=0x5558a5486650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bb27fc640 0 client.26719 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 shutdown_connections 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 1 -- 172.21.15.29:0/133386928 wait complete. 2024-09-21T01:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.731+0000 7f0bd847b740 20 client.26719 trim_cache size 0 max 16384 2024-09-21T01:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 msgr2=0x7f0bb80b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 0x7f0bb80b8bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ad410 tx=0x7f0bbc00cf20 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 msgr2=0x7f0bcc1a8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00e9f0 tx=0x7f0bbc00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 shutdown_connections 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0bb80b66e0 0x7f0bb80b8bf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bcc10d8e0 0x7f0bcc1a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc10cf40 0x7f0bcc1a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 --2- 172.21.15.29:0/113193900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc10bec0 0x7f0bcc1a8cb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 >> 172.21.15.29:0/113193900 conn(0x7f0bcc072a40 msgr2=0x7f0bcc10e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 shutdown_connections 2024-09-21T01:55:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:40.732+0000 7f0bd1430640 1 -- 172.21.15.29:0/113193900 wait complete. 2024-09-21T01:55:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:41 smithi190 ceph-mon[34368]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 811 op/s 2024-09-21T01:55:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:41 smithi066 ceph-mon[35001]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 811 op/s 2024-09-21T01:55:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:41 smithi029 ceph-mon[31228]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 811 op/s 2024-09-21T01:55:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 425.0 s, 2653.8 tps, lat 37.790 ms stddev 43.202 2024-09-21T01:55:42.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:42 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:43.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:42 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:43.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:42 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:43 smithi190 ceph-mon[34368]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-21T01:55:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:43 smithi066 ceph-mon[35001]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-21T01:55:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:43 smithi029 ceph-mon[31228]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-21T01:55:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:45 smithi190 ceph-mon[34368]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-09-21T01:55:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:45 smithi066 ceph-mon[35001]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-09-21T01:55:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:45 smithi029 ceph-mon[31228]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-09-21T01:55:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 430.0 s, 2718.0 tps, lat 36.694 ms stddev 41.121 2024-09-21T01:55:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:47 smithi190 ceph-mon[34368]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 750 op/s 2024-09-21T01:55:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:47 smithi066 ceph-mon[35001]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 750 op/s 2024-09-21T01:55:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:47 smithi029 ceph-mon[31228]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 750 op/s 2024-09-21T01:55:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:49 smithi190 ceph-mon[34368]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-09-21T01:55:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:49 smithi066 ceph-mon[35001]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-09-21T01:55:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:49 smithi029 ceph-mon[31228]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-09-21T01:55:51.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:51 smithi190 ceph-mon[34368]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-09-21T01:55:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:51 smithi066 ceph-mon[35001]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-09-21T01:55:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:51 smithi029 ceph-mon[31228]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-09-21T01:55:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 435.0 s, 2468.2 tps, lat 40.665 ms stddev 48.144 2024-09-21T01:55:52.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:52 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:53.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:52 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:53.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:52 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:55:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.890+0000 7f54deffd640 1 -- 172.21.15.29:0/2954013639 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d80318a0 con 0x7f54e010f9d0 2024-09-21T01:55:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.891+0000 7f54e5f83640 1 -- 172.21.15.29:0/2954013639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010f9d0 msgr2=0x7f54e011ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.891+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2954013639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010f9d0 0x7f54e011ab90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009900 tx=0x7f54d802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 -- 172.21.15.29:0/2954013639 shutdown_connections 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2954013639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010f9d0 0x7f54e011ab90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2954013639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 0x7f54e010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2954013639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54e010a3c0 0x7f54e010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 -- 172.21.15.29:0/2954013639 >> 172.21.15.29:0/2954013639 conn(0x7f54e0070810 msgr2=0x7f54e0072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 -- 172.21.15.29:0/2954013639 shutdown_connections 2024-09-21T01:55:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.892+0000 7f54e5f83640 1 -- 172.21.15.29:0/2954013639 wait complete. 2024-09-21T01:55:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.893+0000 7f54e5f83640 1 Processor -- start 2024-09-21T01:55:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.893+0000 7f54e5f83640 1 -- start start 2024-09-21T01:55:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.893+0000 7f54e5f83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54e5f83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 0x7f54e01a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54e5f83640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54e010f9d0 0x7f54e01a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38130/0 (socket says 172.21.15.29:38130) 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 -- 172.21.15.29:0/2573333102 learned_addr learned my addr 172.21.15.29:0/2573333102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54e011e0f0 con 0x7f54e010f9d0 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54e011df70 con 0x7f54e010cdb0 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54e011e270 con 0x7f54e010a3c0 2024-09-21T01:55:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 0x7f54e01a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 -- 172.21.15.29:0/2573333102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 msgr2=0x7f54e01a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 0x7f54e01a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 -- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54e010f9d0 msgr2=0x7f54e01a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54e010f9d0 0x7f54e01a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.894+0000 7f54df7fe640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01aa750 con 0x7f54e010a3c0 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.895+0000 7f54df7fe640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54d000eab0 tx=0x7f54d000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.895+0000 7f54dd7fa640 1 -- 172.21.15.29:0/2573333102 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000ce40 con 0x7f54e010a3c0 2024-09-21T01:55:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.895+0000 7f54dd7fa640 1 -- 172.21.15.29:0/2573333102 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0011920 con 0x7f54e010a3c0 2024-09-21T01:55:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.895+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01aaa40 con 0x7f54e010a3c0 2024-09-21T01:55:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.895+0000 7f54dd7fa640 1 -- 172.21.15.29:0/2573333102 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0010a30 con 0x7f54e010a3c0 2024-09-21T01:55:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.896+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01b1130 con 0x7f54e010a3c0 2024-09-21T01:55:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.899+0000 7f54dd7fa640 1 -- 172.21.15.29:0/2573333102 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f54d0010bd0 con 0x7f54e010a3c0 2024-09-21T01:55:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.901+0000 7f54ecfc8740 1 Processor -- start 2024-09-21T01:55:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.902+0000 7f54dd7fa640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 0x7f54cc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.903+0000 7f54dd7fa640 1 -- 172.21.15.29:0/2573333102 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54d00e78e0 con 0x7f54e010a3c0 2024-09-21T01:55:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 -- start start 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba440e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba4414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0784d0 0x5649ba442010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649ba3b36f0 con 0x5649ba0784d0 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649ba3a2b20 con 0x5649ba0991b0 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649ba3a6930 con 0x5649ba0702f0 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba440e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba440e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:42780/0 (socket says 172.21.15.29:42780) 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.905+0000 7f54df7fe640 1 -- 172.21.15.29:0/99293496 learned_addr learned my addr 172.21.15.29:0/99293496 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba4414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 -- 172.21.15.29:0/99293496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 msgr2=0x5649ba440e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba440e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 -- 172.21.15.29:0/99293496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0784d0 msgr2=0x5649ba442010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0784d0 0x5649ba442010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 -- 172.21.15.29:0/99293496 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ba368530 con 0x5649ba0702f0 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba4414c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009c00 tx=0x7f54c8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d32aa9d9df5f457 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54dffff640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 0x7f54cc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.906+0000 7f54d5ffb640 1 -- 172.21.15.29:0/99293496 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8004310 con 0x5649ba0702f0 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.907+0000 7f54d5ffb640 1 -- 172.21.15.29:0/99293496 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c80044b0 con 0x5649ba0702f0 2024-09-21T01:55:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.907+0000 7f54d5ffb640 1 -- 172.21.15.29:0/99293496 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80035f0 con 0x5649ba0702f0 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54dffff640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 0x7f54cc0b8aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54d802a910 tx=0x7f54d8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- 172.21.15.29:0/99293496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 msgr2=0x5649ba4414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba4414c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009c00 tx=0x7f54c8011440 comp rx=0 tx=0).stop 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- 172.21.15.29:0/99293496 shutdown_connections 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0784d0 0x5649ba442010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba4414c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/99293496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba440e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- 172.21.15.29:0/99293496 >> 172.21.15.29:0/99293496 conn(0x5649ba08d300 msgr2=0x5649ba43c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- 172.21.15.29:0/99293496 shutdown_connections 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- 172.21.15.29:0/99293496 wait complete. 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.908+0000 7f54ecfc8740 1 -- start start 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 Processor -- start 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0702f0 0x5649ba44a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba44b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649ba2daab0 con 0x5649ba0702f0 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649ba348730 con 0x5649ba0991b0 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.909+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649ba442b30 con 0x5649ba0784d0 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54ecfc8740 10 client.0 _ll_register_callbacks cb 0x5649ba39c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38150/0 (socket says 172.21.15.29:38150) 2024-09-21T01:55:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 -- 172.21.15.29:0/984571286 learned_addr learned my addr 172.21.15.29:0/984571286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 -- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 msgr2=0x5649ba44b800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba44b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 -- 172.21.15.29:0/984571286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0702f0 msgr2=0x5649ba44a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0702f0 0x5649ba44a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.910+0000 7f54d6ffd640 1 -- 172.21.15.29:0/984571286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ba104790 con 0x5649ba0784d0 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.911+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005890 tx=0x7f54c8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0ec4123e9b8bed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.911+0000 7f54beffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.911+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8028020 con 0x5649ba0784d0 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.911+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54c8004ca0 con 0x5649ba0784d0 2024-09-21T01:55:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.911+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c80044b0 con 0x5649ba0784d0 2024-09-21T01:55:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.912+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f54c8003e50 con 0x5649ba0784d0 2024-09-21T01:55:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.912+0000 7f54ecfc8740 10 client.26731 fetch_fsmap learned FSMap version 43 2024-09-21T01:55:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.912+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5649ba17d620 con 0x5649ba0784d0 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.912+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c80044b0 con 0x5649ba0784d0 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f54c8018070 con 0x5649ba0784d0 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 10 client.26731 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 10 client.26731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 20 client.26731 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba057d80 0x5649ba457b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 4 client.26731 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:55:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.913+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ba3b5640 con 0x5649ba057d80 2024-09-21T01:55:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.914+0000 7f54df7fe640 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba057d80 0x5649ba457b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.915+0000 7f54df7fe640 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba057d80 0x5649ba457b70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d997bad48a7a2e9c server_cookie=83e680ee949da348 in_seq=0 out_seq=0 2024-09-21T01:55:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.915+0000 7f54beffd640 10 client.26731 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.916+0000 7f54beffd640 1 -- 172.21.15.29:0/984571286 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649ba3b5640 con 0x5649ba057d80 2024-09-21T01:55:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.916+0000 7f54beffd640 10 client.26731 handle_command_reply: tid=0 2024-09-21T01:55:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.916+0000 7f54ecfc8740 1 client.26731 shutdown 2024-09-21T01:55:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.916+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 msgr2=0x5649ba44ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.916+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005890 tx=0x7f54c8003820 comp rx=0 tx=0).stop 2024-09-21T01:55:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 shutdown_connections 2024-09-21T01:55:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba057d80 0x5649ba457b70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0991b0 0x5649ba44b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0784d0 0x5649ba44ac90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/984571286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba0702f0 0x5649ba44a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54beffd640 0 client.26731 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 >> 172.21.15.29:0/984571286 conn(0x5649ba08d300 msgr2=0x5649ba43aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 shutdown_connections 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 1 -- 172.21.15.29:0/984571286 wait complete. 2024-09-21T01:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.917+0000 7f54ecfc8740 20 client.26731 trim_cache size 0 max 16384 2024-09-21T01:55:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 Processor -- start 2024-09-21T01:55:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 -- start start 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba448ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba449020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b9db0 con 0x5649ba057d80 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01bb9b0 con 0x5649ba0784d0 2024-09-21T01:55:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.942+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01ba480 con 0x5649ba0702f0 2024-09-21T01:55:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba448ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:42788/0 (socket says 172.21.15.29:42788) 2024-09-21T01:55:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba448ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43424/0 (socket says 172.21.15.29:43424) 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 -- 172.21.15.29:0/2204183413 learned_addr learned my addr 172.21.15.29:0/2204183413 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba449020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 -- 172.21.15.29:0/2204183413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 msgr2=0x5649ba449020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba449020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 -- 172.21.15.29:0/2204183413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 msgr2=0x5649ba448ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba448ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 -- 172.21.15.29:0/2204183413 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ba44bb70 con 0x5649ba0784d0 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.943+0000 7f54dffff640 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54d8031840 tx=0x7f54d803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd0229855e17c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.944+0000 7f54d57fa640 1 -- 172.21.15.29:0/2204183413 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d803b960 con 0x5649ba0784d0 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.944+0000 7f54d57fa640 1 -- 172.21.15.29:0/2204183413 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d804c070 con 0x5649ba0784d0 2024-09-21T01:55:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.944+0000 7f54d57fa640 1 -- 172.21.15.29:0/2204183413 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8047e50 con 0x5649ba0784d0 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 -- 172.21.15.29:0/2204183413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 msgr2=0x5649ba44a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54d8031840 tx=0x7f54d803b000 comp rx=0 tx=0).stop 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 -- 172.21.15.29:0/2204183413 shutdown_connections 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba44a5d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba449020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/2204183413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba448ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.945+0000 7f54ecfc8740 1 -- 172.21.15.29:0/2204183413 >> 172.21.15.29:0/2204183413 conn(0x5649ba08d300 msgr2=0x5649ba440df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.946+0000 7f54ecfc8740 1 -- 172.21.15.29:0/2204183413 shutdown_connections 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.946+0000 7f54ecfc8740 1 -- 172.21.15.29:0/2204183413 wait complete. 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.946+0000 7f54ecfc8740 1 -- start start 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.946+0000 7f54ecfc8740 1 Processor -- start 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba44fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba4505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01bb2e0 con 0x5649ba057d80 2024-09-21T01:55:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01bac10 con 0x5649ba0784d0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b7d90 con 0x5649ba0702f0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43426/0 (socket says 172.21.15.29:43426) 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54df7fe640 1 -- 172.21.15.29:0/956923754 learned_addr learned my addr 172.21.15.29:0/956923754 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54ecfc8740 10 client.0 _ll_register_callbacks cb 0x7f54c8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba44fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.947+0000 7f54dffff640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba4505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54d6ffd640 1 -- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 msgr2=0x5649ba4505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba4505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54d6ffd640 1 -- 172.21.15.29:0/956923754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 msgr2=0x5649ba44f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54d6ffd640 1 -- 172.21.15.29:0/956923754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ba450d40 con 0x5649ba0702f0 2024-09-21T01:55:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.948+0000 7f54df7fe640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:55:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.949+0000 7f54d6ffd640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba44fac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54c8031040 tx=0x7f54c8005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785d9e6ab32397f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:55:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.949+0000 7f54beffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:55:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c803d060 con 0x5649ba0702f0 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d804c070 con 0x5649ba0702f0 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54c80249c0 con 0x5649ba0702f0 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f54c8028040 con 0x5649ba0702f0 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54ecfc8740 10 client.26737 fetch_fsmap learned FSMap version 43 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.950+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5649ba452070 con 0x5649ba0702f0 2024-09-21T01:55:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c801f040 con 0x5649ba0702f0 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f54c8018040 con 0x5649ba0702f0 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 10 client.26737 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 10 client.26737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 20 client.26737 populate_metadata read hostname 'smithi029' 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba0991b0 0x5649ba4c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 4 client.26737 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:55:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.951+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f54d804c070 con 0x5649ba0991b0 2024-09-21T01:55:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.952+0000 7f54df7fe640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba0991b0 0x5649ba4c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:55:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.952+0000 7f54beffd640 10 client.26737 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.952+0000 7f54df7fe640 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba0991b0 0x5649ba4c45e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47ad5b986e4046 server_cookie=bdae0b7c6432d73e in_seq=0 out_seq=0 2024-09-21T01:55:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.965+0000 7f54beffd640 1 -- 172.21.15.29:0/956923754 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f54d804c070 con 0x5649ba0991b0 2024-09-21T01:55:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.966+0000 7f54beffd640 10 client.26737 handle_command_reply: tid=0 2024-09-21T01:55:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.966+0000 7f54ecfc8740 1 client.26737 shutdown 2024-09-21T01:55:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 msgr2=0x5649ba44fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba44fac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54c8031040 tx=0x7f54c8005660 comp rx=0 tx=0).stop 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 shutdown_connections 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649ba0991b0 0x5649ba4c45e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649ba0784d0 0x5649ba4505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54beffd640 0 client.26737 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649ba0702f0 0x5649ba44fac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 --2- 172.21.15.29:0/956923754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649ba057d80 0x5649ba44f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 >> 172.21.15.29:0/956923754 conn(0x5649ba08d300 msgr2=0x5649ba448aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 shutdown_connections 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.967+0000 7f54ecfc8740 1 -- 172.21.15.29:0/956923754 wait complete. 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54ecfc8740 20 client.26737 trim_cache size 0 max 16384 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:55:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 msgr2=0x7f54cc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 0x7f54cc0b8aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54d802a910 tx=0x7f54d8005d50 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 msgr2=0x7f54e01a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54d000eab0 tx=0x7f54d000ef70 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 shutdown_connections 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54cc0b6590 0x7f54cc0b8aa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54e010f9d0 0x7f54e01a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54e010cdb0 0x7f54e01a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 --2- 172.21.15.29:0/2573333102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e010a3c0 0x7f54e01a8d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:55:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.968+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 >> 172.21.15.29:0/2573333102 conn(0x7f54e0070810 msgr2=0x7f54e010b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:55:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.970+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 shutdown_connections 2024-09-21T01:55:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:55:53.970+0000 7f54e5f83640 1 -- 172.21.15.29:0/2573333102 wait complete. 2024-09-21T01:55:53.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:53 smithi190 ceph-mon[34368]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-09-21T01:55:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:53 smithi066 ceph-mon[35001]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-09-21T01:55:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:53 smithi029 ceph-mon[31228]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-09-21T01:55:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:55 smithi066 ceph-mon[35001]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-21T01:55:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:55 smithi029 ceph-mon[31228]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-21T01:55:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:55 smithi190 ceph-mon[34368]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-21T01:55:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 440.0 s, 2855.4 tps, lat 35.016 ms stddev 40.804 2024-09-21T01:55:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:57 smithi066 ceph-mon[35001]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-21T01:55:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:57 smithi029 ceph-mon[31228]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-21T01:55:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:57 smithi190 ceph-mon[34368]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-21T01:55:59.158 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:55:59.581 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:55:59 smithi066 ceph-mon[35001]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-21T01:56:00.084 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:55:59 smithi029 ceph-mon[31228]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-21T01:56:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:55:59 smithi190 ceph-mon[34368]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.229+0000 7ff556607640 1 -- 172.21.15.29:0/1112284734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102650 msgr2=0x7ff550102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.229+0000 7ff556607640 1 --2- 172.21.15.29:0/1112284734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102650 0x7ff550102a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff540009900 tx=0x7ff54002a480 comp rx=0 tx=0).stop 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 -- 172.21.15.29:0/1112284734 shutdown_connections 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 --2- 172.21.15.29:0/1112284734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff550112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 --2- 172.21.15.29:0/1112284734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102fa0 0x7ff55010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 --2- 172.21.15.29:0/1112284734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102650 0x7ff550102a60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 -- 172.21.15.29:0/1112284734 >> 172.21.15.29:0/1112284734 conn(0x7ff550079730 msgr2=0x7ff550079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 -- 172.21.15.29:0/1112284734 shutdown_connections 2024-09-21T01:56:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.230+0000 7ff556607640 1 -- 172.21.15.29:0/1112284734 wait complete. 2024-09-21T01:56:00.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.231+0000 7ff556607640 1 Processor -- start 2024-09-21T01:56:00.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.232+0000 7ff556607640 1 -- start start 2024-09-21T01:56:00.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.233+0000 7ff556607640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff556607640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102fa0 0x7ff55019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:42808/0 (socket says 172.21.15.29:42808) 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff54ffff640 1 -- 172.21.15.29:0/13398012 learned_addr learned my addr 172.21.15.29:0/13398012 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff5501a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff550115c70 con 0x7ff550102fa0 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff550115af0 con 0x7ff550102650 2024-09-21T01:56:00.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff550115df0 con 0x7ff550110200 2024-09-21T01:56:00.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.234+0000 7ff554ba4640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff5501a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 -- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 msgr2=0x7ff5501a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff5501a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 -- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102fa0 msgr2=0x7ff55019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102fa0 0x7ff55019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5501a23d0 con 0x7ff550102650 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff554ba4640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff5501a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54ffff640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff5400305d0 tx=0x7ff540030600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54d7fa640 1 -- 172.21.15.29:0/13398012 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54003f070 con 0x7ff550102650 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff54d7fa640 1 -- 172.21.15.29:0/13398012 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff540004460 con 0x7ff550102650 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.235+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5501a2660 con 0x7ff550102650 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.236+0000 7ff54d7fa640 1 -- 172.21.15.29:0/13398012 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540035030 con 0x7ff550102650 2024-09-21T01:56:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.236+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5501a2a80 con 0x7ff550102650 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.239+0000 7ff55cb7b740 1 Processor -- start 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.239+0000 7ff55cb7b740 1 -- start start 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.239+0000 7ff54d7fa640 1 -- 172.21.15.29:0/13398012 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff540042070 con 0x7ff550102650 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 0x563f865e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f8655ee50 con 0x563f86220390 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f8654e280 con 0x563f862181b0 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f86552090 con 0x563f86241070 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff54f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff54f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43470/0 (socket says 172.21.15.29:43470) 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff54f7fe640 1 -- 172.21.15.29:0/1353026157 learned_addr learned my addr 172.21.15.29:0/1353026157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:00.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff554ba4640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 0x563f865e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.240+0000 7ff54ffff640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54f7fe640 1 -- 172.21.15.29:0/1353026157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 msgr2=0x563f865e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54d7fa640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 0x7ff5380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 0x563f865e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54f7fe640 1 -- 172.21.15.29:0/1353026157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 msgr2=0x563f865e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54f7fe640 1 -- 172.21.15.29:0/1353026157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f862ac650 con 0x563f86220390 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff554ba4640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 0x563f865e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:00.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff54d7fa640 1 -- 172.21.15.29:0/13398012 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff540107d00 con 0x7ff550102650 2024-09-21T01:56:00.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.241+0000 7ff554ba4640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 0x7ff5380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.242+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff544009be0 tx=0x7ff54400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fc47a07d97a560 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.242+0000 7ff5367fc640 1 -- 172.21.15.29:0/1353026157 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544005020 con 0x563f86220390 2024-09-21T01:56:00.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.242+0000 7ff5367fc640 1 -- 172.21.15.29:0/1353026157 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5440047a0 con 0x563f86220390 2024-09-21T01:56:00.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.242+0000 7ff5367fc640 1 -- 172.21.15.29:0/1353026157 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544005020 con 0x563f86220390 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 -- 172.21.15.29:0/1353026157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 msgr2=0x563f865e8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff544009be0 tx=0x7ff54400e720 comp rx=0 tx=0).stop 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff554ba4640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 0x7ff5380b8aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff548005fd0 tx=0x7ff548004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 -- 172.21.15.29:0/1353026157 shutdown_connections 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f865e8cf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/1353026157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86241070 0x563f865e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 -- 172.21.15.29:0/1353026157 >> 172.21.15.29:0/1353026157 conn(0x563f86235460 msgr2=0x563f865e6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 -- 172.21.15.29:0/1353026157 shutdown_connections 2024-09-21T01:56:00.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.244+0000 7ff55cb7b740 1 -- 172.21.15.29:0/1353026157 wait complete. 2024-09-21T01:56:00.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.245+0000 7ff55cb7b740 1 -- start start 2024-09-21T01:56:00.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.245+0000 7ff55cb7b740 1 Processor -- start 2024-09-21T01:56:00.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 0x563f865fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f863af700 con 0x563f86241070 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f864e9110 con 0x563f862181b0 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:42818/0 (socket says 172.21.15.29:42818) 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff54ffff640 1 -- 172.21.15.29:0/307288177 learned_addr learned my addr 172.21.15.29:0/307288177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.246+0000 7ff554ba4640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 0x563f865fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f865ff1d0 con 0x563f86220390 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff55cb7b740 10 client.0 _ll_register_callbacks cb 0x563f86502990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 -- 172.21.15.29:0/307288177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 msgr2=0x563f865fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 -- 172.21.15.29:0/307288177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 msgr2=0x563f865fcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 0x563f865fcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 -- 172.21.15.29:0/307288177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f86516c40 con 0x563f862181b0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff554ba4640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 0x563f865fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff54ffff640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5400cdb40 tx=0x7ff540003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4b1cc1536db215 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.247+0000 7ff52f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ca490 con 0x563f862181b0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff5400b52b0 con 0x563f862181b0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5440047a0 con 0x563f862181b0 2024-09-21T01:56:00.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5400cd5b0 con 0x563f862181b0 2024-09-21T01:56:00.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff55cb7b740 10 client.26747 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:00.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563f8660ae60 con 0x563f862181b0 2024-09-21T01:56:00.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5440047a0 con 0x563f862181b0 2024-09-21T01:56:00.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.248+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff540110030 con 0x563f862181b0 2024-09-21T01:56:00.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 10 client.26747 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:00.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 20 client.26747 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f861ff9f0 0x563f8660e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 4 client.26747 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f86560da0 con 0x563f861ff9f0 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.249+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f861ff9f0 0x563f8660e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.250+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f861ff9f0 0x563f8660e4f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5613bf54fc9f2c76 server_cookie=b74155686799f2df in_seq=0 out_seq=0 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.250+0000 7ff52f7fe640 10 client.26747 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.250+0000 7ff52f7fe640 1 -- 172.21.15.29:0/307288177 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f86560da0 con 0x563f861ff9f0 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.250+0000 7ff52f7fe640 10 client.26747 handle_command_reply: tid=0 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.250+0000 7ff55cb7b740 1 client.26747 shutdown 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 msgr2=0x563f865fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5400cdb40 tx=0x7ff540003800 comp rx=0 tx=0).stop 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 shutdown_connections 2024-09-21T01:56:00.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f861ff9f0 0x563f8660e4f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff52f7fe640 0 client.26747 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86241070 0x563f865fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/307288177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f862181b0 0x563f865fc050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 >> 172.21.15.29:0/307288177 conn(0x563f86235460 msgr2=0x563f865e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 shutdown_connections 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 1 -- 172.21.15.29:0/307288177 wait complete. 2024-09-21T01:56:00.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.251+0000 7ff55cb7b740 20 client.26747 trim_cache size 0 max 16384 2024-09-21T01:56:00.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 Processor -- start 2024-09-21T01:56:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 -- start start 2024-09-21T01:56:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 0x563f865e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f865f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501ad110 con 0x563f862181b0 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38212/0 (socket says 172.21.15.29:38212) 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54f7fe640 1 -- 172.21.15.29:0/2503837541 learned_addr learned my addr 172.21.15.29:0/2503837541 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.280+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501aed10 con 0x563f86241070 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501ad7e0 con 0x563f86220390 2024-09-21T01:56:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 0x563f865e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54ffff640 1 -- 172.21.15.29:0/2503837541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 msgr2=0x563f865fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.281+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.282+0000 7ff54ffff640 1 -- 172.21.15.29:0/2503837541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 msgr2=0x563f865f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.282+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f865f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.282+0000 7ff54ffff640 1 -- 172.21.15.29:0/2503837541 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f865e6ae0 con 0x563f862181b0 2024-09-21T01:56:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.282+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 0x563f865e6100 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff5400cddd0 tx=0x7ff54000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b19000ce06e8cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.287+0000 7ff535ffb640 1 -- 172.21.15.29:0/2503837541 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400d0020 con 0x563f862181b0 2024-09-21T01:56:00.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.287+0000 7ff535ffb640 1 -- 172.21.15.29:0/2503837541 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5400efe50 con 0x563f862181b0 2024-09-21T01:56:00.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.287+0000 7ff535ffb640 1 -- 172.21.15.29:0/2503837541 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400cd790 con 0x563f862181b0 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 msgr2=0x563f865e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 0x563f865e6100 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff5400cddd0 tx=0x7ff54000b040 comp rx=0 tx=0).stop 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 shutdown_connections 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f865f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f86220390 0x563f865fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2503837541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f862181b0 0x563f865e6100 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff5400cddd0 tx=0x7ff54000b040 comp rx=0 tx=0).stop 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.288+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 >> 172.21.15.29:0/2503837541 conn(0x563f86235460 msgr2=0x563f865fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.291+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 shutdown_connections 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.291+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2503837541 wait complete. 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.292+0000 7ff55cb7b740 1 -- start start 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.292+0000 7ff55cb7b740 1 Processor -- start 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.293+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f866005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.293+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f86600b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.293+0000 7ff55cb7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 0x563f86601050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.293+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501ae640 con 0x563f86220390 2024-09-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.294+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501adf70 con 0x563f86241070 2024-09-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.294+0000 7ff55cb7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501ab130 con 0x563f861d4da0 2024-09-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.294+0000 7ff55cb7b740 10 client.0 _ll_register_callbacks cb 0x7ff540005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.295+0000 7ff554ba4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 0x563f86601050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.295+0000 7ff54f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f86600b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.295+0000 7ff554ba4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 0x563f86601050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38214/0 (socket says 172.21.15.29:38214) 2024-09-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.295+0000 7ff554ba4640 1 -- 172.21.15.29:0/2915623061 learned_addr learned my addr 172.21.15.29:0/2915623061 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.298+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f866005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 -- 172.21.15.29:0/2915623061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 msgr2=0x563f86601050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 0x563f86601050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 -- 172.21.15.29:0/2915623061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 msgr2=0x563f866005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f866005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 -- 172.21.15.29:0/2915623061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f86678020 con 0x563f86241070 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff54f7fe640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f86600b10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff544009250 tx=0x7ff544004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b69431e81c62ed88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff52f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54402f070 con 0x563f86241070 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.300+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5400efe50 con 0x563f86241070 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.301+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff544009690 con 0x563f86241070 2024-09-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.301+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff54403b050 con 0x563f86241070 2024-09-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.301+0000 7ff55cb7b740 10 client.26750 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.301+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563f866792b0 con 0x563f86241070 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.301+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5440051c0 con 0x563f86241070 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff544034440 con 0x563f86241070 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 10 client.26750 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 10 client.26750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 20 client.26750 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f866797e0 0x563f8667bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 4 client.26750 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.303+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff5400efe50 con 0x563f866797e0 2024-09-21T01:56:00.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.304+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f866797e0 0x563f8667bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:00.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.307+0000 7ff52f7fe640 10 client.26750 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:00.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.307+0000 7ff54ffff640 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f866797e0 0x563f8667bc00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead33cd7f935e172 server_cookie=86cdc5e543c1b903 in_seq=0 out_seq=0 2024-09-21T01:56:00.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.308+0000 7ff52f7fe640 1 -- 172.21.15.29:0/2915623061 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x7ff5400efe50 con 0x563f866797e0 2024-09-21T01:56:00.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.308+0000 7ff52f7fe640 10 client.26750 handle_command_reply: tid=0 2024-09-21T01:56:00.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.308+0000 7ff55cb7b740 1 client.26750 shutdown 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 msgr2=0x563f86600b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f86600b10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff544009250 tx=0x7ff544004830 comp rx=0 tx=0).stop 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 shutdown_connections 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563f866797e0 0x563f8667bc00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f861d4da0 0x563f86601050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f86241070 0x563f86600b10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 --2- 172.21.15.29:0/2915623061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f86220390 0x563f866005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 >> 172.21.15.29:0/2915623061 conn(0x563f86235460 msgr2=0x563f86607450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff52f7fe640 0 client.26750 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 shutdown_connections 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.309+0000 7ff55cb7b740 1 -- 172.21.15.29:0/2915623061 wait complete. 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.310+0000 7ff55cb7b740 20 client.26750 trim_cache size 0 max 16384 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:56:00.312 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:56:00.314 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:00.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:00.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:00.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:00.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:00.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:00.318 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 msgr2=0x7ff5380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 0x7ff5380b8aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff548005fd0 tx=0x7ff548004a40 comp rx=0 tx=0).stop 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 msgr2=0x7ff55019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff5400305d0 tx=0x7ff540030600 comp rx=0 tx=0).stop 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 shutdown_connections 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff5380b6590 0x7ff5380b8aa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550110200 0x7ff5501a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff550102fa0 0x7ff55019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 --2- 172.21.15.29:0/13398012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff550102650 0x7ff55019c460 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:00.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.311+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 >> 172.21.15.29:0/13398012 conn(0x7ff550079730 msgr2=0x7ff5500ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:00.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.312+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 shutdown_connections 2024-09-21T01:56:00.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:00.312+0000 7ff556607640 1 -- 172.21.15.29:0/13398012 wait complete. 2024-09-21T01:56:01.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:56:01.279 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:56:01.710 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:01 smithi066 ceph-mon[35001]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 821 op/s 2024-09-21T01:56:02.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:01 smithi029 ceph-mon[31228]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 821 op/s 2024-09-21T01:56:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:01 smithi190 ceph-mon[34368]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 821 op/s 2024-09-21T01:56:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 445.0 s, 2506.6 tps, lat 39.771 ms stddev 56.596 2024-09-21T01:56:03.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 -- 172.21.15.29:0/1303944067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 msgr2=0x7f0e10075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:03.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 --2- 172.21.15.29:0/1303944067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e10075b60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02a410 comp rx=0 tx=0).stop 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 -- 172.21.15.29:0/1303944067 shutdown_connections 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 --2- 172.21.15.29:0/1303944067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e10112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 --2- 172.21.15.29:0/1303944067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 --2- 172.21.15.29:0/1303944067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e10075b60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.218+0000 7f0e17d40640 1 -- 172.21.15.29:0/1303944067 >> 172.21.15.29:0/1303944067 conn(0x7f0e100ff610 msgr2=0x7f0e10101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.219+0000 7f0e17d40640 1 -- 172.21.15.29:0/1303944067 shutdown_connections 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.219+0000 7f0e17d40640 1 -- 172.21.15.29:0/1303944067 wait complete. 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.220+0000 7f0e17d40640 1 Processor -- start 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.220+0000 7f0e17d40640 1 -- start start 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.220+0000 7f0e17d40640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e1019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.220+0000 7f0e17d40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.220+0000 7f0e17d40640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e101a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:03.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e17d40640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10115d00 con 0x7f0e10075750 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e17d40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10115b80 con 0x7f0e100760a0 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e17d40640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10115e80 con 0x7f0e10110140 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e152db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e152db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38840/0 (socket says 172.21.15.29:38840) 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e152db640 1 -- 172.21.15.29:0/3124886946 learned_addr learned my addr 172.21.15.29:0/3124886946 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e101a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e15adc640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e1019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 -- 172.21.15.29:0/3124886946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 msgr2=0x7f0e1019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 -- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 msgr2=0x7f0e1019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e1019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e162dd640 1 -- 172.21.15.29:0/3124886946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101a26a0 con 0x7f0e10110140 2024-09-21T01:56:03.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e152db640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.221+0000 7f0e15adc640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e1019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e162dd640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e101a1f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009fd0 tx=0x7f0e0800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800ce20 con 0x7f0e10110140 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e08005020 con 0x7f0e10110140 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08010690 con 0x7f0e10110140 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101a2990 con 0x7f0e10110140 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.222+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101a90f0 con 0x7f0e10110140 2024-09-21T01:56:03.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.223+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e10075b60 con 0x7f0e10110140 2024-09-21T01:56:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.227+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0e080054d0 con 0x7f0e10110140 2024-09-21T01:56:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.228+0000 7f0e06ffd640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 0x7f0dfc0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.228+0000 7f0e15adc640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 0x7f0dfc0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.228+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0e080e79b0 con 0x7f0e10110140 2024-09-21T01:56:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.229+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e080ae250 con 0x7f0e10110140 2024-09-21T01:56:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.229+0000 7f0e15adc640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 0x7f0dfc0b8ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009850 tx=0x7f0e0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:03.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.612+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e1019d860 con 0x7f0e10110140 2024-09-21T01:56:03.615 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:56:03.615 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"btime":"2024-09-21T01:55:51:710037+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:55:51.255007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:56:03.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.613+0000 7f0e06ffd640 1 -- 172.21.15.29:0/3124886946 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7f0e1019d860 con 0x7f0e10110140 2024-09-21T01:56:03.615 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 msgr2=0x7f0dfc0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 0x7f0dfc0b8ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009850 tx=0x7f0e0c000bc0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 msgr2=0x7f0e101a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e101a1f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009fd0 tx=0x7f0e0800ef10 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 shutdown_connections 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dfc0b65d0 0x7f0dfc0b8ae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e10110140 0x7f0e101a1f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e100760a0 0x7f0e1019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 --2- 172.21.15.29:0/3124886946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e10075750 0x7f0e1019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:03.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 >> 172.21.15.29:0/3124886946 conn(0x7f0e100ff610 msgr2=0x7f0e10109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:03.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 shutdown_connections 2024-09-21T01:56:03.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:03.616+0000 7f0e17d40640 1 -- 172.21.15.29:0/3124886946 wait complete. 2024-09-21T01:56:03.971 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:03 smithi029 ceph-mon[31228]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-21T01:56:03.971 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:03 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3124886946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:03 smithi066 ceph-mon[35001]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-21T01:56:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:03 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3124886946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:03 smithi190 ceph-mon[34368]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-21T01:56:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:03 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3124886946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:04.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:56:04.806 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 -- 172.21.15.29:0/1756880299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 msgr2=0x7fc828104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 --2- 172.21.15.29:0/1756880299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc828104dc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 -- 172.21.15.29:0/1756880299 shutdown_connections 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 --2- 172.21.15.29:0/1756880299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc82810de10 0x7fc828110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 --2- 172.21.15.29:0/1756880299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc828105300 0x7fc82810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 --2- 172.21.15.29:0/1756880299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc828104dc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 -- 172.21.15.29:0/1756880299 >> 172.21.15.29:0/1756880299 conn(0x7fc828079730 msgr2=0x7fc828079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 -- 172.21.15.29:0/1756880299 shutdown_connections 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.517+0000 7fc82d43b640 1 -- 172.21.15.29:0/1756880299 wait complete. 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.518+0000 7fc82d43b640 1 Processor -- start 2024-09-21T01:56:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.518+0000 7fc82d43b640 1 -- start start 2024-09-21T01:56:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.518+0000 7fc82d43b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc8281026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.518+0000 7fc82d43b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 0x7fc828102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.518+0000 7fc82d43b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 0x7fc828103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc82d43b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc828113ac0 con 0x7fc8281049b0 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc8281026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc8281026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59340/0 (socket says 172.21.15.29:59340) 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc826ffd640 1 -- 172.21.15.29:0/155354926 learned_addr learned my addr 172.21.15.29:0/155354926 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 0x7fc828102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc828113940 con 0x7fc828105300 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc828113c40 con 0x7fc82810de10 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8277fe640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 0x7fc828103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 -- 172.21.15.29:0/155354926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 msgr2=0x7fc828103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 0x7fc828103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 -- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 msgr2=0x7fc8281026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc8281026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 -- 172.21.15.29:0/155354926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8281a8790 con 0x7fc828105300 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8277fe640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 0x7fc828103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.519+0000 7fc8267fc640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 0x7fc828102c30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc82806ad30 tx=0x7fc81c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.520+0000 7fc80ffff640 1 -- 172.21.15.29:0/155354926 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00cc40 con 0x7fc828105300 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.520+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281a8a80 con 0x7fc828105300 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.520+0000 7fc80ffff640 1 -- 172.21.15.29:0/155354926 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c00cde0 con 0x7fc828105300 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.520+0000 7fc80ffff640 1 -- 172.21.15.29:0/155354926 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c018780 con 0x7fc828105300 2024-09-21T01:56:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.520+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a8fc0 con 0x7fc828105300 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.524+0000 7fc8339af740 1 Processor -- start 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 -- start start 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc80ffff640 1 -- 172.21.15.29:0/155354926 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc81c010a90 con 0x7fc828105300 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 0x55c586ba2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867da390 0x55c586ba2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c586b18ea0 con 0x55c5867fb070 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c586b082d0 con 0x55c5867da390 2024-09-21T01:56:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8339af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c586b0c0e0 con 0x55c5867d21b0 2024-09-21T01:56:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59356/0 (socket says 172.21.15.29:59356) 2024-09-21T01:56:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8277fe640 1 -- 172.21.15.29:0/2540110275 learned_addr learned my addr 172.21.15.29:0/2540110275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc826ffd640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 0x55c586ba2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.525+0000 7fc8267fc640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867da390 0x55c586ba2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 -- 172.21.15.29:0/2540110275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 msgr2=0x55c586ba2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 0x55c586ba2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 -- 172.21.15.29:0/2540110275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867da390 msgr2=0x55c586ba2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867da390 0x55c586ba2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 -- 172.21.15.29:0/2540110275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c586866650 con 0x55c5867fb070 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc826ffd640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 0x55c586ba2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc8277fe640 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc8200107e0 tx=0x7fc820010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482c0d87e7a9d1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc80d7fa640 1 -- 172.21.15.29:0/2540110275 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820003b70 con 0x55c5867fb070 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc80ffff640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 0x7fc8100b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc80d7fa640 1 -- 172.21.15.29:0/2540110275 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82000ede0 con 0x55c5867fb070 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc80d7fa640 1 -- 172.21.15.29:0/2540110275 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820004710 con 0x55c5867fb070 2024-09-21T01:56:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc826ffd640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 0x7fc8100b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.526+0000 7fc80ffff640 1 -- 172.21.15.29:0/155354926 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc81c0e75f0 con 0x7fc828105300 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.527+0000 7fc8339af740 1 -- 172.21.15.29:0/2540110275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 msgr2=0x55c586ba22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.527+0000 7fc826ffd640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 0x7fc8100b8a20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.527+0000 7fc8339af740 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc8200107e0 tx=0x7fc820010ca0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 -- 172.21.15.29:0/2540110275 shutdown_connections 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867da390 0x55c586ba2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867d21b0 0x55c586ba2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 --2- 172.21.15.29:0/2540110275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586ba22c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 -- 172.21.15.29:0/2540110275 >> 172.21.15.29:0/2540110275 conn(0x55c5867ef460 msgr2=0x55c586ba01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 -- 172.21.15.29:0/2540110275 shutdown_connections 2024-09-21T01:56:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.528+0000 7fc8339af740 1 -- 172.21.15.29:0/2540110275 wait complete. 2024-09-21T01:56:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.529+0000 7fc8339af740 1 -- start start 2024-09-21T01:56:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.529+0000 7fc8339af740 1 Processor -- start 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.529+0000 7fc8339af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.529+0000 7fc8339af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.529+0000 7fc8339af740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586bc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586bc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49556/0 (socket says 172.21.15.29:49556) 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 -- 172.21.15.29:0/4012532345 learned_addr learned my addr 172.21.15.29:0/4012532345 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8339af740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c586969700 con 0x55c5867fb070 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c586aa3160 con 0x55c5867d21b0 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c586bb0aa0 con 0x55c5867da390 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8339af740 10 client.0 _ll_register_callbacks cb 0x55c586abc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 -- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 msgr2=0x55c586bada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 -- 172.21.15.29:0/4012532345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 msgr2=0x55c586bc25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586bc25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c586ad0c90 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc8267fc640 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0ad770 tx=0x7fc81c0ace10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a228c3f9bc36217 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc80aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0a9680 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.530+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc81c0ac6c0 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82000ede0 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc81c0acf50 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 10 client.26773 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c586bc50a0 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82000ede0 con 0x55c5867da390 2024-09-21T01:56:05.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc81c0e9b40 con 0x55c5867da390 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 10 client.26773 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 10 client.26773 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 20 client.26773 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867b99f0 0x55c586bc84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 4 client.26773 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.531+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c586b1adf0 con 0x55c5867b99f0 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.532+0000 7fc826ffd640 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867b99f0 0x55c586bc84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.532+0000 7fc80aff5640 10 client.26773 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.532+0000 7fc826ffd640 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867b99f0 0x55c586bc84e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee9703093e384278 server_cookie=80b01e5d5274ad6d in_seq=0 out_seq=0 2024-09-21T01:56:05.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.533+0000 7fc80aff5640 1 -- 172.21.15.29:0/4012532345 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c586b1adf0 con 0x55c5867b99f0 2024-09-21T01:56:05.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.533+0000 7fc80aff5640 10 client.26773 handle_command_reply: tid=0 2024-09-21T01:56:05.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.533+0000 7fc8339af740 1 client.26773 shutdown 2024-09-21T01:56:05.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.533+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 msgr2=0x55c586badf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.533+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0ad770 tx=0x7fc81c0ace10 comp rx=0 tx=0).stop 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 shutdown_connections 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867b99f0 0x55c586bc84e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc80aff5640 0 client.26773 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867fb070 0x55c586bc25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586badf50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 --2- 172.21.15.29:0/4012532345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.535+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 >> 172.21.15.29:0/4012532345 conn(0x55c5867ef460 msgr2=0x55c586ba54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.536+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 shutdown_connections 2024-09-21T01:56:05.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.536+0000 7fc8339af740 1 -- 172.21.15.29:0/4012532345 wait complete. 2024-09-21T01:56:05.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.536+0000 7fc8339af740 20 client.26773 trim_cache size 0 max 16384 2024-09-21T01:56:05.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.552+0000 7fc8339af740 1 Processor -- start 2024-09-21T01:56:05.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.552+0000 7fc8339af740 1 -- start start 2024-09-21T01:56:05.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.553+0000 7fc8339af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867b99f0 0x55c586ba0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.553+0000 7fc8339af740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 0x55c586bc2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.553+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867b99f0 0x55c586ba0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8267fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 0x55c586bc2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8267fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 0x55c586bc2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59380/0 (socket says 172.21.15.29:59380) 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8267fc640 1 -- 172.21.15.29:0/632333781 learned_addr learned my addr 172.21.15.29:0/632333781 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8339af740 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586ba53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ad1f0 con 0x55c5867d21b0 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281aedf0 con 0x55c5867b99f0 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ad8c0 con 0x55c5867da390 2024-09-21T01:56:05.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.554+0000 7fc8277fe640 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586ba53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 -- 172.21.15.29:0/632333781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867b99f0 msgr2=0x55c586ba0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867b99f0 0x55c586ba0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 -- 172.21.15.29:0/632333781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 msgr2=0x55c586bc2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 0x55c586bc2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 -- 172.21.15.29:0/632333781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c586baecd0 con 0x55c5867da390 2024-09-21T01:56:05.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.555+0000 7fc8277fe640 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586ba53c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc8200107b0 tx=0x7fc820010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6108987bf284f1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.556+0000 7fc80cff9640 1 -- 172.21.15.29:0/632333781 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820003970 con 0x55c5867da390 2024-09-21T01:56:05.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.556+0000 7fc80cff9640 1 -- 172.21.15.29:0/632333781 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82000e930 con 0x55c5867da390 2024-09-21T01:56:05.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.556+0000 7fc80cff9640 1 -- 172.21.15.29:0/632333781 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820005770 con 0x55c5867da390 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 msgr2=0x55c586ba53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586ba53c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc8200107b0 tx=0x7fc820010f60 comp rx=0 tx=0).stop 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 shutdown_connections 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586ba53c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867d21b0 0x55c586bc2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 --2- 172.21.15.29:0/632333781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867b99f0 0x55c586ba0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 >> 172.21.15.29:0/632333781 conn(0x55c5867ef460 msgr2=0x55c586b9fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 shutdown_connections 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- 172.21.15.29:0/632333781 wait complete. 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 -- start start 2024-09-21T01:56:05.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.557+0000 7fc8339af740 1 Processor -- start 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 0x55c586bba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bbae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ae720 con 0x55c5867b99f0 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ae050 con 0x55c5867d21b0 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ab1f0 con 0x55c5867da390 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8339af740 10 client.0 _ll_register_callbacks cb 0x7fc81c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 0x55c586bba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49574/0 (socket says 172.21.15.29:49574) 2024-09-21T01:56:05.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 -- 172.21.15.29:0/3755592576 learned_addr learned my addr 172.21.15.29:0/3755592576 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8267fc640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bbae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 -- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 msgr2=0x55c586bbae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bbae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 -- 172.21.15.29:0/3755592576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 msgr2=0x55c586bba8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 0x55c586bba8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8277fe640 1 -- 172.21.15.29:0/3755592576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c586c323e0 con 0x55c5867da390 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc826ffd640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 0x55c586bba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.558+0000 7fc8267fc640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bbae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc8277fe640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc820000f80 tx=0x7fc82000eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c5820c6aa44ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc80aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820003b70 con 0x55c5867da390 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82000e930 con 0x55c5867da390 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc820003d10 con 0x55c5867da390 2024-09-21T01:56:05.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc8200053f0 con 0x55c5867da390 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc8339af740 10 client.26779 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.559+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c586c334a0 con 0x55c5867da390 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82000e930 con 0x55c5867da390 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc82002e030 con 0x55c5867da390 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 10 client.26779 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 10 client.26779 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 20 client.26779 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867fb070 0x55c586c35930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 4 client.26779 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c586b168a0 con 0x55c5867fb070 2024-09-21T01:56:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.560+0000 7fc826ffd640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867fb070 0x55c586c35930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc80aff5640 10 client.26779 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc826ffd640 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867fb070 0x55c586c35930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12ebb72216d9aae3 server_cookie=31ead62699785121 in_seq=0 out_seq=0 2024-09-21T01:56:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc80aff5640 1 -- 172.21.15.29:0/3755592576 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c586b168a0 con 0x55c5867fb070 2024-09-21T01:56:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc80aff5640 10 client.26779 handle_command_reply: tid=0 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 client.26779 shutdown 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 msgr2=0x55c586c31ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc820000f80 tx=0x7fc82000eb70 comp rx=0 tx=0).stop 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 shutdown_connections 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55c5867fb070 0x55c586c35930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5867da390 0x55c586c31ea0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5867d21b0 0x55c586bbae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.561+0000 7fc8339af740 1 --2- 172.21.15.29:0/3755592576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c5867b99f0 0x55c586bba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 >> 172.21.15.29:0/3755592576 conn(0x55c5867ef460 msgr2=0x55c586ba53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc80aff5640 0 client.26779 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 shutdown_connections 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc8339af740 1 -- 172.21.15.29:0/3755592576 wait complete. 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc8339af740 20 client.26779 trim_cache size 0 max 16384 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 msgr2=0x7fc8100b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 0x7fc8100b8a20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818005a20 comp rx=0 tx=0).stop 2024-09-21T01:56:05.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 msgr2=0x7fc828102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.562+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 0x7fc828102c30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc82806ad30 tx=0x7fc81c00ee50 comp rx=0 tx=0).stop 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 shutdown_connections 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc8100b6510 0x7fc8100b8a20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82810de10 0x7fc828103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc828105300 0x7fc828102c30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 --2- 172.21.15.29:0/155354926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8281049b0 0x7fc8281026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 >> 172.21.15.29:0/155354926 conn(0x7fc828079730 msgr2=0x7fc8281007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 shutdown_connections 2024-09-21T01:56:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:05.563+0000 7fc82d43b640 1 -- 172.21.15.29:0/155354926 wait complete. 2024-09-21T01:56:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:05 smithi029 ceph-mon[31228]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-09-21T01:56:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:05 smithi066 ceph-mon[35001]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-09-21T01:56:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:05 smithi190 ceph-mon[34368]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-09-21T01:56:06.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:56:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 -- 172.21.15.29:0/3496613603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a201072f0 msgr2=0x7f5a2010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 --2- 172.21.15.29:0/3496613603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a201072f0 0x7f5a2010dc90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009ca0 tx=0x7f5a1802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 -- 172.21.15.29:0/3496613603 shutdown_connections 2024-09-21T01:56:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 --2- 172.21.15.29:0/3496613603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a201072f0 0x7f5a2010dc90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 --2- 172.21.15.29:0/3496613603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a20106950 0x7f5a20106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 --2- 172.21.15.29:0/3496613603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201058d0 0x7f5a20105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 -- 172.21.15.29:0/3496613603 >> 172.21.15.29:0/3496613603 conn(0x7f5a201011f0 msgr2=0x7f5a201036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 -- 172.21.15.29:0/3496613603 shutdown_connections 2024-09-21T01:56:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.139+0000 7f5a2555e640 1 -- 172.21.15.29:0/3496613603 wait complete. 2024-09-21T01:56:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 Processor -- start 2024-09-21T01:56:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 -- start start 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 0x7f5a2019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 0x7f5a2019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20111570 con 0x7f5a20106950 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201113f0 con 0x7f5a201072f0 2024-09-21T01:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a2555e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201116f0 con 0x7f5a201058d0 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.140+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 0x7f5a2019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 0x7f5a2019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49586/0 (socket says 172.21.15.29:49586) 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 0x7f5a2019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38912/0 (socket says 172.21.15.29:38912) 2024-09-21T01:56:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 0x7f5a2019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59392/0 (socket says 172.21.15.29:59392) 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 -- 172.21.15.29:0/2284771661 learned_addr learned my addr 172.21.15.29:0/2284771661 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 -- 172.21.15.29:0/2284771661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 msgr2=0x7f5a2019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 0x7f5a2019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 -- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 msgr2=0x7f5a2019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 0x7f5a2019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 -- 172.21.15.29:0/2284771661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2019db40 con 0x7f5a201058d0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a1effd640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000ea60 tx=0x7f5a1000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.141+0000 7f5a07fff640 1 -- 172.21.15.29:0/2284771661 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1000cde0 con 0x7f5a201058d0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.142+0000 7f5a07fff640 1 -- 172.21.15.29:0/2284771661 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a100035f0 con 0x7f5a201058d0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.142+0000 7f5a07fff640 1 -- 172.21.15.29:0/2284771661 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10010690 con 0x7f5a201058d0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.144+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2019de30 con 0x7f5a201058d0 2024-09-21T01:56:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.144+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201a4520 con 0x7f5a201058d0 2024-09-21T01:56:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.147+0000 7f5a2c4cc740 1 Processor -- start 2024-09-21T01:56:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 -- start start 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f991cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f992820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55879f8fb6f0 con 0x55879f5c04d0 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55879f8eab20 con 0x55879f5e11b0 2024-09-21T01:56:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.148+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55879f8ee930 con 0x55879f5b82f0 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38918/0 (socket says 172.21.15.29:38918) 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/3757575195 learned_addr learned my addr 172.21.15.29:0/3757575195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f992820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/3757575195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 msgr2=0x55879f991cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:56:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f991cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/3757575195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 msgr2=0x55879f992820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f992820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a07fff640 1 -- 172.21.15.29:0/2284771661 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5a100037a0 con 0x7f5a201058d0 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/3757575195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879f8b0530 con 0x55879f5e11b0 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.149+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a18035000 tx=0x7f5a1802f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db0cee48efb5d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a057fa640 1 -- 172.21.15.29:0/3757575195 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18031650 con 0x55879f5e11b0 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a057fa640 1 -- 172.21.15.29:0/3757575195 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a18036030 con 0x55879f5e11b0 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a057fa640 1 -- 172.21.15.29:0/3757575195 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004790 con 0x55879f5e11b0 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a07fff640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 0x7f5a080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a07fff640 1 -- 172.21.15.29:0/2284771661 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a100e7520 con 0x7f5a201058d0 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.150+0000 7f5a1effd640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 0x7f5a080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/3757575195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 msgr2=0x55879f991660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a18035000 tx=0x7f5a1802f450 comp rx=0 tx=0).stop 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a1effd640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 0x7f5a080b8aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009920 tx=0x7f5a10003e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/3757575195 shutdown_connections 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f992820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f991cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/3757575195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f991660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/3757575195 >> 172.21.15.29:0/3757575195 conn(0x55879f5d5300 msgr2=0x55879f9848e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/3757575195 shutdown_connections 2024-09-21T01:56:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.151+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/3757575195 wait complete. 2024-09-21T01:56:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.152+0000 7f5a2c4cc740 1 -- start start 2024-09-21T01:56:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.152+0000 7f5a2c4cc740 1 Processor -- start 2024-09-21T01:56:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.152+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.152+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f99b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38934/0 (socket says 172.21.15.29:38934) 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/4010680237 learned_addr learned my addr 172.21.15.29:0/4010680237 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f99bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55879f822ab0 con 0x55879f5c04d0 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f99b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55879f890730 con 0x55879f5b82f0 2024-09-21T01:56:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55879f993160 con 0x55879f5e11b0 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.153+0000 7f5a2c4cc740 10 client.0 _ll_register_callbacks cb 0x55879f8e46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.154+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f99bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/4010680237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 msgr2=0x55879f99bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f99bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/4010680237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 msgr2=0x55879f99b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f99b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879f64c790 con 0x55879f5b82f0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004270 tx=0x7f5a18009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c325693a8dc8a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18058060 con 0x55879f5b82f0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a18003d80 con 0x55879f5b82f0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.155+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a18036030 con 0x55879f5b82f0 2024-09-21T01:56:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5a1804a040 con 0x55879f5b82f0 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 10 client.26786 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55879f6c5620 con 0x55879f5b82f0 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a18036030 con 0x55879f5b82f0 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f5a18050070 con 0x55879f5b82f0 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 10 client.26786 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 10 client.26786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 20 client.26786 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f59fd80 0x55879f9a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 4 client.26786 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.156+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55879f8fd640 con 0x55879f59fd80 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.157+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f59fd80 0x55879f9a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.157+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f59fd80 0x55879f9a8000 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1360eb8a32c11eb server_cookie=54743cbae4ae9c48 in_seq=0 out_seq=0 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.157+0000 7f5a02ff5640 10 client.26786 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a02ff5640 1 -- 172.21.15.29:0/4010680237 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55879f8fd640 con 0x55879f59fd80 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a02ff5640 10 client.26786 handle_command_reply: tid=0 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 client.26786 shutdown 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 msgr2=0x55879f99ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004270 tx=0x7f5a18009540 comp rx=0 tx=0).stop 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 shutdown_connections 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f59fd80 0x55879f9a8000 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f99bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f99b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/4010680237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f99ac90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 >> 172.21.15.29:0/4010680237 conn(0x55879f5d5300 msgr2=0x55879f982ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a02ff5640 0 client.26786 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 shutdown_connections 2024-09-21T01:56:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/4010680237 wait complete. 2024-09-21T01:56:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.158+0000 7f5a2c4cc740 20 client.26786 trim_cache size 0 max 16384 2024-09-21T01:56:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.180+0000 7f5a2c4cc740 1 Processor -- start 2024-09-21T01:56:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.180+0000 7f5a2c4cc740 1 -- start start 2024-09-21T01:56:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f985a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f9833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201a8c70 con 0x55879f5c04d0 2024-09-21T01:56:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201aa870 con 0x55879f5b82f0 2024-09-21T01:56:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201a9340 con 0x55879f5e11b0 2024-09-21T01:56:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f985a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f985a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38950/0 (socket says 172.21.15.29:38950) 2024-09-21T01:56:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2468821313 learned_addr learned my addr 172.21.15.29:0/2468821313 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59420/0 (socket says 172.21.15.29:59420) 2024-09-21T01:56:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f9833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.181+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2468821313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 msgr2=0x55879f985a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f985a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2468821313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 msgr2=0x55879f982ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2468821313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879f97ff50 con 0x55879f5e11b0 2024-09-21T01:56:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f9833e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a1802ade0 tx=0x7f5a18004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534b927a3fce62f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.182+0000 7f5a04ff9640 1 -- 172.21.15.29:0/2468821313 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18062040 con 0x55879f5e11b0 2024-09-21T01:56:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.183+0000 7f5a04ff9640 1 -- 172.21.15.29:0/2468821313 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a18036070 con 0x55879f5e11b0 2024-09-21T01:56:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.183+0000 7f5a04ff9640 1 -- 172.21.15.29:0/2468821313 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18058040 con 0x55879f5e11b0 2024-09-21T01:56:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2468821313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 msgr2=0x55879f9833e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f9833e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a1802ade0 tx=0x7f5a18004840 comp rx=0 tx=0).stop 2024-09-21T01:56:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2468821313 shutdown_connections 2024-09-21T01:56:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5e11b0 0x55879f9833e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f982ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2468821313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5b82f0 0x55879f985a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2468821313 >> 172.21.15.29:0/2468821313 conn(0x55879f5d5300 msgr2=0x55879f985340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2468821313 shutdown_connections 2024-09-21T01:56:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2468821313 wait complete. 2024-09-21T01:56:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 -- start start 2024-09-21T01:56:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.185+0000 7f5a2c4cc740 1 Processor -- start 2024-09-21T01:56:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f9a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f9a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f9a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201aa1a0 con 0x55879f5c04d0 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201a9ad0 con 0x55879f5e11b0 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201a6c50 con 0x55879f5b82f0 2024-09-21T01:56:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a2c4cc740 10 client.0 _ll_register_callbacks cb 0x7f5a1802f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f9a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f9a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f9a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f9a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59428/0 (socket says 172.21.15.29:59428) 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1e7fc640 1 -- 172.21.15.29:0/2303939565 learned_addr learned my addr 172.21.15.29:0/2303939565 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 msgr2=0x55879f9a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f9a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2303939565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 msgr2=0x55879f9a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.186+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f9a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a1f7fe640 1 -- 172.21.15.29:0/2303939565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879f9a1a90 con 0x55879f5b82f0 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a1f7fe640 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f9a2320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a18030760 tx=0x7f5a18005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=846eb8b670891ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18058070 con 0x55879f5b82f0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a180308d0 con 0x55879f5b82f0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a18036070 con 0x55879f5b82f0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5a18030ce0 con 0x55879f5b82f0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a2c4cc740 10 client.26794 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.187+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55879f9a08c0 con 0x55879f5b82f0 2024-09-21T01:56:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a18036070 con 0x55879f5b82f0 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f5a18050050 con 0x55879f5b82f0 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a2c4cc740 10 client.26794 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a2c4cc740 10 client.26794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a2c4cc740 20 client.26794 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f5751c0 0x55879fa14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.188+0000 7f5a2c4cc740 4 client.26794 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.189+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55879f8f90f0 con 0x55879f5751c0 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.189+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f5751c0 0x55879fa14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.190+0000 7f5a02ff5640 10 client.26794 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.190+0000 7f5a1e7fc640 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f5751c0 0x55879fa14fd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6adde4d8f040e797 server_cookie=9d6e3315e6017363 in_seq=0 out_seq=0 2024-09-21T01:56:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a02ff5640 1 -- 172.21.15.29:0/2303939565 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55879f8f90f0 con 0x55879f5751c0 2024-09-21T01:56:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a02ff5640 10 client.26794 handle_command_reply: tid=0 2024-09-21T01:56:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 client.26794 shutdown 2024-09-21T01:56:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 msgr2=0x55879f9a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f9a2320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a18030760 tx=0x7f5a18005df0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 shutdown_connections 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55879f5751c0 0x55879fa14fd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55879f5e11b0 0x55879f9a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55879f5c04d0 0x55879f9a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 --2- 172.21.15.29:0/2303939565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55879f5b82f0 0x55879f9a2320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.200+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 >> 172.21.15.29:0/2303939565 conn(0x55879f5d5300 msgr2=0x55879f985c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.201+0000 7f5a02ff5640 0 client.26794 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.202+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 shutdown_connections 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.202+0000 7f5a2c4cc740 1 -- 172.21.15.29:0/2303939565 wait complete. 2024-09-21T01:56:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.202+0000 7f5a2c4cc740 20 client.26794 trim_cache size 0 max 16384 2024-09-21T01:56:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:56:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:56:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:56:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:56:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 msgr2=0x7f5a080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 0x7f5a080b8aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009920 tx=0x7f5a10003e90 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 msgr2=0x7f5a2019c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000ea60 tx=0x7f5a1000ef20 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 shutdown_connections 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a080b6590 0x7f5a080b8aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a201072f0 0x7f5a2019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a20106950 0x7f5a2019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 --2- 172.21.15.29:0/2284771661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a201058d0 0x7f5a2019c100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.203+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 >> 172.21.15.29:0/2284771661 conn(0x7f5a201011f0 msgr2=0x7f5a20109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.204+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 shutdown_connections 2024-09-21T01:56:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:07.204+0000 7f5a2555e640 1 -- 172.21.15.29:0/2284771661 wait complete. 2024-09-21T01:56:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 450.0 s, 1891.2 tps, lat 52.489 ms stddev 58.572 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 -- 172.21.15.29:0/4082887191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc811cba0 msgr2=0x7f8dc811efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/4082887191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc811cba0 0x7f8dc811efc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009860 tx=0x7f8db802a360 comp rx=0 tx=0).stop 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 -- 172.21.15.29:0/4082887191 shutdown_connections 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/4082887191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc811cba0 0x7f8dc811efc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/4082887191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc810dbf0 0x7f8dc810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/4082887191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc8111cd0 0x7f8dc810d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.995+0000 7f8dcd92a640 1 -- 172.21.15.29:0/4082887191 >> 172.21.15.29:0/4082887191 conn(0x7f8dc806e9f0 msgr2=0x7f8dc806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.998+0000 7f8dcd92a640 1 -- 172.21.15.29:0/4082887191 shutdown_connections 2024-09-21T01:56:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.998+0000 7f8dcd92a640 1 -- 172.21.15.29:0/4082887191 wait complete. 2024-09-21T01:56:08.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:07.999+0000 7f8dcd92a640 1 Processor -- start 2024-09-21T01:56:08.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 -- start start 2024-09-21T01:56:08.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 0x7f8dc8118760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc811cba0 0x7f8dc8118d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8114c00 con 0x7f8dc810dbf0 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8114a80 con 0x7f8dc811cba0 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd92a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8114d80 con 0x7f8dc8111cd0 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59442/0 (socket says 172.21.15.29:59442) 2024-09-21T01:56:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcc928640 1 -- 172.21.15.29:0/327044404 learned_addr learned my addr 172.21.15.29:0/327044404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 0x7f8dc8118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.000+0000 7f8dcd129640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc811cba0 0x7f8dc8118d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.003 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:07 smithi029 ceph-mon[31228]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 755 op/s 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dcc928640 1 -- 172.21.15.29:0/327044404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 msgr2=0x7f8dc8118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dcc928640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 0x7f8dc8118760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dcc928640 1 -- 172.21.15.29:0/327044404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc811cba0 msgr2=0x7f8dc8118d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dcc928640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc811cba0 0x7f8dc8118d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dcc928640 1 -- 172.21.15.29:0/327044404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc8119550 con 0x7f8dc810dbf0 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.001+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 0x7f8dc8118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dcc928640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001fd60 tx=0x7f8dc001fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dc5ffb640 1 -- 172.21.15.29:0/327044404 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc001dbb0 con 0x7f8dc810dbf0 2024-09-21T01:56:08.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dc5ffb640 1 -- 172.21.15.29:0/327044404 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc0003e20 con 0x7f8dc810dbf0 2024-09-21T01:56:08.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dc5ffb640 1 -- 172.21.15.29:0/327044404 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0020410 con 0x7f8dc810dbf0 2024-09-21T01:56:08.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc81acd20 con 0x7f8dc810dbf0 2024-09-21T01:56:08.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.002+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81ad1d0 con 0x7f8dc810dbf0 2024-09-21T01:56:08.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.007+0000 7f8dd5100740 1 Processor -- start 2024-09-21T01:56:08.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.007+0000 7f8dd5100740 1 -- start start 2024-09-21T01:56:08.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dc5ffb640 1 -- 172.21.15.29:0/327044404 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8dc0004030 con 0x7f8dc810dbf0 2024-09-21T01:56:08.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f3471769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 0x55f347176ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f3470ecea0 con 0x55f346dcf070 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3470dc2d0 con 0x55f346dae390 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dd5100740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f3470e00e0 con 0x55f346da61b0 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dcd129640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dcd129640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59452/0 (socket says 172.21.15.29:59452) 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.008+0000 7f8dcd129640 1 -- 172.21.15.29:0/4238856673 learned_addr learned my addr 172.21.15.29:0/4238856673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.009+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 0x55f347176ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.009+0000 7f8dcc928640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f3471769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.009+0000 7f8dc5ffb640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 0x7f8db00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcd129640 1 -- 172.21.15.29:0/4238856673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 msgr2=0x55f3471769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcd129640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f3471769a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcd129640 1 -- 172.21.15.29:0/4238856673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 msgr2=0x55f347176ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcd129640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 0x55f347176ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcd129640 1 -- 172.21.15.29:0/4238856673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f346e3a650 con 0x55f346dcf070 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 0x55f347176ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:08.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.010+0000 7f8dcc928640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f3471769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:08.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.011+0000 7f8dcd129640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 0x7f8db00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.012+0000 7f8dcd129640 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009860 tx=0x7f8db8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9cb9ffc27f37cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.012+0000 7f8dc5ffb640 1 -- 172.21.15.29:0/327044404 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8dc00f7dc0 con 0x7f8dc810dbf0 2024-09-21T01:56:08.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.013+0000 7f8dcd129640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 0x7f8db00b8bb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8db804a000 tx=0x7f8db804d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.014+0000 7f8daeffd640 1 -- 172.21.15.29:0/4238856673 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8044040 con 0x55f346dcf070 2024-09-21T01:56:08.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.014+0000 7f8daeffd640 1 -- 172.21.15.29:0/4238856673 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db8038040 con 0x55f346dcf070 2024-09-21T01:56:08.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.014+0000 7f8daeffd640 1 -- 172.21.15.29:0/4238856673 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8037070 con 0x55f346dcf070 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 -- 172.21.15.29:0/4238856673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 msgr2=0x55f347176460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009860 tx=0x7f8db8009580 comp rx=0 tx=0).stop 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 -- 172.21.15.29:0/4238856673 shutdown_connections 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dae390 0x55f347176ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f3471769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.015+0000 7f8dd5100740 1 --2- 172.21.15.29:0/4238856673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f347176460 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.016+0000 7f8dd5100740 1 -- 172.21.15.29:0/4238856673 >> 172.21.15.29:0/4238856673 conn(0x55f346dc3460 msgr2=0x55f347174350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.016+0000 7f8dd5100740 1 -- 172.21.15.29:0/4238856673 shutdown_connections 2024-09-21T01:56:08.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.016+0000 7f8dd5100740 1 -- 172.21.15.29:0/4238856673 wait complete. 2024-09-21T01:56:08.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.016+0000 7f8dd5100740 1 -- start start 2024-09-21T01:56:08.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.017+0000 7f8dd5100740 1 Processor -- start 2024-09-21T01:56:08.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.017+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.017+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f347182430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49646/0 (socket says 172.21.15.29:49646) 2024-09-21T01:56:08.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 -- 172.21.15.29:0/3927904497 learned_addr learned my addr 172.21.15.29:0/3927904497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:08.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dcf070 0x55f34718df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f346f3d700 con 0x55f346dae390 2024-09-21T01:56:08.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f347077160 con 0x55f346dcf070 2024-09-21T01:56:08.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f347184c40 con 0x55f346da61b0 2024-09-21T01:56:08.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcd129640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dcf070 0x55f34718df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 -- 172.21.15.29:0/3927904497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dcf070 msgr2=0x55f34718df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dcf070 0x55f34718df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 -- 172.21.15.29:0/3927904497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 msgr2=0x55f347182430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f347182430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.018+0000 7f8dcc928640 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3470a4c90 con 0x55f346da61b0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.019+0000 7f8dd5100740 10 client.0 _ll_register_callbacks cb 0x55f3470909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.019+0000 7f8dcc928640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001fd30 tx=0x7f8dc00043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d9cee787dac54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.019+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db8038040 con 0x55f346da61b0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.020+0000 7f8da7fff640 10 client.26803 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.020+0000 7f8da7fff640 1 -- 172.21.15.29:0/3927904497 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00bd6d0 con 0x55f346da61b0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.020+0000 7f8da7fff640 1 -- 172.21.15.29:0/3927904497 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8dc00b98b0 con 0x55f346da61b0 2024-09-21T01:56:08.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.020+0000 7f8da7fff640 1 -- 172.21.15.29:0/3927904497 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8dc00ba5b0 con 0x55f346da61b0 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 10 client.26803 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55f347190a00 con 0x55f346da61b0 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8da7fff640 1 -- 172.21.15.29:0/3927904497 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8dc00e51a0 con 0x55f346da61b0 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 10 client.26803 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 10 client.26803 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 20 client.26803 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d8d9f0 0x55f347193de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 4 client.26803 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8db8038040 con 0x55f346d8d9f0 2024-09-21T01:56:08.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.021+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d8d9f0 0x55f347193de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.022+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d8d9f0 0x55f347193de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=683d51af82881e66 server_cookie=a497848e1ca3b111 in_seq=0 out_seq=0 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.022+0000 7f8da7fff640 10 client.26803 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8da7fff640 1 -- 172.21.15.29:0/3927904497 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8db8038040 con 0x55f346d8d9f0 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8da7fff640 10 client.26803 handle_command_reply: tid=0 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 client.26803 shutdown 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 msgr2=0x55f347181ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001fd30 tx=0x7f8dc00043d0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 shutdown_connections 2024-09-21T01:56:08.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346dcf070 0x55f34718df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f347182430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d8d9f0 0x55f347193de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3927904497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346da61b0 0x55f347181ef0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 >> 172.21.15.29:0/3927904497 conn(0x55f346dc3460 msgr2=0x55f347179640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8da7fff640 0 client.26803 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 shutdown_connections 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 1 -- 172.21.15.29:0/3927904497 wait complete. 2024-09-21T01:56:08.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.023+0000 7f8dd5100740 20 client.26803 trim_cache size 0 max 16384 2024-09-21T01:56:08.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.043+0000 7f8dd5100740 1 Processor -- start 2024-09-21T01:56:08.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.043+0000 7f8dd5100740 1 -- start start 2024-09-21T01:56:08.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 0x55f34717eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f34717f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b5880 con 0x55f346dcf070 2024-09-21T01:56:08.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b7480 con 0x55f346da61b0 2024-09-21T01:56:08.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dd5100740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b5f50 con 0x55f346dae390 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dcc928640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39010/0 (socket says 172.21.15.29:39010) 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dcc928640 1 -- 172.21.15.29:0/3592579288 learned_addr learned my addr 172.21.15.29:0/3592579288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 0x55f34717eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dcd129640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f34717f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 -- 172.21.15.29:0/3592579288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 msgr2=0x55f347183bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 -- 172.21.15.29:0/3592579288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 msgr2=0x55f34717f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f34717f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dc7fff640 1 -- 172.21.15.29:0/3592579288 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f347178750 con 0x55f346dae390 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.044+0000 7f8dcc928640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:08.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.045+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 0x55f34717eee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55f347194550 tx=0x7f8dbc009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb40955e2d27566 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.045+0000 7f8dae7fc640 1 -- 172.21.15.29:0/3592579288 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc016c30 con 0x55f346dae390 2024-09-21T01:56:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.045+0000 7f8dae7fc640 1 -- 172.21.15.29:0/3592579288 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dbc022070 con 0x55f346dae390 2024-09-21T01:56:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.045+0000 7f8dae7fc640 1 -- 172.21.15.29:0/3592579288 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc01c540 con 0x55f346dae390 2024-09-21T01:56:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- 172.21.15.29:0/3592579288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 msgr2=0x55f34717eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 0x55f34717eee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55f347194550 tx=0x7f8dbc009110 comp rx=0 tx=0).stop 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- 172.21.15.29:0/3592579288 shutdown_connections 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dcf070 0x55f34717f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dae390 0x55f34717eee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 --2- 172.21.15.29:0/3592579288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347183bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- 172.21.15.29:0/3592579288 >> 172.21.15.29:0/3592579288 conn(0x55f346dc3460 msgr2=0x55f347179bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- 172.21.15.29:0/3592579288 shutdown_connections 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- 172.21.15.29:0/3592579288 wait complete. 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 -- start start 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.046+0000 7f8dd5100740 1 Processor -- start 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347188190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f3471886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 0x55f3471fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b6db0 con 0x55f346dae390 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b66e0 con 0x55f346da61b0 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc81b3860 con 0x55f346dcf070 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dd5100740 10 client.0 _ll_register_callbacks cb 0x7f8dc00b9ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dcd129640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 0x55f3471fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dcd129640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 0x55f3471fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49662/0 (socket says 172.21.15.29:49662) 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dcd129640 1 -- 172.21.15.29:0/2905930211 learned_addr learned my addr 172.21.15.29:0/2905930211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:08.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f3471886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.052 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:07 smithi066 ceph-mon[35001]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 755 op/s 2024-09-21T01:56:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.047+0000 7f8dcc928640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347188190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 -- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 msgr2=0x55f3471fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 0x55f3471fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 -- 172.21.15.29:0/2905930211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 msgr2=0x55f3471886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f3471886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 -- 172.21.15.29:0/2905930211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f347188f70 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f3471886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8dcc928640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347188190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00bdc60 tx=0x7f8dc00f57a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca2ee6a2719c950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.048+0000 7f8da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00bd6d0 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8dc00b98b0 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dbc022070 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8dc00bda70 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8dd5100740 10 client.26810 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.049+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f3471fed80 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.050+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dbc022070 con 0x55f346da61b0 2024-09-21T01:56:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.050+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8dc00b9ab0 con 0x55f346da61b0 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.050+0000 7f8dd5100740 10 client.26810 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.050+0000 7f8dd5100740 10 client.26810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.050+0000 7f8dd5100740 20 client.26810 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d62da0 0x55f347201450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dd5100740 4 client.26810 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3470eedf0 con 0x55f346d62da0 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d62da0 0x55f347201450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8da7fff640 10 client.26810 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dc7fff640 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d62da0 0x55f347201450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3db7f67dfa507d07 server_cookie=523d4a10229ef394 in_seq=0 out_seq=0 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8da7fff640 1 -- 172.21.15.29:0/2905930211 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3470eedf0 con 0x55f346d62da0 2024-09-21T01:56:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8da7fff640 10 client.26810 handle_command_reply: tid=0 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.051+0000 7f8dd5100740 1 client.26810 shutdown 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 msgr2=0x55f347188190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347188190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00bdc60 tx=0x7f8dc00f57a0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 shutdown_connections 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55f346d62da0 0x55f347201450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8da7fff640 0 client.26810 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f346dcf070 0x55f3471fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f346dae390 0x55f3471886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 --2- 172.21.15.29:0/2905930211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f346da61b0 0x55f347188190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 >> 172.21.15.29:0/2905930211 conn(0x55f346dc3460 msgr2=0x55f347179bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 shutdown_connections 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 1 -- 172.21.15.29:0/2905930211 wait complete. 2024-09-21T01:56:08.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.052+0000 7f8dd5100740 20 client.26810 trim_cache size 0 max 16384 2024-09-21T01:56:08.058 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:08.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 msgr2=0x7f8db00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 0x7f8db00b8bb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8db804a000 tx=0x7f8db804d000 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 msgr2=0x7f8dc811c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001fd60 tx=0x7f8dc001fe50 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 shutdown_connections 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8db00b66a0 0x7f8db00b8bb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dc811cba0 0x7f8dc8118d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dc8111cd0 0x7f8dc8118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 --2- 172.21.15.29:0/327044404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dc810dbf0 0x7f8dc811c480 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 >> 172.21.15.29:0/327044404 conn(0x7f8dc806e9f0 msgr2=0x7f8dc8110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 shutdown_connections 2024-09-21T01:56:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:08.053+0000 7f8dcd92a640 1 -- 172.21.15.29:0/327044404 wait complete. 2024-09-21T01:56:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:07 smithi190 ceph-mon[34368]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 755 op/s 2024-09-21T01:56:08.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:56:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:09.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:09 smithi029 ceph-mon[31228]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.032+0000 7f1d43cc6640 1 -- 172.21.15.29:0/4004162513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c108cf0 msgr2=0x7f1d3c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.032+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/4004162513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c10f520 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009ca0 tx=0x7f1d3402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.033+0000 7f1d43cc6640 1 -- 172.21.15.29:0/4004162513 shutdown_connections 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.033+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/4004162513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c10f520 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.033+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/4004162513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c1083a0 0x7f1d3c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.033+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/4004162513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c102a60 0x7f1d3c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.033+0000 7f1d43cc6640 1 -- 172.21.15.29:0/4004162513 >> 172.21.15.29:0/4004162513 conn(0x7f1d3c079730 msgr2=0x7f1d3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.035+0000 7f1d43cc6640 1 -- 172.21.15.29:0/4004162513 shutdown_connections 2024-09-21T01:56:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.035+0000 7f1d43cc6640 1 -- 172.21.15.29:0/4004162513 wait complete. 2024-09-21T01:56:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.036+0000 7f1d43cc6640 1 Processor -- start 2024-09-21T01:56:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.036+0000 7f1d43cc6640 1 -- start start 2024-09-21T01:56:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 0x7f1d3c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c1083a0 0x7f1d3c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c112960 con 0x7f1d3c1083a0 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c1127e0 con 0x7f1d3c102a60 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d43cc6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c112ae0 con 0x7f1d3c108cf0 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 0x7f1d3c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 0x7f1d3c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39030/0 (socket says 172.21.15.29:39030) 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d41261640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c1083a0 0x7f1d3c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d41a62640 1 -- 172.21.15.29:0/527053216 learned_addr learned my addr 172.21.15.29:0/527053216 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 -- 172.21.15.29:0/527053216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 msgr2=0x7f1d3c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 0x7f1d3c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 -- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c1083a0 msgr2=0x7f1d3c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c1083a0 0x7f1d3c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.037+0000 7f1d42263640 1 -- 172.21.15.29:0/527053216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c19f2b0 con 0x7f1d3c108cf0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d42263640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c19ebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1d3402ade0 tx=0x7f1d34031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d32ffd640 1 -- 172.21.15.29:0/527053216 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34003b70 con 0x7f1d3c108cf0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d32ffd640 1 -- 172.21.15.29:0/527053216 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d34031910 con 0x7f1d3c108cf0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d32ffd640 1 -- 172.21.15.29:0/527053216 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402f8c0 con 0x7f1d3c108cf0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c19f540 con 0x7f1d3c108cf0 2024-09-21T01:56:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.038+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c06ae60 con 0x7f1d3c108cf0 2024-09-21T01:56:10.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.041+0000 7f1d32ffd640 1 -- 172.21.15.29:0/527053216 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1d3402fa60 con 0x7f1d3c108cf0 2024-09-21T01:56:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.044+0000 7f1d4a23a740 1 Processor -- start 2024-09-21T01:56:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.045+0000 7f1d4a23a740 1 -- start start 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.045+0000 7f1d32ffd640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 0x7f1d280b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.045+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.045+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 0x562875c68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562875bdeea0 con 0x5628758981b0 2024-09-21T01:56:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562875bce2d0 con 0x5628758c1070 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562875bd20e0 con 0x5628758a0390 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41261640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 0x7f1d280b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d32ffd640 1 -- 172.21.15.29:0/527053216 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d341080c0 con 0x7f1d3c108cf0 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41261640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 0x562875c68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41261640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 0x562875c68be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59510/0 (socket says 172.21.15.29:59510) 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41261640 1 -- 172.21.15.29:0/2691068962 learned_addr learned my addr 172.21.15.29:0/2691068962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39032/0 (socket says 172.21.15.29:39032) 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 -- 172.21.15.29:0/2691068962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 msgr2=0x562875c69120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c69120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 -- 172.21.15.29:0/2691068962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 msgr2=0x562875c68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 0x562875c68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 -- 172.21.15.29:0/2691068962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287592c650 con 0x5628758c1070 2024-09-21T01:56:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41261640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 0x7f1d280b8ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c011930 tx=0x7f1d2c009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.046+0000 7f1d41a62640 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009cf0 tx=0x7f1d38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c192d36616a4351 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.047+0000 7f1d27fff640 1 -- 172.21.15.29:0/2691068962 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38005220 con 0x5628758c1070 2024-09-21T01:56:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.047+0000 7f1d27fff640 1 -- 172.21.15.29:0/2691068962 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38004c50 con 0x5628758c1070 2024-09-21T01:56:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.047+0000 7f1d27fff640 1 -- 172.21.15.29:0/2691068962 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38019780 con 0x5628758c1070 2024-09-21T01:56:10.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:09 smithi066 ceph-mon[35001]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T01:56:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- 172.21.15.29:0/2691068962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 msgr2=0x562875c686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009cf0 tx=0x7f1d38011810 comp rx=0 tx=0).stop 2024-09-21T01:56:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- 172.21.15.29:0/2691068962 shutdown_connections 2024-09-21T01:56:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758981b0 0x562875c68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/2691068962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758c1070 0x562875c686a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- 172.21.15.29:0/2691068962 >> 172.21.15.29:0/2691068962 conn(0x5628758b5460 msgr2=0x562875c66590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- 172.21.15.29:0/2691068962 shutdown_connections 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- 172.21.15.29:0/2691068962 wait complete. 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 -- start start 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.048+0000 7f1d4a23a740 1 Processor -- start 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 0x562875c7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 0x562875c7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 0x562875c7cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562875a2f700 con 0x5628758c1070 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562875b69160 con 0x5628758a0390 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562875c7f5b0 con 0x5628758981b0 2024-09-21T01:56:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d4a23a740 10 client.0 _ll_register_callbacks cb 0x562875b829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 0x562875c7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d41261640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 0x562875c7c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d41a62640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 0x562875c7c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49702/0 (socket says 172.21.15.29:49702) 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d41a62640 1 -- 172.21.15.29:0/1079301918 learned_addr learned my addr 172.21.15.29:0/1079301918 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 0x562875c7cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 -- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 msgr2=0x562875c7c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 0x562875c7c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 -- 172.21.15.29:0/1079301918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 msgr2=0x562875c7c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 0x562875c7c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d42263640 1 -- 172.21.15.29:0/1079301918 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562875b96c90 con 0x5628758c1070 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.049+0000 7f1d41261640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 0x562875c7c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d42263640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 0x562875c7cf00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d34105c20 tx=0x7f1d340ce0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2303d139161b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340cdd70 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d340f1be0 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d38004c50 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1d340ce960 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d4a23a740 10 client.17040 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.050+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562875c8b2b0 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38004c50 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1d340cd4c0 con 0x5628758c1070 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 10 client.17040 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 20 client.17040 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56287587f9f0 0x562875c8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 4 client.17040 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562875be0df0 con 0x56287587f9f0 2024-09-21T01:56:10.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.051+0000 7f1d41a62640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56287587f9f0 0x562875c8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.052+0000 7f1d257fa640 10 client.17040 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:56:10.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.054+0000 7f1d41a62640 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56287587f9f0 0x562875c8e870 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa1c50b0497d7735 server_cookie=34141f49d9d4a21c in_seq=0 out_seq=0 2024-09-21T01:56:10.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.055+0000 7f1d257fa640 1 -- 172.21.15.29:0/1079301918 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562875be0df0 con 0x56287587f9f0 2024-09-21T01:56:10.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.055+0000 7f1d257fa640 10 client.17040 handle_command_reply: tid=0 2024-09-21T01:56:10.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.055+0000 7f1d4a23a740 1 client.17040 shutdown 2024-09-21T01:56:10.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 msgr2=0x562875c7cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 0x562875c7cf00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d34105c20 tx=0x7f1d340ce0b0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 shutdown_connections 2024-09-21T01:56:10.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56287587f9f0 0x562875c8e870 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758c1070 0x562875c7cf00 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758a0390 0x562875c7c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/1079301918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758981b0 0x562875c7c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d257fa640 0 client.17040 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 >> 172.21.15.29:0/1079301918 conn(0x5628758b5460 msgr2=0x562875c64bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 shutdown_connections 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.056+0000 7f1d4a23a740 1 -- 172.21.15.29:0/1079301918 wait complete. 2024-09-21T01:56:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.057+0000 7f1d4a23a740 20 client.17040 trim_cache size 0 max 16384 2024-09-21T01:56:10.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.080+0000 7f1d4a23a740 1 Processor -- start 2024-09-21T01:56:10.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.080+0000 7f1d4a23a740 1 -- start start 2024-09-21T01:56:10.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.080+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 0x562875c7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.080+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c79670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d42263640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d42263640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59534/0 (socket says 172.21.15.29:59534) 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d42263640 1 -- 172.21.15.29:0/444250121 learned_addr learned my addr 172.21.15.29:0/444250121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41261640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c79670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000e350 con 0x5628758a0390 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000ff50 con 0x5628758981b0 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000ea20 con 0x56287587f9f0 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 0x562875c7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 -- 172.21.15.29:0/444250121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 msgr2=0x562875c79670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c79670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 -- 172.21.15.29:0/444250121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 msgr2=0x562875c79bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41a62640 1 -- 172.21.15.29:0/444250121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562875c6e7f0 con 0x56287587f9f0 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d41261640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c79670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.081+0000 7f1d42263640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.082+0000 7f1d41a62640 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 0x562875c7e0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009cc0 tx=0x7f1d38005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e1a0a8a86ff87c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.082+0000 7f1d277fe640 1 -- 172.21.15.29:0/444250121 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3802d3f0 con 0x56287587f9f0 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.082+0000 7f1d277fe640 1 -- 172.21.15.29:0/444250121 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3802da40 con 0x56287587f9f0 2024-09-21T01:56:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.082+0000 7f1d277fe640 1 -- 172.21.15.29:0/444250121 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38019990 con 0x56287587f9f0 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 msgr2=0x562875c7e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 0x562875c7e0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009cc0 tx=0x7f1d38005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 shutdown_connections 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5628758a0390 0x562875c79bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c79670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/444250121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56287587f9f0 0x562875c7e0b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 >> 172.21.15.29:0/444250121 conn(0x5628758b5460 msgr2=0x562875c7d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 shutdown_connections 2024-09-21T01:56:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.083+0000 7f1d4a23a740 1 -- 172.21.15.29:0/444250121 wait complete. 2024-09-21T01:56:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.084+0000 7f1d4a23a740 1 -- start start 2024-09-21T01:56:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.084+0000 7f1d4a23a740 1 Processor -- start 2024-09-21T01:56:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 0x562875c808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c81370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000f880 con 0x56287587f9f0 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000f1b0 con 0x5628758981b0 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d4a23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2000c330 con 0x5628758a0390 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d41261640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d41261640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39058/0 (socket says 172.21.15.29:39058) 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d41261640 1 -- 172.21.15.29:0/504471803 learned_addr learned my addr 172.21.15.29:0/504471803 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.086+0000 7f1d42263640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c81370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.087+0000 7f1d4a23a740 10 client.0 _ll_register_callbacks cb 0x7f1d34009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:10.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.087+0000 7f1d41a62640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 0x562875c808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 -- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 msgr2=0x562875c81370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c81370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 -- 172.21.15.29:0/504471803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 msgr2=0x562875c808f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 0x562875c808f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 -- 172.21.15.29:0/504471803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562875cf8280 con 0x5628758981b0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41a62640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 0x562875c808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d41261640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ff60 tx=0x7f1d2c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44457b484a2cc770 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.088+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c026040 con 0x5628758981b0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d2c01a070 con 0x5628758981b0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3802da40 con 0x5628758981b0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1d2c01fe50 con 0x5628758981b0 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d4a23a740 10 client.26828 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:10.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562875c6eb00 con 0x5628758981b0 2024-09-21T01:56:10.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.089+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c0097a0 con 0x5628758981b0 2024-09-21T01:56:10.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1d2c0039f0 con 0x5628758981b0 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 10 client.26828 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 10 client.26828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 20 client.26828 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628758c1070 0x562875c71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 4 client.26828 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:10.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.090+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1d3802da40 con 0x5628758c1070 2024-09-21T01:56:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.099+0000 7f1d41a62640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628758c1070 0x562875c71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:10.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.108+0000 7f1d41a62640 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628758c1070 0x562875c71430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33788aba8fd64181 server_cookie=d9b06c0cb44d528b in_seq=0 out_seq=0 2024-09-21T01:56:10.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.109+0000 7f1d257fa640 10 client.26828 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:56:10.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.110+0000 7f1d257fa640 1 -- 172.21.15.29:0/504471803 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1d3802da40 con 0x5628758c1070 2024-09-21T01:56:10.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.110+0000 7f1d257fa640 10 client.26828 handle_command_reply: tid=0 2024-09-21T01:56:10.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.111+0000 7f1d4a23a740 1 client.26828 shutdown 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 msgr2=0x562875c80e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ff60 tx=0x7f1d2c005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 shutdown_connections 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5628758c1070 0x562875c71430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628758a0390 0x562875c81370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628758981b0 0x562875c80e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 --2- 172.21.15.29:0/504471803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56287587f9f0 0x562875c808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.115+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 >> 172.21.15.29:0/504471803 conn(0x5628758b5460 msgr2=0x562875c7d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.116+0000 7f1d257fa640 0 client.26828 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:56:10.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.117+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 shutdown_connections 2024-09-21T01:56:10.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.117+0000 7f1d4a23a740 1 -- 172.21.15.29:0/504471803 wait complete. 2024-09-21T01:56:10.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.120+0000 7f1d4a23a740 20 client.26828 trim_cache size 0 max 16384 2024-09-21T01:56:10.122 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:10.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.121+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 msgr2=0x7f1d280b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.121+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 0x7f1d280b8ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c011930 tx=0x7f1d2c009b60 comp rx=0 tx=0).stop 2024-09-21T01:56:10.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 msgr2=0x7f1d3c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:10.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c19ebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1d3402ade0 tx=0x7f1d34031660 comp rx=0 tx=0).stop 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 shutdown_connections 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d280b65c0 0x7f1d280b8ad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d3c108cf0 0x7f1d3c19ebb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c1083a0 0x7f1d3c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 --2- 172.21.15.29:0/527053216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c102a60 0x7f1d3c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 >> 172.21.15.29:0/527053216 conn(0x7f1d3c079730 msgr2=0x7f1d3c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.122+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 shutdown_connections 2024-09-21T01:56:10.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:10.123+0000 7f1d43cc6640 1 -- 172.21.15.29:0/527053216 wait complete. 2024-09-21T01:56:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:09 smithi190 ceph-mon[34368]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T01:56:11.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:56:11.576 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:11 smithi066 ceph-mon[35001]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T01:56:12.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:11 smithi029 ceph-mon[31228]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T01:56:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:11 smithi190 ceph-mon[34368]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T01:56:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 455.0 s, 2145.8 tps, lat 45.930 ms stddev 53.535 2024-09-21T01:56:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 -- 172.21.15.29:0/3683123798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd011cc30 msgr2=0x7fadd011f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3683123798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd011cc30 0x7fadd011f140 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009860 tx=0x7fadc802a340 comp rx=0 tx=0).stop 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 -- 172.21.15.29:0/3683123798 shutdown_connections 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3683123798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd011cc30 0x7fadd011f140 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3683123798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd010a9e0 0x7fadd010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3683123798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd0111f40 0x7fadd010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 -- 172.21.15.29:0/3683123798 >> 172.21.15.29:0/3683123798 conn(0x7fadd0070ec0 msgr2=0x7fadd00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.240+0000 7fadd62c5640 1 -- 172.21.15.29:0/3683123798 shutdown_connections 2024-09-21T01:56:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.241+0000 7fadd62c5640 1 -- 172.21.15.29:0/3683123798 wait complete. 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.241+0000 7fadd62c5640 1 Processor -- start 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 -- start start 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 0x7fadd01a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd011cc30 0x7fadd01aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01228b0 con 0x7fadd010a9e0 2024-09-21T01:56:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0122730 con 0x7fadd011cc30 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd62c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0122a30 con 0x7fadd0111f40 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadcffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadcffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59286/0 (socket says 172.21.15.29:59286) 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadcffff640 1 -- 172.21.15.29:0/3732121816 learned_addr learned my addr 172.21.15.29:0/3732121816 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 0x7fadd01a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadd4862640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd011cc30 0x7fadd01aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 -- 172.21.15.29:0/3732121816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd011cc30 msgr2=0x7fadd01aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd011cc30 0x7fadd01aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 -- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 msgr2=0x7fadd01a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadc77fe640 1 -- 172.21.15.29:0/3732121816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01af190 con 0x7fadd0111f40 2024-09-21T01:56:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.242+0000 7fadcffff640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadc77fe640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 0x7fadd01a97c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadd006c6e0 tx=0x7fadb800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadcdffb640 1 -- 172.21.15.29:0/3732121816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800cc40 con 0x7fadd0111f40 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadcdffb640 1 -- 172.21.15.29:0/3732121816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb800cde0 con 0x7fadd0111f40 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01af480 con 0x7fadd0111f40 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadcdffb640 1 -- 172.21.15.29:0/3732121816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8010430 con 0x7fadd0111f40 2024-09-21T01:56:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.243+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01b5be0 con 0x7fadd0111f40 2024-09-21T01:56:13.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.247+0000 7faddc839740 1 Processor -- start 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 -- start start 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 0x55f6915382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691538d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6914aeea0 con 0x55f691170390 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f69149e2d0 con 0x55f6911681b0 2024-09-21T01:56:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.248+0000 7faddc839740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f6914a20e0 con 0x55f691191070 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadd4862640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 0x55f6915382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41050/0 (socket says 172.21.15.29:41050) 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadd4862640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 0x55f6915382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45400/0 (socket says 172.21.15.29:45400) 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 -- 172.21.15.29:0/3093062834 learned_addr learned my addr 172.21.15.29:0/3093062834 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 -- 172.21.15.29:0/3093062834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 msgr2=0x55f6915382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 0x55f6915382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcdffb640 1 -- 172.21.15.29:0/3732121816 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fadb8003ed0 con 0x7fadd0111f40 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 -- 172.21.15.29:0/3093062834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 msgr2=0x55f691538d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691538d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.249+0000 7fadcffff640 1 -- 172.21.15.29:0/3093062834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6911fc650 con 0x55f6911681b0 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.250+0000 7fadcffff640 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009c30 tx=0x7fadc000ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b6c76426df2952 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.250+0000 7fadc67fc640 1 -- 172.21.15.29:0/3093062834 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000cdc0 con 0x55f6911681b0 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.250+0000 7fadc67fc640 1 -- 172.21.15.29:0/3093062834 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc001d070 con 0x55f6911681b0 2024-09-21T01:56:13.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.250+0000 7fadc67fc640 1 -- 172.21.15.29:0/3093062834 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000b5c0 con 0x55f6911681b0 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7fadcdffb640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 0x7fadbc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 -- 172.21.15.29:0/3093062834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 msgr2=0x55f691538800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009c30 tx=0x7fadc000ede0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 -- 172.21.15.29:0/3093062834 shutdown_connections 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691538d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6911681b0 0x55f691538800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 --2- 172.21.15.29:0/3093062834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f691191070 0x55f6915382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 -- 172.21.15.29:0/3093062834 >> 172.21.15.29:0/3093062834 conn(0x55f691185460 msgr2=0x55f6915361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7fadd4862640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 0x7fadbc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.251+0000 7faddc839740 1 -- 172.21.15.29:0/3093062834 shutdown_connections 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.252+0000 7faddc839740 1 -- 172.21.15.29:0/3093062834 wait complete. 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.252+0000 7fadcdffb640 1 -- 172.21.15.29:0/3732121816 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fadb80e7760 con 0x7fadd0111f40 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.252+0000 7faddc839740 1 -- start start 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.252+0000 7faddc839740 1 Processor -- start 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadd4862640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 0x7fadbc0b8aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fadc80303f0 tx=0x7fadc802c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691543970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 0x55f691543eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 0x55f69154fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6912ff700 con 0x55f691191070 2024-09-21T01:56:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f691439160 con 0x55f691170390 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f691546aa0 con 0x55f6911681b0 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7faddc839740 10 client.0 _ll_register_callbacks cb 0x55f6914529e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadc77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 0x55f691543eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadcffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691543970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadc77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 0x55f691543eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41066/0 (socket says 172.21.15.29:41066) 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadc77fe640 1 -- 172.21.15.29:0/1975626425 learned_addr learned my addr 172.21.15.29:0/1975626425 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.253+0000 7fadd4862640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 0x55f69154fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 -- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 msgr2=0x55f691543eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 0x55f691543eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 -- 172.21.15.29:0/1975626425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 msgr2=0x55f69154fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 0x55f69154fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 -- 172.21.15.29:0/1975626425 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f691466c90 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadd4862640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 0x55f69154fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadcffff640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691543970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000f80 tx=0x7fadc0004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e0ff87b7cde156 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00185f0 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fadc0018790 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc001d070 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fadc0009df0 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7faddc839740 10 client.26836 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.254+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f6915527b0 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc001d070 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fadc000b040 con 0x55f6911681b0 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 10 client.26836 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 10 client.26836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:56:13.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 20 client.26836 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f69114f9f0 0x55f691555c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 4 client.26836 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6914b0df0 con 0x55f69114f9f0 2024-09-21T01:56:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.255+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f69114f9f0 0x55f691555c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.256+0000 7fadaf7fe640 10 client.26836 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.256+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f69114f9f0 0x55f691555c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2127b71ae862b47 server_cookie=2babc2758429068f in_seq=0 out_seq=0 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.256+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1975626425 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6914b0df0 con 0x55f69114f9f0 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.256+0000 7fadaf7fe640 10 client.26836 handle_command_reply: tid=0 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.256+0000 7faddc839740 1 client.26836 shutdown 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 msgr2=0x55f691543970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691543970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000f80 tx=0x7fadc0004c70 comp rx=0 tx=0).stop 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 shutdown_connections 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f69114f9f0 0x55f691555c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691191070 0x55f69154fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691170390 0x55f691543eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 --2- 172.21.15.29:0/1975626425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691543970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.257+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 >> 172.21.15.29:0/1975626425 conn(0x55f691185460 msgr2=0x55f69153b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.259+0000 7fadaf7fe640 0 client.26836 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:56:13.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.259+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 shutdown_connections 2024-09-21T01:56:13.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.259+0000 7faddc839740 1 -- 172.21.15.29:0/1975626425 wait complete. 2024-09-21T01:56:13.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.260+0000 7faddc839740 20 client.26836 trim_cache size 0 max 16384 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 Processor -- start 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 -- start start 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691545850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f69154fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01be1a0 con 0x55f691170390 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01bfda0 con 0x55f691191070 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7faddc839740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01be870 con 0x55f6911681b0 2024-09-21T01:56:13.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.279+0000 7fadd4862640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.280+0000 7fadd4862640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41070/0 (socket says 172.21.15.29:41070) 2024-09-21T01:56:13.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.280+0000 7fadd4862640 1 -- 172.21.15.29:0/2383512276 learned_addr learned my addr 172.21.15.29:0/2383512276 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:13.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.280+0000 7fadc77fe640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f69154fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.281+0000 7fadc77fe640 1 -- 172.21.15.29:0/2383512276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 msgr2=0x55f691545850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.281+0000 7fadcffff640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691545850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadc77fe640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691545850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadc77fe640 1 -- 172.21.15.29:0/2383512276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 msgr2=0x55f691539a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadc77fe640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadc77fe640 1 -- 172.21.15.29:0/2383512276 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f69114fc20 con 0x55f691170390 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadd4862640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.282+0000 7fadc77fe640 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f69154fd00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fadb80aab10 tx=0x7fadb80045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e459a87ce32625da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.283+0000 7fadc5ffb640 1 -- 172.21.15.29:0/2383512276 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80ad3b0 con 0x55f691170390 2024-09-21T01:56:13.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.283+0000 7fadc5ffb640 1 -- 172.21.15.29:0/2383512276 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb80adfa0 con 0x55f691170390 2024-09-21T01:56:13.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.284+0000 7faddc839740 1 -- 172.21.15.29:0/2383512276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 msgr2=0x55f69154fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.284+0000 7faddc839740 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f69154fd00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fadb80aab10 tx=0x7fadb80045d0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.290+0000 7faddc839740 1 -- 172.21.15.29:0/2383512276 shutdown_connections 2024-09-21T01:56:13.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.290+0000 7faddc839740 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f691539a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.290+0000 7faddc839740 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f69154fd00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.290+0000 7faddc839740 1 --2- 172.21.15.29:0/2383512276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691545850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.290+0000 7faddc839740 1 -- 172.21.15.29:0/2383512276 >> 172.21.15.29:0/2383512276 conn(0x55f691185460 msgr2=0x55f6915392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.295+0000 7faddc839740 1 -- 172.21.15.29:0/2383512276 shutdown_connections 2024-09-21T01:56:13.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.299+0000 7faddc839740 1 -- 172.21.15.29:0/2383512276 wait complete. 2024-09-21T01:56:13.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.299+0000 7faddc839740 1 -- start start 2024-09-21T01:56:13.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.299+0000 7faddc839740 1 Processor -- start 2024-09-21T01:56:13.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691558ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691559280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f6915b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01bf6d0 con 0x55f691170390 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01bf000 con 0x55f691191070 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01bc180 con 0x55f6911681b0 2024-09-21T01:56:13.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.300+0000 7faddc839740 10 client.0 _ll_register_callbacks cb 0x7fadc0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:13.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.302+0000 7fadd4862640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f6915b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.302+0000 7fadd4862640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f6915b8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41080/0 (socket says 172.21.15.29:41080) 2024-09-21T01:56:13.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.302+0000 7fadd4862640 1 -- 172.21.15.29:0/1918006023 learned_addr learned my addr 172.21.15.29:0/1918006023 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:13.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.302+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691559280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.303+0000 7fadc77fe640 1 -- 172.21.15.29:0/1918006023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 msgr2=0x55f691558ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.303+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691558ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.303+0000 7fadc77fe640 1 -- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 msgr2=0x55f6915b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:13.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.303+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f6915b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.303+0000 7fadc77fe640 1 -- 172.21.15.29:0/1918006023 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6915b90a0 con 0x55f691170390 2024-09-21T01:56:13.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.304+0000 7fadc77fe640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691559280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ad180 tx=0x7fadb800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9630eb1e740c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.304+0000 7fadaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:13.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.307+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80b2380 con 0x55f691170390 2024-09-21T01:56:13.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.308+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fadb80acb00 con 0x55f691170390 2024-09-21T01:56:13.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.310+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb80adfa0 con 0x55f691170390 2024-09-21T01:56:13.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.314+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fadb80b72c0 con 0x55f691170390 2024-09-21T01:56:13.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.316+0000 7faddc839740 10 client.17061 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:13.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.316+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f6915c0470 con 0x55f691170390 2024-09-21T01:56:13.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.322+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb80adfa0 con 0x55f691170390 2024-09-21T01:56:13.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.322+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fadb80b3510 con 0x55f691170390 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.324+0000 7faddc839740 10 client.17061 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.324+0000 7faddc839740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.324+0000 7faddc839740 20 client.17061 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.324+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f691124da0 0x55f6915c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.325+0000 7faddc839740 4 client.17061 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:13.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.325+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6914ac8a0 con 0x55f691124da0 2024-09-21T01:56:13.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.326+0000 7fadcffff640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f691124da0 0x55f6915c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.334+0000 7fadcffff640 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f691124da0 0x55f6915c2c10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2c5f6cffa888cb1 server_cookie=3ef470d2c86559de in_seq=0 out_seq=0 2024-09-21T01:56:13.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.335+0000 7fadaf7fe640 10 client.17061 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:56:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.335+0000 7fadaf7fe640 1 -- 172.21.15.29:0/1918006023 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6914ac8a0 con 0x55f691124da0 2024-09-21T01:56:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.335+0000 7fadaf7fe640 10 client.17061 handle_command_reply: tid=0 2024-09-21T01:56:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.335+0000 7faddc839740 1 client.17061 shutdown 2024-09-21T01:56:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 msgr2=0x55f691559280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691559280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ad180 tx=0x7fadb800c750 comp rx=0 tx=0).stop 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 shutdown_connections 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55f691124da0 0x55f6915c2c10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f691191070 0x55f6915b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f691170390 0x55f691559280 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 --2- 172.21.15.29:0/1918006023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6911681b0 0x55f691558ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.336+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 >> 172.21.15.29:0/1918006023 conn(0x55f691185460 msgr2=0x55f691544490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.339+0000 7fadaf7fe640 0 client.17061 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.339+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 shutdown_connections 2024-09-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.339+0000 7faddc839740 1 -- 172.21.15.29:0/1918006023 wait complete. 2024-09-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.339+0000 7faddc839740 20 client.17061 trim_cache size 0 max 16384 2024-09-21T01:56:13.342 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 msgr2=0x7fadbc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 0x7fadbc0b8aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fadc80303f0 tx=0x7fadc802c040 comp rx=0 tx=0).stop 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 msgr2=0x7fadd01a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 0x7fadd01a97c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadd006c6e0 tx=0x7fadb800ee50 comp rx=0 tx=0).stop 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 shutdown_connections 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.341+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fadbc0b6590 0x7fadbc0b8aa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadd011cc30 0x7fadd01aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd0111f40 0x7fadd01a97c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 --2- 172.21.15.29:0/3732121816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadd010a9e0 0x7fadd01a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 >> 172.21.15.29:0/3732121816 conn(0x7fadd0070ec0 msgr2=0x7fadd011e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 shutdown_connections 2024-09-21T01:56:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:13.342+0000 7fadd62c5640 1 -- 172.21.15.29:0/3732121816 wait complete. 2024-09-21T01:56:14.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:13 smithi066 ceph-mon[35001]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-21T01:56:14.065 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:13 smithi029 ceph-mon[31228]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-21T01:56:14.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:56:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:13 smithi190 ceph-mon[34368]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-21T01:56:14.504 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:15.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 -- 172.21.15.29:0/2817457230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 msgr2=0x7f897010fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 --2- 172.21.15.29:0/2817457230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f897010fee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8960009900 tx=0x7f896002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 -- 172.21.15.29:0/2817457230 shutdown_connections 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 --2- 172.21.15.29:0/2817457230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 0x7f8970116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 --2- 172.21.15.29:0/2817457230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f897010fee0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 --2- 172.21.15.29:0/2817457230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 0x7f897010a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 -- 172.21.15.29:0/2817457230 >> 172.21.15.29:0/2817457230 conn(0x7f89700780f0 msgr2=0x7f89701081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.081+0000 7f8975f81640 1 -- 172.21.15.29:0/2817457230 shutdown_connections 2024-09-21T01:56:15.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.082+0000 7f8975f81640 1 -- 172.21.15.29:0/2817457230 wait complete. 2024-09-21T01:56:15.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.082+0000 7f8975f81640 1 Processor -- start 2024-09-21T01:56:15.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 -- start start 2024-09-21T01:56:15.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 0x7f89701a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 0x7f89701a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f897011a090 con 0x7f897010fad0 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59338/0 (socket says 172.21.15.29:59338) 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f896effd640 1 -- 172.21.15.29:0/1526422800 learned_addr learned my addr 172.21.15.29:0/1526422800 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:15.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8970119f10 con 0x7f897010a190 2024-09-21T01:56:15.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.083+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f897011a210 con 0x7f8970110420 2024-09-21T01:56:15.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896f7fe640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 0x7f89701a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896ffff640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 0x7f89701a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896effd640 1 -- 172.21.15.29:0/1526422800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 msgr2=0x7f89701a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896effd640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 0x7f89701a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896effd640 1 -- 172.21.15.29:0/1526422800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 msgr2=0x7f89701a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896effd640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 0x7f89701a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.084+0000 7f896effd640 1 -- 172.21.15.29:0/1526422800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701a6a00 con 0x7f897010fad0 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f896effd640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f896002a930 tx=0x7f8960031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f896cff9640 1 -- 172.21.15.29:0/1526422800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896003f030 con 0x7f897010fad0 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89701a6c90 con 0x7f897010fad0 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f896cff9640 1 -- 172.21.15.29:0/1526422800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8960004490 con 0x7f897010fad0 2024-09-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f896cff9640 1 -- 172.21.15.29:0/1526422800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960035070 con 0x7f897010fad0 2024-09-21T01:56:15.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.085+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897007bc10 con 0x7f897010fad0 2024-09-21T01:56:15.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.088+0000 7f897c4f5740 1 Processor -- start 2024-09-21T01:56:15.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.089+0000 7f896cff9640 1 -- 172.21.15.29:0/1526422800 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8960042030 con 0x7f897010fad0 2024-09-21T01:56:15.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.090+0000 7f896cff9640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 0x7f89580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 -- start start 2024-09-21T01:56:15.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f896ffff640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 0x7f89580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f896cff9640 1 -- 172.21.15.29:0/1526422800 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f89601075d0 con 0x7f897010fad0 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c2f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649b1ba4ea0 con 0x5649b185e1b0 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649b1b942d0 con 0x5649b1866390 2024-09-21T01:56:15.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f897c4f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649b1b980e0 con 0x5649b1887070 2024-09-21T01:56:15.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.091+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59348/0 (socket says 172.21.15.29:59348) 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 -- 172.21.15.29:0/2080092074 learned_addr learned my addr 172.21.15.29:0/2080092074 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896ffff640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c2f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 -- 172.21.15.29:0/2080092074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 msgr2=0x5649b1c2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896f7fe640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 -- 172.21.15.29:0/2080092074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 msgr2=0x5649b1c2f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c2f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 -- 172.21.15.29:0/2080092074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b18f2650 con 0x5649b185e1b0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896f7fe640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896ffff640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 0x7f89580b8aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8964009bd0 tx=0x7f8964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.092+0000 7f896effd640 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f89600cd750 tx=0x7f896002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6686773e721aff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.093+0000 7f8955ffb640 1 -- 172.21.15.29:0/2080092074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960049040 con 0x5649b185e1b0 2024-09-21T01:56:15.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.093+0000 7f8955ffb640 1 -- 172.21.15.29:0/2080092074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89600d7130 con 0x5649b185e1b0 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.093+0000 7f8955ffb640 1 -- 172.21.15.29:0/2080092074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89601070e0 con 0x5649b185e1b0 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- 172.21.15.29:0/2080092074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 msgr2=0x5649b1c2ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f89600cd750 tx=0x7f896002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- 172.21.15.29:0/2080092074 shutdown_connections 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c2f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c2ebe0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2080092074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- 172.21.15.29:0/2080092074 >> 172.21.15.29:0/2080092074 conn(0x5649b187b460 msgr2=0x5649b1c2c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- 172.21.15.29:0/2080092074 shutdown_connections 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- 172.21.15.29:0/2080092074 wait complete. 2024-09-21T01:56:15.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.094+0000 7f897c4f5740 1 -- start start 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.095+0000 7f897c4f5740 1 Processor -- start 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c45db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59358/0 (socket says 172.21.15.29:59358) 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f896f7fe640 1 -- 172.21.15.29:0/3180139060 learned_addr learned my addr 172.21.15.29:0/3180139060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f896effd640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f896ffff640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c45db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649b19f5700 con 0x5649b185e1b0 2024-09-21T01:56:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649b1b2f160 con 0x5649b1866390 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.096+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649b1ba4ea0 con 0x5649b1887070 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 -- 172.21.15.29:0/3180139060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 msgr2=0x5649b1c45db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c45db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 -- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 msgr2=0x5649b1c3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c3a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b1b5cc90 con 0x5649b185e1b0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896ffff640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c45db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f896f7fe640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8968012d20 tx=0x7f89680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d0e452073abf71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f894effd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f897c4f5740 10 client.0 _ll_register_callbacks cb 0x5649b1b489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f894effd640 1 -- 172.21.15.29:0/3180139060 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896801c5f0 con 0x5649b185e1b0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.097+0000 7f894effd640 1 -- 172.21.15.29:0/3180139060 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f896801c790 con 0x5649b185e1b0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89600d7130 con 0x5649b185e1b0 2024-09-21T01:56:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f894effd640 1 -- 172.21.15.29:0/3180139060 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f896801c990 con 0x5649b185e1b0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 10 client.17073 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5649b1c48600 con 0x5649b185e1b0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f894effd640 1 -- 172.21.15.29:0/3180139060 <== mon.0 v2:172.21.15.29:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f896801cb30 con 0x5649b185e1b0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 10 client.17073 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 20 client.17073 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b18459f0 0x5649b1c4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 4 client.17073 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649b1ba6df0 con 0x5649b18459f0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.098+0000 7f896effd640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b18459f0 0x5649b1c4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.099+0000 7f896effd640 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b18459f0 0x5649b1c4bff0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c176328268067682 server_cookie=4aed811331b55bf2 in_seq=0 out_seq=0 2024-09-21T01:56:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.099+0000 7f894effd640 10 client.17073 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.100+0000 7f894effd640 1 -- 172.21.15.29:0/3180139060 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649b1ba6df0 con 0x5649b18459f0 2024-09-21T01:56:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.100+0000 7f894effd640 10 client.17073 handle_command_reply: tid=0 2024-09-21T01:56:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.100+0000 7f897c4f5740 1 client.17073 shutdown 2024-09-21T01:56:15.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.100+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 msgr2=0x5649b1c39e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.100+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8968012d20 tx=0x7f89680107a0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 shutdown_connections 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b18459f0 0x5649b1c4bff0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1887070 0x5649b1c45db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1866390 0x5649b1c3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 --2- 172.21.15.29:0/3180139060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b185e1b0 0x5649b1c39e30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f894effd640 0 client.17073 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 >> 172.21.15.29:0/3180139060 conn(0x5649b187b460 msgr2=0x5649b1c31830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 shutdown_connections 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.101+0000 7f897c4f5740 1 -- 172.21.15.29:0/3180139060 wait complete. 2024-09-21T01:56:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.102+0000 7f897c4f5740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T01:56:15.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.123+0000 7f897c4f5740 1 Processor -- start 2024-09-21T01:56:15.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.123+0000 7f897c4f5740 1 -- start start 2024-09-21T01:56:15.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b185e1b0 0x5649b1c3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1866390 0x5649b1c317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8970084060 con 0x5649b1887070 2024-09-21T01:56:15.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701bfce0 con 0x5649b185e1b0 2024-09-21T01:56:15.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.124+0000 7f897c4f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f89701be7b0 con 0x5649b1866390 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59362/0 (socket says 172.21.15.29:59362) 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 -- 172.21.15.29:0/2413753019 learned_addr learned my addr 172.21.15.29:0/2413753019 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 -- 172.21.15.29:0/2413753019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1866390 msgr2=0x5649b1c317a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1866390 0x5649b1c317a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 -- 172.21.15.29:0/2413753019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b185e1b0 msgr2=0x5649b1c3a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b185e1b0 0x5649b1c3a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.125+0000 7f896ffff640 1 -- 172.21.15.29:0/2413753019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b1c3aed0 con 0x5649b1887070 2024-09-21T01:56:15.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.126+0000 7f896ffff640 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8964003640 tx=0x7f8964014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5737f1dc56f11f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.126+0000 7f89557fa640 1 -- 172.21.15.29:0/2413753019 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964003cf0 con 0x5649b1887070 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.127+0000 7f89557fa640 1 -- 172.21.15.29:0/2413753019 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89640054d0 con 0x5649b1887070 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.127+0000 7f89557fa640 1 -- 172.21.15.29:0/2413753019 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896401b430 con 0x5649b1887070 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- 172.21.15.29:0/2413753019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 msgr2=0x5649b1c31ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8964003640 tx=0x7f8964014000 comp rx=0 tx=0).stop 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- 172.21.15.29:0/2413753019 shutdown_connections 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1887070 0x5649b1c31ce0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b1866390 0x5649b1c317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2413753019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b185e1b0 0x5649b1c3a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- 172.21.15.29:0/2413753019 >> 172.21.15.29:0/2413753019 conn(0x5649b187b460 msgr2=0x5649b1c39be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- 172.21.15.29:0/2413753019 shutdown_connections 2024-09-21T01:56:15.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- 172.21.15.29:0/2413753019 wait complete. 2024-09-21T01:56:15.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.128+0000 7f897c4f5740 1 -- start start 2024-09-21T01:56:15.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 Processor -- start 2024-09-21T01:56:15.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b185e1b0 0x5649b1c42180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1887070 0x5649b1c3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f89701bf610 con 0x5649b1866390 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701bef40 con 0x5649b1887070 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8970082040 con 0x5649b185e1b0 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f897c4f5740 10 client.0 _ll_register_callbacks cb 0x7f8968009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59372/0 (socket says 172.21.15.29:59372) 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 -- 172.21.15.29:0/2286374175 learned_addr learned my addr 172.21.15.29:0/2286374175 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896ffff640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1887070 0x5649b1c3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896f7fe640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b185e1b0 0x5649b1c42180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 -- 172.21.15.29:0/2286374175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b185e1b0 msgr2=0x5649b1c42180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b185e1b0 0x5649b1c42180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 -- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1887070 msgr2=0x5649b1c3d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1887070 0x5649b1c3d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.129+0000 7f896effd640 1 -- 172.21.15.29:0/2286374175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b1c3df90 con 0x5649b1866390 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.130+0000 7f896effd640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89600c9d90 tx=0x7f896003b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e4a6b4d36a4eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.130+0000 7f894effd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960049070 con 0x5649b1866390 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f89600c9a30 con 0x5649b1866390 2024-09-21T01:56:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89640054d0 con 0x5649b1866390 2024-09-21T01:56:15.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8960107110 con 0x5649b1866390 2024-09-21T01:56:15.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f897c4f5740 10 client.17079 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:15.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5649b1cb65d0 con 0x5649b1866390 2024-09-21T01:56:15.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.131+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89600f4060 con 0x5649b1866390 2024-09-21T01:56:15.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8960031ac0 con 0x5649b1866390 2024-09-21T01:56:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 10 client.17079 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 20 client.17079 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b181ada0 0x5649b1cb8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 4 client.17079 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f89640054d0 con 0x5649b181ada0 2024-09-21T01:56:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.132+0000 7f896f7fe640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b181ada0 0x5649b1cb8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f896f7fe640 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b181ada0 0x5649b1cb8930 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4dc3f525c30b2a server_cookie=b4ad2c5457c0d472 in_seq=0 out_seq=0 2024-09-21T01:56:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f894effd640 10 client.17079 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f894effd640 1 -- 172.21.15.29:0/2286374175 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f89640054d0 con 0x5649b181ada0 2024-09-21T01:56:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f894effd640 10 client.17079 handle_command_reply: tid=0 2024-09-21T01:56:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 client.17079 shutdown 2024-09-21T01:56:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 msgr2=0x5649b1c42740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89600c9d90 tx=0x7f896003b560 comp rx=0 tx=0).stop 2024-09-21T01:56:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 shutdown_connections 2024-09-21T01:56:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5649b181ada0 0x5649b1cb8930 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649b1887070 0x5649b1c3d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649b1866390 0x5649b1c42740 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f894effd640 0 client.17079 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 --2- 172.21.15.29:0/2286374175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649b185e1b0 0x5649b1c42180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 >> 172.21.15.29:0/2286374175 conn(0x5649b187b460 msgr2=0x5649b1c31f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 shutdown_connections 2024-09-21T01:56:15.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 1 -- 172.21.15.29:0/2286374175 wait complete. 2024-09-21T01:56:15.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.133+0000 7f897c4f5740 20 client.17079 trim_cache size 0 max 16384 2024-09-21T01:56:15.143 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 msgr2=0x7f89580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 0x7f89580b8aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8964009bd0 tx=0x7f8964005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 msgr2=0x7f89701a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f896002a930 tx=0x7f8960031a20 comp rx=0 tx=0).stop 2024-09-21T01:56:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 shutdown_connections 2024-09-21T01:56:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f89580b6590 0x7f89580b8aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970110420 0x7f89701a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f897010fad0 0x7f89701a1060 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 --2- 172.21.15.29:0/1526422800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897010a190 0x7f89701a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 >> 172.21.15.29:0/1526422800 conn(0x7f89700780f0 msgr2=0x7f89701086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 shutdown_connections 2024-09-21T01:56:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:15.134+0000 7f8975f81640 1 -- 172.21.15.29:0/1526422800 wait complete. 2024-09-21T01:56:16.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:15 smithi066 ceph-mon[35001]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 669 op/s 2024-09-21T01:56:16.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:15 smithi029 ceph-mon[31228]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 669 op/s 2024-09-21T01:56:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:15 smithi190 ceph-mon[34368]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 669 op/s 2024-09-21T01:56:16.871 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:56:16.871 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:56:17.265 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 460.0 s, 1736.2 tps, lat 58.534 ms stddev 68.669 2024-09-21T01:56:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:17 smithi066 ceph-mon[35001]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-09-21T01:56:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:17 smithi029 ceph-mon[31228]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-09-21T01:56:18.122 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:56:18.122 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:56:18.123 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "7e20e03c-6c73-4734-b7ca-ec40b4b4216e", 2024-09-21T01:56:18.123 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:56:18.123 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:56:18.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 -- 172.21.15.29:0/1276555161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c102a60 msgr2=0x7fc10c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 --2- 172.21.15.29:0/1276555161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c102a60 0x7fc10c102e70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc100009900 tx=0x7fc10002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 -- 172.21.15.29:0/1276555161 shutdown_connections 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 --2- 172.21.15.29:0/1276555161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c108cf0 0x7fc10c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 --2- 172.21.15.29:0/1276555161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 --2- 172.21.15.29:0/1276555161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c102a60 0x7fc10c102e70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 -- 172.21.15.29:0/1276555161 >> 172.21.15.29:0/1276555161 conn(0x7fc10c079730 msgr2=0x7fc10c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 -- 172.21.15.29:0/1276555161 shutdown_connections 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.898+0000 7fc113026640 1 -- 172.21.15.29:0/1276555161 wait complete. 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.899+0000 7fc113026640 1 Processor -- start 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.899+0000 7fc113026640 1 -- start start 2024-09-21T01:56:18.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 0x7fc10c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 0x7fc10c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1129b0 con 0x7fc10c102a60 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c112830 con 0x7fc10c1083a0 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc113026640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c112b30 con 0x7fc10c108cf0 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41138/0 (socket says 172.21.15.29:41138) 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 -- 172.21.15.29:0/2038200148 learned_addr learned my addr 172.21.15.29:0/2038200148 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc1115c3640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 0x7fc10c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc110dc2640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 0x7fc10c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 -- 172.21.15.29:0/2038200148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 msgr2=0x7fc10c1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 0x7fc10c1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 -- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 msgr2=0x7fc10c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 0x7fc10c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc10bfff640 1 -- 172.21.15.29:0/2038200148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c1a25e0 con 0x7fc10c1083a0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc110dc2640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 0x7fc10c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.900+0000 7fc1115c3640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 0x7fc10c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc10bfff640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800e9c0 tx=0x7fc0f800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc109ffb640 1 -- 172.21.15.29:0/2038200148 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800ccc0 con 0x7fc10c1083a0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c1a28d0 con 0x7fc10c1083a0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc109ffb640 1 -- 172.21.15.29:0/2038200148 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f80054d0 con 0x7fc10c1083a0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc109ffb640 1 -- 172.21.15.29:0/2038200148 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8010430 con 0x7fc10c1083a0 2024-09-21T01:56:18.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.901+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c1a9030 con 0x7fc10c1083a0 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.905+0000 7fc11959a740 1 Processor -- start 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.905+0000 7fc11959a740 1 -- start start 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.905+0000 7fc11959a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc11959a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc11959a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc11959a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563cc841e60 con 0x5563cc503670 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc11959a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563cc831290 con 0x5563cc524350 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc11959a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563cc8350a0 con 0x5563cc4fb490 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc109ffb640 1 -- 172.21.15.29:0/2038200148 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc0f80105d0 con 0x7fc10c1083a0 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41148/0 (socket says 172.21.15.29:41148) 2024-09-21T01:56:18.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 -- 172.21.15.29:0/2443848807 learned_addr learned my addr 172.21.15.29:0/2443848807 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc10bfff640 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc110dc2640 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 -- 172.21.15.29:0/2443848807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 msgr2=0x5563cc8cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 -- 172.21.15.29:0/2443848807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 msgr2=0x5563cc8cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.906+0000 7fc1115c3640 1 -- 172.21.15.29:0/2443848807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cc58f930 con 0x5563cc524350 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.907+0000 7fc1115c3640 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc104011810 tx=0x7fc104011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f31cbfb7fc91809 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.907+0000 7fc0f6ffd640 1 -- 172.21.15.29:0/2443848807 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104011310 con 0x5563cc524350 2024-09-21T01:56:18.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.908+0000 7fc0f6ffd640 1 -- 172.21.15.29:0/2443848807 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10400f9a0 con 0x5563cc524350 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.908+0000 7fc0f6ffd640 1 -- 172.21.15.29:0/2443848807 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104005630 con 0x5563cc524350 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc109ffb640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 0x7fc0fc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 -- 172.21.15.29:0/2443848807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 msgr2=0x5563cc8cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc104011810 tx=0x7fc104011cd0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc110dc2640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 0x7fc0fc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 -- 172.21.15.29:0/2443848807 shutdown_connections 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 --2- 172.21.15.29:0/2443848807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc524350 0x5563cc8cb430 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 -- 172.21.15.29:0/2443848807 >> 172.21.15.29:0/2443848807 conn(0x5563cc518740 msgr2=0x5563cc8c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc109ffb640 1 -- 172.21.15.29:0/2038200148 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc0f80e79d0 con 0x7fc10c1083a0 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 -- 172.21.15.29:0/2443848807 shutdown_connections 2024-09-21T01:56:18.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.909+0000 7fc11959a740 1 -- 172.21.15.29:0/2443848807 wait complete. 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.910+0000 7fc110dc2640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 0x7fc0fc0b8aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc100000c00 tx=0x7fc100009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.910+0000 7fc11959a740 1 -- start start 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.910+0000 7fc11959a740 1 Processor -- start 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 0x5563cc8e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563cc692890 con 0x5563cc503670 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563cc7cbd40 con 0x5563cc4fb490 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563cc8d9c10 con 0x5563cc524350 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41158/0 (socket says 172.21.15.29:41158) 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 -- 172.21.15.29:0/1498883311 learned_addr learned my addr 172.21.15.29:0/1498883311 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc1115c3640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 0x5563cc8e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 10 client.0 _ll_register_callbacks cb 0x5563cc82fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 -- 172.21.15.29:0/1498883311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 msgr2=0x5563cc8e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 0x5563cc8e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 -- 172.21.15.29:0/1498883311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 msgr2=0x5563cc8d7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 -- 172.21.15.29:0/1498883311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cc8e34f0 con 0x5563cc4fb490 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc1115c3640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 0x5563cc8e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc110dc2640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc100033000 tx=0x7fc100009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5fc3ea0eef9e986 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc0ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10004d070 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc100004c50 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10400f9a0 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc100059050 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 10 client.26870 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.912+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563cc8e5de0 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10400f9a0 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc1000324f0 con 0x5563cc4fb490 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 10 client.26870 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 20 client.26870 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc4e2cd0 0x5563cc8e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 4 client.26870 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.914+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563cc843db0 con 0x5563cc4e2cd0 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.915+0000 7fc10bfff640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc4e2cd0 0x5563cc8e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.915+0000 7fc0ebfff640 10 client.26870 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.916+0000 7fc10bfff640 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc4e2cd0 0x5563cc8e9370 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae1167e9395f889 server_cookie=2c5eb1d748611df5 in_seq=0 out_seq=0 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.916+0000 7fc0ebfff640 1 -- 172.21.15.29:0/1498883311 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563cc843db0 con 0x5563cc4e2cd0 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.916+0000 7fc0ebfff640 10 client.26870 handle_command_reply: tid=0 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 client.26870 shutdown 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 msgr2=0x5563cc8d7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc100033000 tx=0x7fc100009260 comp rx=0 tx=0).stop 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 shutdown_connections 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc4e2cd0 0x5563cc8e9370 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc0ebfff640 0 client.26870 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc524350 0x5563cc8e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 --2- 172.21.15.29:0/1498883311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8d7000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 >> 172.21.15.29:0/1498883311 conn(0x5563cc518740 msgr2=0x5563cc8cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 shutdown_connections 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.917+0000 7fc11959a740 1 -- 172.21.15.29:0/1498883311 wait complete. 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.918+0000 7fc11959a740 20 client.26870 trim_cache size 0 max 16384 2024-09-21T01:56:18.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.951+0000 7fc11959a740 1 Processor -- start 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.954+0000 7fc11959a740 1 -- start start 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.954+0000 7fc11959a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 0x5563cc8d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc11959a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc11959a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc11959a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1b1630 con 0x5563cc503670 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc11959a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1b3230 con 0x5563cc4e2cd0 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 0x5563cc8d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 0x5563cc8d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41174/0 (socket says 172.21.15.29:41174) 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc110dc2640 1 -- 172.21.15.29:0/3976443796 learned_addr learned my addr 172.21.15.29:0/3976443796 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.955+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1b1d00 con 0x5563cc4fb490 2024-09-21T01:56:18.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 -- 172.21.15.29:0/3976443796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 msgr2=0x5563cc8d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 0x5563cc8d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 -- 172.21.15.29:0/3976443796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 msgr2=0x5563cc8d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 -- 172.21.15.29:0/3976443796 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cc8ce290 con 0x5563cc4fb490 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.956+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8e2e00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80aad80 tx=0x7fc0f80e5350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7e1a7f22034950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.958+0000 7fc0f67fc640 1 -- 172.21.15.29:0/3976443796 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80a99b0 con 0x5563cc4fb490 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.958+0000 7fc0f67fc640 1 -- 172.21.15.29:0/3976443796 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f80950c0 con 0x5563cc4fb490 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.958+0000 7fc0f67fc640 1 -- 172.21.15.29:0/3976443796 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80a9280 con 0x5563cc4fb490 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.959+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 msgr2=0x5563cc8e2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.959+0000 7fc11959a740 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8e2e00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80aad80 tx=0x7fc0f80e5350 comp rx=0 tx=0).stop 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.960+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 shutdown_connections 2024-09-21T01:56:18.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.960+0000 7fc11959a740 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc503670 0x5563cc8d3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.960+0000 7fc11959a740 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc4fb490 0x5563cc8e2e00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.960+0000 7fc11959a740 1 --2- 172.21.15.29:0/3976443796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4e2cd0 0x5563cc8d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.960+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 >> 172.21.15.29:0/3976443796 conn(0x5563cc518740 msgr2=0x5563cc8d5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.961+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 shutdown_connections 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.962+0000 7fc11959a740 1 -- 172.21.15.29:0/3976443796 wait complete. 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.962+0000 7fc11959a740 1 -- start start 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.962+0000 7fc11959a740 1 Processor -- start 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.963+0000 7fc11959a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.963+0000 7fc11959a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.963+0000 7fc11959a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc503670 0x5563cc95b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc11959a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1b2b60 con 0x5563cc4e2cd0 2024-09-21T01:56:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc11959a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1b2490 con 0x5563cc4fb490 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc11959a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1af610 con 0x5563cc503670 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc11959a740 10 client.0 _ll_register_callbacks cb 0x7fc0f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59434/0 (socket says 172.21.15.29:59434) 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 -- 172.21.15.29:0/3531750946 learned_addr learned my addr 172.21.15.29:0/3531750946 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc1115c3640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc503670 0x5563cc95b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 -- 172.21.15.29:0/3531750946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc503670 msgr2=0x5563cc95b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc503670 0x5563cc95b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 -- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 msgr2=0x5563cc8dbbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8dbbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.964+0000 7fc110dc2640 1 -- 172.21.15.29:0/3531750946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cc95bb50 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.965+0000 7fc110dc2640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc100009230 tx=0x7fc100032760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234248991c79fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.965+0000 7fc0ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.965+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100061020 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.965+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1000035f0 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.966+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f80950c0 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.966+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc10004d050 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.966+0000 7fc11959a740 10 client.17094 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.966+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563cc95ce10 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1000044d0 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc100049450 con 0x5563cc4e2cd0 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 10 client.17094 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:18.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 20 client.17094 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc524350 0x5563cc95f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 4 client.17094 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc0f80950c0 con 0x5563cc524350 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.967+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc524350 0x5563cc95f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.968+0000 7fc0ebfff640 10 client.17094 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.968+0000 7fc10bfff640 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc524350 0x5563cc95f570 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fe3b2b69c723eb server_cookie=feb090bf791fad3d in_seq=0 out_seq=0 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.971+0000 7fc0ebfff640 1 -- 172.21.15.29:0/3531750946 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc0f80950c0 con 0x5563cc524350 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.971+0000 7fc0ebfff640 10 client.17094 handle_command_reply: tid=0 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.971+0000 7fc11959a740 1 client.17094 shutdown 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 msgr2=0x5563cc8db630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc100009230 tx=0x7fc100032760 comp rx=0 tx=0).stop 2024-09-21T01:56:18.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 shutdown_connections 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cc524350 0x5563cc95f570 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cc503670 0x5563cc95b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cc4fb490 0x5563cc8dbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 --2- 172.21.15.29:0/3531750946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cc4e2cd0 0x5563cc8db630 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 >> 172.21.15.29:0/3531750946 conn(0x5563cc518740 msgr2=0x5563cc8cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc0ebfff640 0 client.17094 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 shutdown_connections 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 1 -- 172.21.15.29:0/3531750946 wait complete. 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.972+0000 7fc11959a740 20 client.17094 trim_cache size 0 max 16384 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 msgr2=0x7fc0fc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 0x7fc0fc0b8aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc100000c00 tx=0x7fc100009560 comp rx=0 tx=0).stop 2024-09-21T01:56:18.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 msgr2=0x7fc10c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800e9c0 tx=0x7fc0f800edc0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 shutdown_connections 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc0fc0b6590 0x7fc0fc0b8aa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10c108cf0 0x7fc10c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc10c1083a0 0x7fc10c19cc10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 --2- 172.21.15.29:0/2038200148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c102a60 0x7fc10c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 >> 172.21.15.29:0/2038200148 conn(0x7fc10c079730 msgr2=0x7fc10c06aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 shutdown_connections 2024-09-21T01:56:18.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:17.973+0000 7fc113026640 1 -- 172.21.15.29:0/2038200148 wait complete. 2024-09-21T01:56:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:17 smithi190 ceph-mon[34368]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-09-21T01:56:19.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:56:19.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:18 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:18 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:18 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:19.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:56:19.620 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:19 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:56:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:19 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:19 smithi066 ceph-mon[35001]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-21T01:56:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:19 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.070 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:19 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:56:20.070 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:19 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.070 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:19 smithi029 ceph-mon[31228]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-21T01:56:20.070 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:19 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 -- 172.21.15.29:0/1133109075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 msgr2=0x7f848c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 --2- 172.21.15.29:0/1133109075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c10d850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f847c009860 tx=0x7f847c02a340 comp rx=0 tx=0).stop 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 -- 172.21.15.29:0/1133109075 shutdown_connections 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 --2- 172.21.15.29:0/1133109075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c10de10 0x7f848c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 --2- 172.21.15.29:0/1133109075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c10d850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 --2- 172.21.15.29:0/1133109075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c1049b0 0x7f848c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 -- 172.21.15.29:0/1133109075 >> 172.21.15.29:0/1133109075 conn(0x7f848c079730 msgr2=0x7f848c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 -- 172.21.15.29:0/1133109075 shutdown_connections 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 -- 172.21.15.29:0/1133109075 wait complete. 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.215+0000 7f8494678640 1 Processor -- start 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 -- start start 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 0x7f848c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 0x7f848c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f848c113a70 con 0x7f848c1049b0 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 0x7f848c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8491c13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 0x7f848c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59440/0 (socket says 172.21.15.29:59440) 2024-09-21T01:56:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8492414640 1 -- 172.21.15.29:0/3240282209 learned_addr learned my addr 172.21.15.29:0/3240282209 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1138f0 con 0x7f848c10de10 2024-09-21T01:56:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.216+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f848c113bf0 con 0x7f848c105300 2024-09-21T01:56:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 0x7f848c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 -- 172.21.15.29:0/3240282209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 msgr2=0x7f848c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 -- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 msgr2=0x7f848c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 0x7f848c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.217+0000 7f8492c15640 1 -- 172.21.15.29:0/3240282209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c1a0490 con 0x7f848c10de10 2024-09-21T01:56:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.218+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 0x7f848c19fd60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f848400ef00 tx=0x7f848400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.219+0000 7f84837fe640 1 -- 172.21.15.29:0/3240282209 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484004b20 con 0x7f848c10de10 2024-09-21T01:56:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.219+0000 7f84837fe640 1 -- 172.21.15.29:0/3240282209 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8484003e20 con 0x7f848c10de10 2024-09-21T01:56:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.219+0000 7f84837fe640 1 -- 172.21.15.29:0/3240282209 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848400b7d0 con 0x7f848c10de10 2024-09-21T01:56:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.220+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c1a0780 con 0x7f848c10de10 2024-09-21T01:56:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.221+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c1a6ee0 con 0x7f848c10de10 2024-09-21T01:56:20.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.224+0000 7f849abec740 1 Processor -- start 2024-09-21T01:56:20.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.224+0000 7f84837fe640 1 -- 172.21.15.29:0/3240282209 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f848401d020 con 0x7f848c10de10 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 -- start start 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56267184be50 con 0x56267150d390 2024-09-21T01:56:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56267183b280 con 0x5626715051b0 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f849abec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56267183f090 con 0x56267152e070 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f8491c13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f8492414640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f8492414640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41192/0 (socket says 172.21.15.29:41192) 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f8492414640 1 -- 172.21.15.29:0/1245585495 learned_addr learned my addr 172.21.15.29:0/1245585495 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.225+0000 7f84837fe640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 0x7f84780b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 -- 172.21.15.29:0/1245585495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 msgr2=0x5626718d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718d30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 -- 172.21.15.29:0/1245585495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 msgr2=0x5626718e5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 -- 172.21.15.29:0/1245585495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562671599650 con 0x5626715051b0 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f84837fe640 1 -- 172.21.15.29:0/3240282209 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f84840e71b0 con 0x7f848c10de10 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 0x7f84780b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8492414640 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8488009cd0 tx=0x7f84880117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dea7a3ce26ce72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8480ff9640 1 -- 172.21.15.29:0/1245585495 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488012450 con 0x5626715051b0 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8480ff9640 1 -- 172.21.15.29:0/1245585495 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488012aa0 con 0x5626715051b0 2024-09-21T01:56:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.226+0000 7f8480ff9640 1 -- 172.21.15.29:0/1245585495 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848800fab0 con 0x5626715051b0 2024-09-21T01:56:20.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:19 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:56:20.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:19 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:19 smithi190 ceph-mon[34368]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-21T01:56:20.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:19 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.229+0000 7f8492c15640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 0x7f84780b8bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f848400cc40 tx=0x7f848400cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.229+0000 7f849abec740 1 -- 172.21.15.29:0/1245585495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 msgr2=0x5626718d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.229+0000 7f849abec740 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8488009cd0 tx=0x7f84880117b0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.229+0000 7f849abec740 1 -- 172.21.15.29:0/1245585495 shutdown_connections 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718e5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718d3630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 --2- 172.21.15.29:0/1245585495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 -- 172.21.15.29:0/1245585495 >> 172.21.15.29:0/1245585495 conn(0x562671522460 msgr2=0x5626718d55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 -- 172.21.15.29:0/1245585495 shutdown_connections 2024-09-21T01:56:20.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.230+0000 7f849abec740 1 -- 172.21.15.29:0/1245585495 wait complete. 2024-09-21T01:56:20.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.231+0000 7f849abec740 1 -- start start 2024-09-21T01:56:20.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.231+0000 7f849abec740 1 Processor -- start 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56267169c700 con 0x56267150d390 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5626717d6110 con 0x5626715051b0 2024-09-21T01:56:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5626718e6500 con 0x56267152e070 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f849abec740 10 client.0 _ll_register_callbacks cb 0x5626717ef990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59456/0 (socket says 172.21.15.29:59456) 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f8492414640 1 -- 172.21.15.29:0/3634531129 learned_addr learned my addr 172.21.15.29:0/3634531129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.232+0000 7f8491c13640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 -- 172.21.15.29:0/3634531129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 msgr2=0x5626718ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 -- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 msgr2=0x5626718e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 -- 172.21.15.29:0/3634531129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562671803c40 con 0x56267150d390 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8491c13640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.233+0000 7f8492414640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8488012fd0 tx=0x7f84880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78637f35b70e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f8475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488028020 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8488012aa0 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8488012860 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8488005050 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f849abec740 10 client.17103 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.234+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5626718f8220 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488012aa0 con 0x56267150d390 2024-09-21T01:56:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f848800f6f0 con 0x56267150d390 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 10 client.17103 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 20 client.17103 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714ec9f0 0x5626718fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 4 client.17103 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56267184dda0 con 0x5626714ec9f0 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.235+0000 7f8491c13640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714ec9f0 0x5626718fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.236+0000 7f8475ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.236+0000 7f8491c13640 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714ec9f0 0x5626718fb640 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e3d79fb934aea5 server_cookie=1e8c10ad8105fba3 in_seq=0 out_seq=0 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f8475ffb640 1 -- 172.21.15.29:0/3634531129 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56267184dda0 con 0x5626714ec9f0 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f8475ffb640 10 client.17103 handle_command_reply: tid=0 2024-09-21T01:56:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 client.17103 shutdown 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 msgr2=0x5626718ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8488012fd0 tx=0x7f84880042f0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 shutdown_connections 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714ec9f0 0x5626718fb640 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718ea2d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 --2- 172.21.15.29:0/3634531129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626715051b0 0x5626718e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 >> 172.21.15.29:0/3634531129 conn(0x562671522460 msgr2=0x5626718d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.237+0000 7f8475ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.238+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 shutdown_connections 2024-09-21T01:56:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.238+0000 7f849abec740 1 -- 172.21.15.29:0/3634531129 wait complete. 2024-09-21T01:56:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.238+0000 7f849abec740 20 client.17103 trim_cache size 0 max 16384 2024-09-21T01:56:20.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 Processor -- start 2024-09-21T01:56:20.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 -- start start 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267150d390 0x5626718d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f8491c13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1ad200 con 0x5626715051b0 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1aee00 con 0x56267150d390 2024-09-21T01:56:20.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f849abec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1ad8d0 con 0x56267152e070 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f8491c13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59468/0 (socket says 172.21.15.29:59468) 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.264+0000 7f8491c13640 1 -- 172.21.15.29:0/3053384694 learned_addr learned my addr 172.21.15.29:0/3053384694 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8492414640 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267150d390 0x5626718d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 -- 172.21.15.29:0/3053384694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 msgr2=0x5626718e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 -- 172.21.15.29:0/3053384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267150d390 msgr2=0x5626718d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267150d390 0x5626718d3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 -- 172.21.15.29:0/3053384694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626718e7ff0 con 0x5626715051b0 2024-09-21T01:56:20.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8491c13640 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f847c005eb0 tx=0x7f847c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e131ba060a9490dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.265+0000 7f8477fff640 1 -- 172.21.15.29:0/3053384694 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c04b070 con 0x5626715051b0 2024-09-21T01:56:20.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.266+0000 7f8477fff640 1 -- 172.21.15.29:0/3053384694 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f847c030900 con 0x5626715051b0 2024-09-21T01:56:20.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.266+0000 7f8477fff640 1 -- 172.21.15.29:0/3053384694 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c037040 con 0x5626715051b0 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 -- 172.21.15.29:0/3053384694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 msgr2=0x5626718d3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f847c005eb0 tx=0x7f847c005e40 comp rx=0 tx=0).stop 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 -- 172.21.15.29:0/3053384694 shutdown_connections 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56267152e070 0x5626718e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267150d390 0x5626718d3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 --2- 172.21.15.29:0/3053384694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626715051b0 0x5626718d3050 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 -- 172.21.15.29:0/3053384694 >> 172.21.15.29:0/3053384694 conn(0x562671522460 msgr2=0x5626718e9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 -- 172.21.15.29:0/3053384694 shutdown_connections 2024-09-21T01:56:20.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.267+0000 7f849abec740 1 -- 172.21.15.29:0/3053384694 wait complete. 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 -- start start 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 Processor -- start 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 0x5626718f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267152e070 0x5626718e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1ae730 con 0x56267150d390 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1ae060 con 0x56267152e070 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1ab1e0 con 0x5626715051b0 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f849abec740 10 client.0 _ll_register_callbacks cb 0x7f8488009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f8491c13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 0x5626718f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f8492414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59470/0 (socket says 172.21.15.29:59470) 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f8492414640 1 -- 172.21.15.29:0/3018433837 learned_addr learned my addr 172.21.15.29:0/3018433837 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.268+0000 7f8491c13640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267152e070 0x5626718e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 -- 172.21.15.29:0/3018433837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 msgr2=0x5626718f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 0x5626718f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 -- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267152e070 msgr2=0x5626718e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267152e070 0x5626718e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 -- 172.21.15.29:0/3018433837 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626718e2bc0 con 0x56267150d390 2024-09-21T01:56:20.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8491c13640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 0x5626718f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.269+0000 7f8492414640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f848800a010 tx=0x7f8488005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcc1152f5ddecf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.272+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848803c020 con 0x56267150d390 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.272+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f848800f640 con 0x56267150d390 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.272+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847c030900 con 0x56267150d390 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.272+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8488028040 con 0x56267150d390 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.272+0000 7f849abec740 10 client.17109 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:20.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.273+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5626718f1fd0 con 0x56267150d390 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.273+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488012a90 con 0x56267150d390 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8488004360 con 0x56267150d390 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f849abec740 10 client.17109 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f849abec740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f849abec740 20 client.17109 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714c1da0 0x562671967fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.274+0000 7f849abec740 4 client.17109 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:56:20.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.275+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f847c030900 con 0x5626714c1da0 2024-09-21T01:56:20.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.278+0000 7f8491c13640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714c1da0 0x562671967fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.292+0000 7f8475ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.292+0000 7f8491c13640 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714c1da0 0x562671967fa0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=600f8e0dba31a32e server_cookie=4365ed7e23ff6c27 in_seq=0 out_seq=0 2024-09-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.293+0000 7f8475ffb640 1 -- 172.21.15.29:0/3018433837 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x7f847c030900 con 0x5626714c1da0 2024-09-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.293+0000 7f8475ffb640 10 client.17109 handle_command_reply: tid=0 2024-09-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.293+0000 7f849abec740 1 client.17109 shutdown 2024-09-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.293+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 msgr2=0x5626718f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.293+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f848800a010 tx=0x7f8488005420 comp rx=0 tx=0).stop 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 shutdown_connections 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5626714c1da0 0x562671967fa0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56267152e070 0x5626718e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56267150d390 0x5626718f60b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 --2- 172.21.15.29:0/3018433837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626715051b0 0x5626718f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.294+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 >> 172.21.15.29:0/3018433837 conn(0x562671522460 msgr2=0x5626718e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.296 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "7e20e03c-6c73-4734-b7ca-ec40b4b4216e", 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:20.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:56:20.300 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.302 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.295+0000 7f8475ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.295+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 shutdown_connections 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.295+0000 7f849abec740 1 -- 172.21.15.29:0/3018433837 wait complete. 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.295+0000 7f849abec740 20 client.17109 trim_cache size 0 max 16384 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 msgr2=0x7f84780b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 0x7f84780b8bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f848400cc40 tx=0x7f848400cf70 comp rx=0 tx=0).stop 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 msgr2=0x7f848c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 0x7f848c19fd60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f848400ef00 tx=0x7f848400c740 comp rx=0 tx=0).stop 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 shutdown_connections 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.298+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f84780b66e0 0x7f84780b8bf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f848c10de10 0x7f848c19fd60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f848c105300 0x7f848c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 --2- 172.21.15.29:0/3240282209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c1049b0 0x7f848c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 >> 172.21.15.29:0/3240282209 conn(0x7f848c079730 msgr2=0x7f848c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 shutdown_connections 2024-09-21T01:56:20.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:20.299+0000 7f8494678640 1 -- 172.21.15.29:0/3240282209 wait complete. 2024-09-21T01:56:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.357+0000 7f1ad6438640 1 -- 172.21.15.29:0/4083964316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 msgr2=0x7f1ad0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.357+0000 7f1ad6438640 1 --2- 172.21.15.29:0/4083964316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad0102ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009900 tx=0x7f1ac002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.357+0000 7f1ad6438640 1 -- 172.21.15.29:0/4083964316 shutdown_connections 2024-09-21T01:56:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 --2- 172.21.15.29:0/4083964316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 0x7f1ad010ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 --2- 172.21.15.29:0/4083964316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 0x7f1ad0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 --2- 172.21.15.29:0/4083964316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad0102ef0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 -- 172.21.15.29:0/4083964316 >> 172.21.15.29:0/4083964316 conn(0x7f1ad00fd690 msgr2=0x7f1ad00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 -- 172.21.15.29:0/4083964316 shutdown_connections 2024-09-21T01:56:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.358+0000 7f1ad6438640 1 -- 172.21.15.29:0/4083964316 wait complete. 2024-09-21T01:56:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.359+0000 7f1ad6438640 1 Processor -- start 2024-09-21T01:56:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.360+0000 7f1ad6438640 1 -- start start 2024-09-21T01:56:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.360+0000 7f1ad6438640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad6438640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 0x7f1ad01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad6438640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 0x7f1ad01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad6438640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010d100 con 0x7f1ad0102ae0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad6438640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010cf80 con 0x7f1ad01040d0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad6438640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010d280 con 0x7f1ad01034c0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59490/0 (socket says 172.21.15.29:59490) 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 -- 172.21.15.29:0/3248154815 learned_addr learned my addr 172.21.15.29:0/3248154815 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 0x7f1ad01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 0x7f1ad01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 -- 172.21.15.29:0/3248154815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 msgr2=0x7f1ad01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 0x7f1ad01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 -- 172.21.15.29:0/3248154815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 msgr2=0x7f1ad01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 0x7f1ad01a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.361+0000 7f1acffff640 1 -- 172.21.15.29:0/3248154815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a1fc0 con 0x7f1ad0102ae0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1acffff640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006010 tx=0x7f1ac00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1acd7fa640 1 -- 172.21.15.29:0/3248154815 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac003f030 con 0x7f1ad0102ae0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a2250 con 0x7f1ad0102ae0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a89a0 con 0x7f1ad0102ae0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1acd7fa640 1 -- 172.21.15.29:0/3248154815 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0031910 con 0x7f1ad0102ae0 2024-09-21T01:56:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.362+0000 7f1acd7fa640 1 -- 172.21.15.29:0/3248154815 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0049060 con 0x7f1ad0102ae0 2024-09-21T01:56:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 Processor -- start 2024-09-21T01:56:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 -- start start 2024-09-21T01:56:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x5582552224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x558255223010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5582551946f0 con 0x558254e512f0 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558255183b20 con 0x558254e594d0 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.365+0000 7f1add485740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558255187930 con 0x558254e7a1b0 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45606/0 (socket says 172.21.15.29:45606) 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 -- 172.21.15.29:0/1591349954 learned_addr learned my addr 172.21.15.29:0/1591349954 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x558255223010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 -- 172.21.15.29:0/1591349954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 msgr2=0x558255223010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x558255223010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 -- 172.21.15.29:0/1591349954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 msgr2=0x5582552224c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x5582552224c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 -- 172.21.15.29:0/1591349954 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558255149530 con 0x558254e7a1b0 2024-09-21T01:56:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1acd7fa640 1 -- 172.21.15.29:0/3248154815 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1ac0042030 con 0x7f1ad0102ae0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x558255223010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.366+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011680 tx=0x7f1ac8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3385807a35568b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1ab67fc640 1 -- 172.21.15.29:0/1591349954 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8019030 con 0x558254e7a1b0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1ab67fc640 1 -- 172.21.15.29:0/1591349954 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac80093a0 con 0x558254e7a1b0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1ab67fc640 1 -- 172.21.15.29:0/1591349954 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004280 con 0x558254e7a1b0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1acd7fa640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 0x7f1ab80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1acd7fa640 1 -- 172.21.15.29:0/3248154815 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac01077d0 con 0x7f1ad0102ae0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.367+0000 7f1acffff640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 0x7f1ab80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- 172.21.15.29:0/1591349954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 msgr2=0x558255221e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011680 tx=0x7f1ac8009920 comp rx=0 tx=0).stop 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- 172.21.15.29:0/1591349954 shutdown_connections 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x558255223010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x5582552224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 --2- 172.21.15.29:0/1591349954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e7a1b0 0x558255221e50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- 172.21.15.29:0/1591349954 >> 172.21.15.29:0/1591349954 conn(0x558254e6e300 msgr2=0x55825521d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- 172.21.15.29:0/1591349954 shutdown_connections 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- 172.21.15.29:0/1591349954 wait complete. 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1acffff640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 0x7f1ab80b8a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01052e0 tx=0x7f1ac0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.368+0000 7f1add485740 1 -- start start 2024-09-21T01:56:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 Processor -- start 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 0x55825522b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x55825522bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x55825522c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5582550bbab0 con 0x558254e7a1b0 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558255129730 con 0x558254e594d0 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558255223b30 con 0x558254e512f0 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1add485740 10 client.0 _ll_register_callbacks cb 0x55825517d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 0x55825522b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 0x55825522b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45618/0 (socket says 172.21.15.29:45618) 2024-09-21T01:56:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1ad49d5640 1 -- 172.21.15.29:0/2541738829 learned_addr learned my addr 172.21.15.29:0/2541738829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x55825522bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x55825522c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 -- 172.21.15.29:0/2541738829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 msgr2=0x55825522b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 0x55825522b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 -- 172.21.15.29:0/2541738829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 msgr2=0x55825522c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x55825522c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.369+0000 7f1acf7fe640 1 -- 172.21.15.29:0/2541738829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558254ee5790 con 0x558254e594d0 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.370+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x55825522bc90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4013560 tx=0x7f1ac4013a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6137d7e6ada10818 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.370+0000 7f1aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.370+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004230 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.370+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac40043d0 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.370+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558255196640 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.372+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ac4005020 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.372+0000 7f1add485740 10 client.26903 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.372+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558254f5e620 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558255196640 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1ac40047a0 con 0x558254e594d0 2024-09-21T01:56:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 10 client.26903 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 10 client.26903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 20 client.26903 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e38d80 0x558255238e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 4 client.26903 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582551920f0 con 0x558254e38d80 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.373+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e38d80 0x558255238e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.374+0000 7f1aaf7fe640 10 client.26903 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.374+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e38d80 0x558255238e10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c161992ad9e454 server_cookie=96657a5866f30ce5 in_seq=0 out_seq=0 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.374+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/2541738829 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582551920f0 con 0x558254e38d80 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.374+0000 7f1aaf7fe640 10 client.26903 handle_command_reply: tid=0 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 client.26903 shutdown 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 msgr2=0x55825522bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x55825522bc90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4013560 tx=0x7f1ac4013a20 comp rx=0 tx=0).stop 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 shutdown_connections 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e38d80 0x558255238e10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x55825522c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e594d0 0x55825522bc90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 --2- 172.21.15.29:0/2541738829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e512f0 0x55825522b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 >> 172.21.15.29:0/2541738829 conn(0x558254e6e300 msgr2=0x55825521bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1aaf7fe640 0 client.26903 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 shutdown_connections 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 1 -- 172.21.15.29:0/2541738829 wait complete. 2024-09-21T01:56:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.375+0000 7f1add485740 20 client.26903 trim_cache size 0 max 16384 2024-09-21T01:56:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 Processor -- start 2024-09-21T01:56:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 -- start start 2024-09-21T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x5582552185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e7a1b0 0x55825522b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00e430 con 0x558254e512f0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc010030 con 0x558254e7a1b0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1add485740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00eb00 con 0x558254e594d0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.395+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59508/0 (socket says 172.21.15.29:59508) 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 -- 172.21.15.29:0/3816784735 learned_addr learned my addr 172.21.15.29:0/3816784735 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x5582552185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e7a1b0 0x55825522b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 -- 172.21.15.29:0/3816784735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 msgr2=0x5582552185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x5582552185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 -- 172.21.15.29:0/3816784735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e7a1b0 msgr2=0x55825522b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e7a1b0 0x55825522b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 -- 172.21.15.29:0/3816784735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55825522a280 con 0x558254e512f0 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x5582552185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8008d70 tx=0x7f1ac800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb746cb2634a2448 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.396+0000 7f1ab5ffb640 1 -- 172.21.15.29:0/3816784735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8019040 con 0x558254e512f0 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1ab5ffb640 1 -- 172.21.15.29:0/3816784735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8007980 con 0x558254e512f0 2024-09-21T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1ab5ffb640 1 -- 172.21.15.29:0/3816784735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac802e3d0 con 0x558254e512f0 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 -- 172.21.15.29:0/3816784735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 msgr2=0x55825521d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8008d70 tx=0x7f1ac800b040 comp rx=0 tx=0).stop 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 -- 172.21.15.29:0/3816784735 shutdown_connections 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e7a1b0 0x55825522b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x5582552185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 --2- 172.21.15.29:0/3816784735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e512f0 0x55825521d220 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.397+0000 7f1add485740 1 -- 172.21.15.29:0/3816784735 >> 172.21.15.29:0/3816784735 conn(0x558254e6e300 msgr2=0x558255222e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.398+0000 7f1add485740 1 -- 172.21.15.29:0/3816784735 shutdown_connections 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.398+0000 7f1add485740 1 -- 172.21.15.29:0/3816784735 wait complete. 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.398+0000 7f1add485740 1 -- start start 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.398+0000 7f1add485740 1 Processor -- start 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x558255230cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x558255231820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00f960 con 0x558254e7a1b0 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00f290 con 0x558254e512f0 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00c410 con 0x558254e594d0 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1add485740 10 client.0 _ll_register_callbacks cb 0x7f1ac4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1ad49d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41296/0 (socket says 172.21.15.29:41296) 2024-09-21T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1ad49d5640 1 -- 172.21.15.29:0/224797917 learned_addr learned my addr 172.21.15.29:0/224797917 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.399+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x558255231820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.400+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x558255230cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 -- 172.21.15.29:0/224797917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 msgr2=0x558255230cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x558255230cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 -- 172.21.15.29:0/224797917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 msgr2=0x558255231820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x558255231820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 -- 172.21.15.29:0/224797917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558255231f80 con 0x558254e512f0 2024-09-21T01:56:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x558255230cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.401+0000 7f1ad49d5640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8008c50 tx=0x7f1ac8009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22628f58dfd85fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8019040 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac8007980 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac8015550 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ac8015750 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1add485740 10 client.26909 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558255233270 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.402+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8007980 con 0x558254e512f0 2024-09-21T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1ac8036c80 con 0x558254e512f0 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 10 client.26909 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 10 client.26909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 20 client.26909 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e0e1c0 0x5582552a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 4 client.26909 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55825518f520 con 0x558254e0e1c0 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.403+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e0e1c0 0x5582552a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.404+0000 7f1aaf7fe640 10 client.26909 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.404+0000 7f1acf7fe640 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e0e1c0 0x5582552a57e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76be4b77b3d7100 server_cookie=d3a273e74e4860fe in_seq=0 out_seq=0 2024-09-21T01:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1aaf7fe640 1 -- 172.21.15.29:0/224797917 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55825518f520 con 0x558254e0e1c0 2024-09-21T01:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1aaf7fe640 10 client.26909 handle_command_reply: tid=0 2024-09-21T01:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 client.26909 shutdown 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 msgr2=0x5582552305f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8008c50 tx=0x7f1ac8009ac0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 shutdown_connections 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x558254e0e1c0 0x5582552a57e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.414+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558254e7a1b0 0x558255231820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558254e594d0 0x558255230cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 1 --2- 172.21.15.29:0/224797917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558254e512f0 0x5582552305f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 >> 172.21.15.29:0/224797917 conn(0x558254e6e300 msgr2=0x5582552185c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1aaf7fe640 0 client.26909 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 shutdown_connections 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 1 -- 172.21.15.29:0/224797917 wait complete. 2024-09-21T01:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.415+0000 7f1add485740 20 client.26909 trim_cache size 0 max 16384 2024-09-21T01:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:56:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 msgr2=0x7f1ab80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 0x7f1ab80b8a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01052e0 tx=0x7f1ac0005ef0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 msgr2=0x7f1ad01a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006010 tx=0x7f1ac00042f0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 shutdown_connections 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ab80b6510 0x7f1ab80b8a20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad01040d0 0x7f1ad01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad01034c0 0x7f1ad01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 --2- 172.21.15.29:0/3248154815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad0102ae0 0x7f1ad01a0580 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.416+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 >> 172.21.15.29:0/3248154815 conn(0x7f1ad00fd690 msgr2=0x7f1ad0104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.417+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 shutdown_connections 2024-09-21T01:56:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:20.417+0000 7f1ad6438640 1 -- 172.21.15.29:0/3248154815 wait complete. 2024-09-21T01:56:21.197 INFO:tasks.cephfs.filesystem:scrub status for tag:7e20e03c-6c73-4734-b7ca-ec40b4b4216e - {'path': '/', 'tag': '7e20e03c-6c73-4734-b7ca-ec40b4b4216e', 'options': 'recursive,force'} 2024-09-21T01:56:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:21 smithi066 ceph-mon[35001]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 777 op/s 2024-09-21T01:56:22.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:21 smithi029 ceph-mon[31228]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 777 op/s 2024-09-21T01:56:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:21 smithi190 ceph-mon[34368]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 777 op/s 2024-09-21T01:56:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 465.0 s, 1801.0 tps, lat 55.599 ms stddev 65.383 2024-09-21T01:56:24.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:23 smithi066 ceph-mon[35001]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 717 op/s 2024-09-21T01:56:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:23 smithi029 ceph-mon[31228]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 717 op/s 2024-09-21T01:56:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:23 smithi190 ceph-mon[34368]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 717 op/s 2024-09-21T01:56:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:25 smithi029 ceph-mon[31228]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.7 MiB/s wr, 763 op/s 2024-09-21T01:56:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:56:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:56:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:56:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:56:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:25 smithi190 ceph-mon[34368]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.7 MiB/s wr, 763 op/s 2024-09-21T01:56:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:56:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:56:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:56:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:56:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:25 smithi066 ceph-mon[35001]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.7 MiB/s wr, 763 op/s 2024-09-21T01:56:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:56:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:56:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:56:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:56:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:26 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:56:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:26 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:56:27.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:26 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:56:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 470.0 s, 1860.4 tps, lat 53.484 ms stddev 60.503 2024-09-21T01:56:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:27 smithi029 ceph-mon[31228]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 829 op/s 2024-09-21T01:56:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:27 smithi190 ceph-mon[34368]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 829 op/s 2024-09-21T01:56:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:27 smithi066 ceph-mon[35001]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 829 op/s 2024-09-21T01:56:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:29 smithi029 ceph-mon[31228]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-21T01:56:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:29 smithi190 ceph-mon[34368]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-21T01:56:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:29 smithi066 ceph-mon[35001]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-21T01:56:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:31 smithi029 ceph-mon[31228]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-09-21T01:56:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:31 smithi190 ceph-mon[34368]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-09-21T01:56:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:31 smithi066 ceph-mon[35001]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-09-21T01:56:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 475.0 s, 2337.8 tps, lat 43.226 ms stddev 47.701 2024-09-21T01:56:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 -- 172.21.15.29:0/4032912009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 msgr2=0x7f1ae0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/4032912009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae0102a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009900 tx=0x7f1ad002a420 comp rx=0 tx=0).stop 2024-09-21T01:56:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 -- 172.21.15.29:0/4032912009 shutdown_connections 2024-09-21T01:56:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/4032912009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103fc0 0x7f1ae010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/4032912009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103620 0x7f1ae0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/4032912009 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae0102a10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 -- 172.21.15.29:0/4032912009 >> 172.21.15.29:0/4032912009 conn(0x7f1ae00fdee0 msgr2=0x7f1ae0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.549+0000 7f1ae51e6640 1 -- 172.21.15.29:0/4032912009 shutdown_connections 2024-09-21T01:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.550+0000 7f1ae51e6640 1 -- 172.21.15.29:0/4032912009 wait complete. 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 Processor -- start 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 -- start start 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103620 0x7f1ae019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 0x7f1ae019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d110 con 0x7f1ae01025a0 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010cf90 con 0x7f1ae0103fc0 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ae51e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d290 con 0x7f1ae0103620 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1aded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1aded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53392/0 (socket says 172.21.15.29:53392) 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1aded9d640 1 -- 172.21.15.29:0/1932351246 learned_addr learned my addr 172.21.15.29:0/1932351246 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 0x7f1ae019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.551+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103620 0x7f1ae019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 -- 172.21.15.29:0/1932351246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103620 msgr2=0x7f1ae019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103620 0x7f1ae019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 -- 172.21.15.29:0/1932351246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 msgr2=0x7f1ae019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 0x7f1ae019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 -- 172.21.15.29:0/1932351246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae019dbe0 con 0x7f1ae01025a0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 0x7f1ae019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009850 tx=0x7f1ad0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1ac7fff640 1 -- 172.21.15.29:0/1932351246 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad003f070 con 0x7f1ae01025a0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1ac7fff640 1 -- 172.21.15.29:0/1932351246 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad0031c50 con 0x7f1ae01025a0 2024-09-21T01:56:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae019de70 con 0x7f1ae01025a0 2024-09-21T01:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1ac7fff640 1 -- 172.21.15.29:0/1932351246 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0035030 con 0x7f1ae01025a0 2024-09-21T01:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.552+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a45c0 con 0x7f1ae01025a0 2024-09-21T01:56:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.556+0000 7f1aec22d740 1 Processor -- start 2024-09-21T01:56:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.556+0000 7f1aec22d740 1 -- start start 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.556+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.556+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.556+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1aec22d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5628322176f0 con 0x562831edc4d0 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1aec22d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562832206b20 con 0x562831efd1b0 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1aec22d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56283220a930 con 0x562831ed42f0 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54546/0 (socket says 172.21.15.29:54546) 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 -- 172.21.15.29:0/2183642965 learned_addr learned my addr 172.21.15.29:0/2183642965 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1ade59c640 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 -- 172.21.15.29:0/2183642965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 msgr2=0x5628322b64d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b64d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 -- 172.21.15.29:0/2183642965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 msgr2=0x5628322b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 -- 172.21.15.29:0/2183642965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628321cc530 con 0x562831efd1b0 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.557+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8011820 tx=0x7f1ad8011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c837cc60edba14e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.558+0000 7f1ac57fa640 1 -- 172.21.15.29:0/2183642965 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011330 con 0x562831efd1b0 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.558+0000 7f1ac57fa640 1 -- 172.21.15.29:0/2183642965 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad800fe50 con 0x562831efd1b0 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.558+0000 7f1ac57fa640 1 -- 172.21.15.29:0/2183642965 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800e3b0 con 0x562831efd1b0 2024-09-21T01:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.558+0000 7f1ac7fff640 1 -- 172.21.15.29:0/1932351246 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1ad0042070 con 0x7f1ae01025a0 2024-09-21T01:56:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- 172.21.15.29:0/2183642965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 msgr2=0x5628322b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8011820 tx=0x7f1ad8011ce0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- 172.21.15.29:0/2183642965 shutdown_connections 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1ac7fff640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 0x7f1ac80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2183642965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322b5e60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- 172.21.15.29:0/2183642965 >> 172.21.15.29:0/2183642965 conn(0x562831ef1300 msgr2=0x5628322a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 0x7f1ac80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- 172.21.15.29:0/2183642965 shutdown_connections 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1ac7fff640 1 -- 172.21.15.29:0/1932351246 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ad0107ae0 con 0x7f1ae01025a0 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- 172.21.15.29:0/2183642965 wait complete. 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.559+0000 7f1aec22d740 1 -- start start 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 Processor -- start 2024-09-21T01:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 0x7f1ac80b8ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80049d0 tx=0x7f1ad8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56283213eab0 con 0x562831edc4d0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5628321ac730 con 0x562831efd1b0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5628322b7b40 con 0x562831ed42f0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.560+0000 7f1aec22d740 10 client.0 _ll_register_callbacks cb 0x5628322006b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:54558/0 (socket says 172.21.15.29:54558) 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 -- 172.21.15.29:0/2637533979 learned_addr learned my addr 172.21.15.29:0/2637533979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1aded9d640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1ade59c640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 -- 172.21.15.29:0/2637533979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 msgr2=0x5628322c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 -- 172.21.15.29:0/2637533979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 msgr2=0x5628322c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 -- 172.21.15.29:0/2637533979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562831f68790 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1adf59e640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800fdb0 tx=0x7f1ad8011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b623fc6911a3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1ac2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8005450 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ad80055f0 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.561+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad800fe50 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.562+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ad80057f0 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.562+0000 7f1aec22d740 10 client.26921 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.562+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562831fe1620 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.562+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad800fe50 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1ad8034420 con 0x562831efd1b0 2024-09-21T01:56:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 10 client.26921 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 20 client.26921 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831ebbd80 0x5628322d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 4 client.26921 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.563+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562832219640 con 0x562831ebbd80 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.565+0000 7f1aded9d640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831ebbd80 0x5628322d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.566+0000 7f1aded9d640 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831ebbd80 0x5628322d53c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4766c46e3a3a61a2 server_cookie=78903072227a5c83 in_seq=0 out_seq=0 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.566+0000 7f1ac2ff5640 10 client.26921 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/2637533979 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562832219640 con 0x562831ebbd80 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1ac2ff5640 10 client.26921 handle_command_reply: tid=0 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 client.26921 shutdown 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 msgr2=0x5628322c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800fdb0 tx=0x7f1ad8011ce0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 shutdown_connections 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831ebbd80 0x5628322d53c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1ac2ff5640 0 client.26921 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831efd1b0 0x5628322c9060 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 --2- 172.21.15.29:0/2637533979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 >> 172.21.15.29:0/2637533979 conn(0x562831ef1300 msgr2=0x56283229eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 shutdown_connections 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.567+0000 7f1aec22d740 1 -- 172.21.15.29:0/2637533979 wait complete. 2024-09-21T01:56:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.568+0000 7f1aec22d740 20 client.26921 trim_cache size 0 max 16384 2024-09-21T01:56:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.590+0000 7f1aec22d740 1 Processor -- start 2024-09-21T01:56:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 -- start start 2024-09-21T01:56:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0071310 con 0x562831edc4d0 2024-09-21T01:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0072f10 con 0x562831ebbd80 2024-09-21T01:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aec22d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae00719e0 con 0x562831ed42f0 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1adf59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53410/0 (socket says 172.21.15.29:53410) 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1adf59e640 1 -- 172.21.15.29:0/910894912 learned_addr learned my addr 172.21.15.29:0/910894912 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1ade59c640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.591+0000 7f1aded9d640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 -- 172.21.15.29:0/910894912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 msgr2=0x5628322b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 -- 172.21.15.29:0/910894912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 msgr2=0x5628322c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 -- 172.21.15.29:0/910894912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628322d0300 con 0x562831edc4d0 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1ade59c640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1adf59e640 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80111d0 tx=0x7f1ad8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5711b971bd0650dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1ac4ff9640 1 -- 172.21.15.29:0/910894912 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8010800 con 0x562831edc4d0 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1ac4ff9640 1 -- 172.21.15.29:0/910894912 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad802f030 con 0x562831edc4d0 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.592+0000 7f1ac4ff9640 1 -- 172.21.15.29:0/910894912 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8034b00 con 0x562831edc4d0 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 -- 172.21.15.29:0/910894912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 msgr2=0x5628322b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80111d0 tx=0x7f1ad8011720 comp rx=0 tx=0).stop 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 -- 172.21.15.29:0/910894912 shutdown_connections 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831edc4d0 0x5628322b6340 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831ed42f0 0x5628322b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 --2- 172.21.15.29:0/910894912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 -- 172.21.15.29:0/910894912 >> 172.21.15.29:0/910894912 conn(0x562831ef1300 msgr2=0x56283229eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 -- 172.21.15.29:0/910894912 shutdown_connections 2024-09-21T01:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.593+0000 7f1aec22d740 1 -- 172.21.15.29:0/910894912 wait complete. 2024-09-21T01:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 -- start start 2024-09-21T01:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 Processor -- start 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 0x5628322cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831edc4d0 0x5628322ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1ade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 0x5628322cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1ade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 0x5628322cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53426/0 (socket says 172.21.15.29:53426) 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1ade59c640 1 -- 172.21.15.29:0/1909331865 learned_addr learned my addr 172.21.15.29:0/1909331865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aded9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0072840 con 0x562831ed42f0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0072170 con 0x562831ebbd80 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae006f2f0 con 0x562831edc4d0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.594+0000 7f1aec22d740 10 client.0 _ll_register_callbacks cb 0x7f1ad400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 -- 172.21.15.29:0/1909331865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831edc4d0 msgr2=0x5628322ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831edc4d0 0x5628322ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 -- 172.21.15.29:0/1909331865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 msgr2=0x5628322cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 0x5628322cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628322ce7b0 con 0x562831ebbd80 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322cce20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005ab0 tx=0x7f1ad003b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656950c634c85b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1ac2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.595+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0049040 con 0x562831ebbd80 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.596+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad802f030 con 0x562831ebbd80 2024-09-21T01:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.596+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ad00b50d0 con 0x562831ebbd80 2024-09-21T01:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.596+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ad007e1a0 con 0x562831ebbd80 2024-09-21T01:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 10 client.26927 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5628322cfb40 con 0x562831ebbd80 2024-09-21T01:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad802f030 con 0x562831ebbd80 2024-09-21T01:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1ad00c9fa0 con 0x562831ebbd80 2024-09-21T01:56:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 10 client.26927 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 20 client.26927 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.597+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831efd1b0 0x562832341ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.598+0000 7f1aec22d740 4 client.26927 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.598+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628322150f0 con 0x562831efd1b0 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.598+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831efd1b0 0x562832341ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.598+0000 7f1ac2ff5640 10 client.26927 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.598+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831efd1b0 0x562832341ec0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8628f16bbb8ebf9 server_cookie=3183d47745d54722 in_seq=0 out_seq=0 2024-09-21T01:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1ac2ff5640 1 -- 172.21.15.29:0/1909331865 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628322150f0 con 0x562831efd1b0 2024-09-21T01:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1ac2ff5640 10 client.26927 handle_command_reply: tid=0 2024-09-21T01:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 client.26927 shutdown 2024-09-21T01:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 msgr2=0x5628322cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322cce20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005ab0 tx=0x7f1ad003b6b0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 shutdown_connections 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562831efd1b0 0x562832341ec0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562831edc4d0 0x5628322ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562831ed42f0 0x5628322cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1ac2ff5640 0 client.26927 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 --2- 172.21.15.29:0/1909331865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562831ebbd80 0x5628322cce20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 >> 172.21.15.29:0/1909331865 conn(0x562831ef1300 msgr2=0x56283229eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 shutdown_connections 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.611+0000 7f1aec22d740 1 -- 172.21.15.29:0/1909331865 wait complete. 2024-09-21T01:56:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1aec22d740 20 client.26927 trim_cache size 0 max 16384 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 msgr2=0x7f1ac80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 0x7f1ac80b8ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80049d0 tx=0x7f1ad8000f40 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 msgr2=0x7f1ae019c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009850 tx=0x7f1ad0003b90 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 shutdown_connections 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac80b65d0 0x7f1ac80b8ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ae0103fc0 0x7f1ae019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae0103620 0x7f1ae019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 --2- 172.21.15.29:0/1932351246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae01025a0 0x7f1ae019c1a0 secure :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009850 tx=0x7f1ad0003b90 comp rx=0 tx=0).stop 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.612+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 >> 172.21.15.29:0/1932351246 conn(0x7f1ae00fdee0 msgr2=0x7f1ae0106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.613+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 shutdown_connections 2024-09-21T01:56:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:33.614+0000 7f1ae51e6640 1 -- 172.21.15.29:0/1932351246 wait complete. 2024-09-21T01:56:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:33 smithi029 ceph-mon[31228]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 763 op/s 2024-09-21T01:56:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:33 smithi190 ceph-mon[34368]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 763 op/s 2024-09-21T01:56:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:33 smithi066 ceph-mon[35001]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 763 op/s 2024-09-21T01:56:35.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:34 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:35.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:34 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:35.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:34 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:35 smithi029 ceph-mon[31228]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 699 op/s 2024-09-21T01:56:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:56:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:35 smithi190 ceph-mon[34368]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 699 op/s 2024-09-21T01:56:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:36.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:56:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:35 smithi066 ceph-mon[35001]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 699 op/s 2024-09-21T01:56:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:56:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 480.0 s, 1920.2 tps, lat 51.685 ms stddev 64.262 2024-09-21T01:56:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:37 smithi029 ceph-mon[31228]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 790 op/s 2024-09-21T01:56:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:37 smithi190 ceph-mon[34368]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 790 op/s 2024-09-21T01:56:38.302 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:37 smithi066 ceph-mon[35001]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 790 op/s 2024-09-21T01:56:40.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:39 smithi029 ceph-mon[31228]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 625 op/s 2024-09-21T01:56:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:39 smithi190 ceph-mon[34368]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 625 op/s 2024-09-21T01:56:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:39 smithi066 ceph-mon[35001]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 625 op/s 2024-09-21T01:56:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:41 smithi190 ceph-mon[34368]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 778 op/s 2024-09-21T01:56:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:41 smithi066 ceph-mon[35001]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 778 op/s 2024-09-21T01:56:42.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:41 smithi029 ceph-mon[31228]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 778 op/s 2024-09-21T01:56:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 485.0 s, 2064.8 tps, lat 48.004 ms stddev 59.920 2024-09-21T01:56:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:43 smithi190 ceph-mon[34368]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 616 op/s 2024-09-21T01:56:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:43 smithi066 ceph-mon[35001]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 616 op/s 2024-09-21T01:56:44.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:43 smithi029 ceph-mon[31228]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 616 op/s 2024-09-21T01:56:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:45 smithi190 ceph-mon[34368]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 651 op/s 2024-09-21T01:56:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:45 smithi066 ceph-mon[35001]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 651 op/s 2024-09-21T01:56:46.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:45 smithi029 ceph-mon[31228]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 651 op/s 2024-09-21T01:56:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.747+0000 7f2c4f912640 1 -- 172.21.15.29:0/1576859220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c48103fd0 msgr2=0x7f2c4810a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.747+0000 7f2c4f912640 1 --2- 172.21.15.29:0/1576859220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c48103fd0 0x7f2c4810a970 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009ca0 tx=0x7f2c4002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 -- 172.21.15.29:0/1576859220 shutdown_connections 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 --2- 172.21.15.29:0/1576859220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c48103fd0 0x7f2c4810a970 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 --2- 172.21.15.29:0/1576859220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103630 0x7f2c48103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 --2- 172.21.15.29:0/1576859220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c481025b0 0x7f2c48102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 -- 172.21.15.29:0/1576859220 >> 172.21.15.29:0/1576859220 conn(0x7f2c480fdf30 msgr2=0x7f2c481003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 -- 172.21.15.29:0/1576859220 shutdown_connections 2024-09-21T01:56:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.748+0000 7f2c4f912640 1 -- 172.21.15.29:0/1576859220 wait complete. 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 Processor -- start 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 -- start start 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c48103630 0x7f2c48198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103fd0 0x7f2c48198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810d170 con 0x7f2c481025b0 2024-09-21T01:56:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810cff0 con 0x7f2c48103fd0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4f912640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810d2f0 con 0x7f2c48103630 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46440/0 (socket says 172.21.15.29:46440) 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3640138393 learned_addr learned my addr 172.21.15.29:0/3640138393 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c48103630 0x7f2c48198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.749+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103fd0 0x7f2c48198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3640138393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c48103630 msgr2=0x7f2c48198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c48103630 0x7f2c48198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3640138393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103fd0 msgr2=0x7f2c48198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103fd0 0x7f2c48198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3640138393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c48199730 con 0x7f2c481025b0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9f0 tx=0x7f2c4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c3a7fc640 1 -- 172.21.15.29:0/3640138393 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400cd80 con 0x7f2c481025b0 2024-09-21T01:56:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c3a7fc640 1 -- 172.21.15.29:0/3640138393 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c440054d0 con 0x7f2c481025b0 2024-09-21T01:56:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c3a7fc640 1 -- 172.21.15.29:0/3640138393 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44010690 con 0x7f2c481025b0 2024-09-21T01:56:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.750+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c48199a20 con 0x7f2c481025b0 2024-09-21T01:56:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.751+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c481a0110 con 0x7f2c481025b0 2024-09-21T01:56:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c3a7fc640 1 -- 172.21.15.29:0/3640138393 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2c44004e90 con 0x7f2c481025b0 2024-09-21T01:56:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 Processor -- start 2024-09-21T01:56:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 -- start start 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb64d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c3a7fc640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 0x7f2c340bad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb64e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560ddb5bf6f0 con 0x560ddb2a51b0 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560ddb5aeb20 con 0x560ddb2844d0 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.754+0000 7f2c56878740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560ddb5b2930 con 0x560ddb27c2f0 2024-09-21T01:56:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4deaf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4deaf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46442/0 (socket says 172.21.15.29:46442) 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4deaf640 1 -- 172.21.15.29:0/2430579122 learned_addr learned my addr 172.21.15.29:0/2430579122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb64d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 0x7f2c340bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb64e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c3a7fc640 1 -- 172.21.15.29:0/3640138393 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2c440e76e0 con 0x7f2c481025b0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/2430579122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 msgr2=0x560ddb64e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb64e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/2430579122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 msgr2=0x560ddb64ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/2430579122 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddb574530 con 0x560ddb27c2f0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb64d4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c440e5240 tx=0x7f2c440139b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc29d4350bb1e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 0x7f2c340bad80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009bd0 tx=0x7f2c3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.755+0000 7f2c337fe640 1 -- 172.21.15.29:0/2430579122 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440ad650 con 0x560ddb27c2f0 2024-09-21T01:56:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.756+0000 7f2c337fe640 1 -- 172.21.15.29:0/2430579122 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c44013190 con 0x560ddb27c2f0 2024-09-21T01:56:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.756+0000 7f2c337fe640 1 -- 172.21.15.29:0/2430579122 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44005180 con 0x560ddb27c2f0 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 -- 172.21.15.29:0/2430579122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 msgr2=0x560ddb64d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb64d4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c440e5240 tx=0x7f2c440139b0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 -- 172.21.15.29:0/2430579122 shutdown_connections 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb64e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb64d4c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 --2- 172.21.15.29:0/2430579122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2a51b0 0x560ddb64ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 -- 172.21.15.29:0/2430579122 >> 172.21.15.29:0/2430579122 conn(0x560ddb299300 msgr2=0x560ddb6488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 -- 172.21.15.29:0/2430579122 shutdown_connections 2024-09-21T01:56:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.758+0000 7f2c56878740 1 -- 172.21.15.29:0/2430579122 wait complete. 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 -- start start 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 Processor -- start 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560ddb4e6ab0 con 0x560ddb2844d0 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560ddb554730 con 0x560ddb27c2f0 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560ddb64eb30 con 0x560ddb2a51b0 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c56878740 10 client.0 _ll_register_callbacks cb 0x560ddb5a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.759+0000 7f2c4cead640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4cead640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46454/0 (socket says 172.21.15.29:46454) 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47872/0 (socket says 172.21.15.29:47872) 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51464/0 (socket says 172.21.15.29:51464) 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4cead640 1 -- 172.21.15.29:0/141087621 learned_addr learned my addr 172.21.15.29:0/141087621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 -- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 msgr2=0x560ddb656590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 -- 172.21.15.29:0/141087621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 msgr2=0x560ddb656c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 -- 172.21.15.29:0/141087621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddb310790 con 0x560ddb2a51b0 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4cead640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009ca0 tx=0x7f2c40005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831d3c93dc754f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4002fc50 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2c4002fdf0 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.760+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ddb5c1640 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2c40052050 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 10 client.26920 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560ddb389620 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ddb5c1640 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f2c4003e4c0 con 0x560ddb2a51b0 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 10 client.26920 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 20 client.26920 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb263d80 0x560ddb663cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 4 client.26920 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ddb5bd0f0 con 0x560ddb263d80 2024-09-21T01:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.761+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb263d80 0x560ddb663cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.762+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb263d80 0x560ddb663cb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f821fb0120fc74 server_cookie=c30d7ccfc8c79b0b in_seq=0 out_seq=0 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.762+0000 7f2c30ff9640 10 client.26920 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c30ff9640 1 -- 172.21.15.29:0/141087621 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ddb5bd0f0 con 0x560ddb263d80 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c30ff9640 10 client.26920 handle_command_reply: tid=0 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 client.26920 shutdown 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 msgr2=0x560ddb6577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009ca0 tx=0x7f2c40005fd0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 shutdown_connections 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb263d80 0x560ddb663cb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb2a51b0 0x560ddb6577c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb656c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c30ff9640 0 client.26920 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 --2- 172.21.15.29:0/141087621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb656590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 >> 172.21.15.29:0/141087621 conn(0x560ddb299300 msgr2=0x560ddb646ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 shutdown_connections 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.763+0000 7f2c56878740 1 -- 172.21.15.29:0/141087621 wait complete. 2024-09-21T01:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.764+0000 7f2c56878740 20 client.26920 trim_cache size 0 max 16384 2024-09-21T01:56:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.796+0000 7f2c56878740 1 Processor -- start 2024-09-21T01:56:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.798+0000 7f2c56878740 1 -- start start 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb263d80 0x560ddb648220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb655060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481a8c90 con 0x560ddb2844d0 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481aa890 con 0x560ddb27c2f0 2024-09-21T01:56:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.799+0000 7f2c56878740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481a9360 con 0x560ddb263d80 2024-09-21T01:56:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.805+0000 7f2c4cead640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.805+0000 7f2c4cead640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47882/0 (socket says 172.21.15.29:47882) 2024-09-21T01:56:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.805+0000 7f2c4cead640 1 -- 172.21.15.29:0/3318740831 learned_addr learned my addr 172.21.15.29:0/3318740831 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.806+0000 7f2c4cead640 1 -- 172.21.15.29:0/3318740831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb263d80 msgr2=0x560ddb648220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:56:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.806+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb263d80 0x560ddb648220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.806+0000 7f2c4cead640 1 -- 172.21.15.29:0/3318740831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 msgr2=0x560ddb655060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.806+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb655060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.806+0000 7f2c4cead640 1 -- 172.21.15.29:0/3318740831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddb647fd0 con 0x560ddb27c2f0 2024-09-21T01:56:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.810+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c004360 tx=0x7f2c3c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9f4fb9cbdcac8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.815+0000 7f2c32ffd640 1 -- 172.21.15.29:0/3318740831 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00eca0 con 0x560ddb27c2f0 2024-09-21T01:56:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.815+0000 7f2c32ffd640 1 -- 172.21.15.29:0/3318740831 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c3c00ee40 con 0x560ddb27c2f0 2024-09-21T01:56:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.815+0000 7f2c32ffd640 1 -- 172.21.15.29:0/3318740831 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c01d8d0 con 0x560ddb27c2f0 2024-09-21T01:56:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.816+0000 7f2c56878740 1 -- 172.21.15.29:0/3318740831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 msgr2=0x560ddb654b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.816+0000 7f2c56878740 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c004360 tx=0x7f2c3c014000 comp rx=0 tx=0).stop 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.819+0000 7f2c56878740 1 -- 172.21.15.29:0/3318740831 shutdown_connections 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.819+0000 7f2c56878740 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb2844d0 0x560ddb655060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.819+0000 7f2c56878740 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb27c2f0 0x560ddb654b20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.819+0000 7f2c56878740 1 --2- 172.21.15.29:0/3318740831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb263d80 0x560ddb648220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.819+0000 7f2c56878740 1 -- 172.21.15.29:0/3318740831 >> 172.21.15.29:0/3318740831 conn(0x560ddb299300 msgr2=0x560ddb64de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.820+0000 7f2c56878740 1 -- 172.21.15.29:0/3318740831 shutdown_connections 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.822+0000 7f2c56878740 1 -- 172.21.15.29:0/3318740831 wait complete. 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.822+0000 7f2c56878740 1 -- start start 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.822+0000 7f2c56878740 1 Processor -- start 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb6cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb6589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481aa1c0 con 0x560ddb263d80 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481a9af0 con 0x560ddb2844d0 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481a6c70 con 0x560ddb27c2f0 2024-09-21T01:56:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.823+0000 7f2c56878740 10 client.0 _ll_register_callbacks cb 0x7f2c40009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.825+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.825+0000 7f2c4d6ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46466/0 (socket says 172.21.15.29:46466) 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.825+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3436682642 learned_addr learned my addr 172.21.15.29:0/3436682642 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.825+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb6cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.826+0000 7f2c4deaf640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb6589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.827+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3436682642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 msgr2=0x560ddb6cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.827+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb6cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.827+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 msgr2=0x560ddb6589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.827+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb6589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.827+0000 7f2c4d6ae640 1 -- 172.21.15.29:0/3436682642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddb659160 con 0x560ddb263d80 2024-09-21T01:56:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.831+0000 7f2c4d6ae640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2c440ad580 tx=0x7f2c440a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8f854e7cb727f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.831+0000 7f2c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.831+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440c7070 con 0x560ddb263d80 2024-09-21T01:56:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.831+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2c440b6e10 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.832+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ddb5ba520 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.832+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2c440ce050 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.832+0000 7f2c56878740 10 client.17157 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.832+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560ddb65e480 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.832+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ddb5ba520 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f2c440aebb0 con 0x560ddb263d80 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 10 client.17157 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 20 client.17157 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb2a51b0 0x560ddb6ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 4 client.17157 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:56:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ddb530800 con 0x560ddb2a51b0 2024-09-21T01:56:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.833+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb2a51b0 0x560ddb6ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.834+0000 7f2c30ff9640 10 client.17157 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.834+0000 7f2c4cead640 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb2a51b0 0x560ddb6ea640 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14dbfe671ac695d2 server_cookie=e59718672f2d8069 in_seq=0 out_seq=0 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c30ff9640 1 -- 172.21.15.29:0/3436682642 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ddb530800 con 0x560ddb2a51b0 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c30ff9640 10 client.17157 handle_command_reply: tid=0 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 client.17157 shutdown 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 msgr2=0x560ddb6cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2c440ad580 tx=0x7f2c440a98e0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 shutdown_connections 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560ddb2a51b0 0x560ddb6ea640 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560ddb2844d0 0x560ddb6589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ddb27c2f0 0x560ddb6cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 --2- 172.21.15.29:0/3436682642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560ddb263d80 0x560ddb6cbd40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 >> 172.21.15.29:0/3436682642 conn(0x560ddb299300 msgr2=0x560ddb648200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.864+0000 7f2c30ff9640 0 client.17157 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.865+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 shutdown_connections 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.865+0000 7f2c56878740 1 -- 172.21.15.29:0/3436682642 wait complete. 2024-09-21T01:56:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.865+0000 7f2c56878740 20 client.17157 trim_cache size 0 max 16384 2024-09-21T01:56:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:56:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:56:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:56:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 msgr2=0x7f2c340bad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 0x7f2c340bad80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009bd0 tx=0x7f2c3c005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 msgr2=0x7f2c48197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9f0 tx=0x7f2c4400edf0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 shutdown_connections 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c340b8870 0x7f2c340bad80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48103fd0 0x7f2c48198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c48103630 0x7f2c48198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 --2- 172.21.15.29:0/3640138393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c481025b0 0x7f2c48197cf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 >> 172.21.15.29:0/3640138393 conn(0x7f2c480fdf30 msgr2=0x7f2c48106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 shutdown_connections 2024-09-21T01:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:46.868+0000 7f2c4f912640 1 -- 172.21.15.29:0/3640138393 wait complete. 2024-09-21T01:56:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 490.0 s, 2214.8 tps, lat 45.873 ms stddev 55.587 2024-09-21T01:56:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:47 smithi190 ceph-mon[34368]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 732 op/s 2024-09-21T01:56:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:47 smithi066 ceph-mon[35001]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 732 op/s 2024-09-21T01:56:48.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:47 smithi029 ceph-mon[31228]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 732 op/s 2024-09-21T01:56:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:49 smithi190 ceph-mon[34368]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-09-21T01:56:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:49 smithi066 ceph-mon[35001]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-09-21T01:56:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:49 smithi029 ceph-mon[31228]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-09-21T01:56:51.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:56:51.589 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:52.059 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:51 smithi029 ceph-mon[31228]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 784 op/s 2024-09-21T01:56:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 -- 172.21.15.29:0/1220588465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c102a60 msgr2=0x7ff43c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 --2- 172.21.15.29:0/1220588465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c102a60 0x7ff43c102e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff438009900 tx=0x7ff43802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 -- 172.21.15.29:0/1220588465 shutdown_connections 2024-09-21T01:56:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 --2- 172.21.15.29:0/1220588465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c108cf0 0x7ff43c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 --2- 172.21.15.29:0/1220588465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c1083a0 0x7ff43c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 --2- 172.21.15.29:0/1220588465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c102a60 0x7ff43c102e70 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff438009900 tx=0x7ff43802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 -- 172.21.15.29:0/1220588465 >> 172.21.15.29:0/1220588465 conn(0x7ff43c079730 msgr2=0x7ff43c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.203+0000 7ff44421b640 1 -- 172.21.15.29:0/1220588465 shutdown_connections 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.204+0000 7ff44421b640 1 -- 172.21.15.29:0/1220588465 wait complete. 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 Processor -- start 2024-09-21T01:56:52.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 -- start start 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c108cf0 0x7ff43c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 0x7ff43c1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff44421b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112960 con 0x7ff43c19bd70 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.205+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40494/0 (socket says 172.21.15.29:40494) 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff44421b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1127e0 con 0x7ff43c108cf0 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 -- 172.21.15.29:0/3882248465 learned_addr learned my addr 172.21.15.29:0/3882248465 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112ae0 con 0x7ff43c1083a0 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 0x7ff43c1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff4417b6640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c108cf0 0x7ff43c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 -- 172.21.15.29:0/3882248465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c108cf0 msgr2=0x7ff43c19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c108cf0 0x7ff43c19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 -- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 msgr2=0x7ff43c1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 0x7ff43c1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 -- 172.21.15.29:0/3882248465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c1a6a70 con 0x7ff43c1083a0 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 0x7ff43c1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:52.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff438006010 tx=0x7ff4380040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.206+0000 7ff432ffd640 1 -- 172.21.15.29:0/3882248465 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43803f030 con 0x7ff43c1083a0 2024-09-21T01:56:52.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.207+0000 7ff432ffd640 1 -- 172.21.15.29:0/3882248465 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4380044c0 con 0x7ff43c1083a0 2024-09-21T01:56:52.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.207+0000 7ff432ffd640 1 -- 172.21.15.29:0/3882248465 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438035070 con 0x7ff43c1083a0 2024-09-21T01:56:52.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.207+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1a6d00 con 0x7ff43c1083a0 2024-09-21T01:56:52.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.207+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a70d0 con 0x7ff43c1083a0 2024-09-21T01:56:52.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.209+0000 7ff432ffd640 1 -- 172.21.15.29:0/3882248465 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff438042030 con 0x7ff43c1083a0 2024-09-21T01:56:52.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.210+0000 7ff44a78f740 1 Processor -- start 2024-09-21T01:56:52.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.210+0000 7ff44a78f740 1 -- start start 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.210+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 0x5563ce12e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.210+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce12eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.210+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce12f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.211+0000 7ff44a78f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563ce0a4e50 con 0x5563cdd87070 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.211+0000 7ff44a78f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563ce094280 con 0x5563cdd66390 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.211+0000 7ff44a78f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563ce098090 con 0x5563cdd5e1b0 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff4427b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 0x5563ce12e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff4427b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 0x5563ce12e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59994/0 (socket says 172.21.15.29:59994) 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff4427b8640 1 -- 172.21.15.29:0/3797303603 learned_addr learned my addr 172.21.15.29:0/3797303603 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff432ffd640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 0x7ff4280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce12eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 -- 172.21.15.29:0/3797303603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 msgr2=0x5563ce12f0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce12f0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 -- 172.21.15.29:0/3797303603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 msgr2=0x5563ce12e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 0x5563ce12e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.212+0000 7ff441fb7640 1 -- 172.21.15.29:0/3797303603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cddf2650 con 0x5563cdd5e1b0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 0x7ff4280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff432ffd640 1 -- 172.21.15.29:0/3882248465 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff4381079b0 con 0x7ff43c1083a0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce12eb90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cadc0 tx=0x7ff4380ce310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69f1a9f54a06b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff427fff640 1 -- 172.21.15.29:0/3797303603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4380c9dc0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff427fff640 1 -- 172.21.15.29:0/3797303603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4380d2020 con 0x5563cdd5e1b0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.213+0000 7ff427fff640 1 -- 172.21.15.29:0/3797303603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4380d2430 con 0x5563cdd5e1b0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 -- 172.21.15.29:0/3797303603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 msgr2=0x5563ce12eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce12eb90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cadc0 tx=0x7ff4380ce310 comp rx=0 tx=0).stop 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 0x7ff4280b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff434004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 -- 172.21.15.29:0/3797303603 shutdown_connections 2024-09-21T01:56:52.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce12f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce12eb90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3797303603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd87070 0x5563ce12e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 -- 172.21.15.29:0/3797303603 >> 172.21.15.29:0/3797303603 conn(0x5563cdd7b460 msgr2=0x5563ce12c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 -- 172.21.15.29:0/3797303603 shutdown_connections 2024-09-21T01:56:52.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.215+0000 7ff44a78f740 1 -- 172.21.15.29:0/3797303603 wait complete. 2024-09-21T01:56:52.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.216+0000 7ff44a78f740 1 -- start start 2024-09-21T01:56:52.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.216+0000 7ff44a78f740 1 Processor -- start 2024-09-21T01:56:52.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.216+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 0x5563ce13a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff4417b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 0x5563ce13a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff4417b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 0x5563ce13a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60000/0 (socket says 172.21.15.29:60000) 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd87070 0x5563ce145d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40520/0 (socket says 172.21.15.29:40520) 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 -- 172.21.15.29:0/703280254 learned_addr learned my addr 172.21.15.29:0/703280254 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5563cdef5700 con 0x5563cdd66390 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563ce02f110 con 0x5563cdd87070 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563ce0a4e50 con 0x5563cdd5e1b0 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff4427b8640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd87070 0x5563ce145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff44a78f740 10 client.0 _ll_register_callbacks cb 0x5563ce048990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 -- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd87070 msgr2=0x5563ce145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd87070 0x5563ce145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.217+0000 7ff441fb7640 1 -- 172.21.15.29:0/703280254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 msgr2=0x5563ce13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.218+0000 7ff441fb7640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 0x5563ce13a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.218+0000 7ff441fb7640 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ce05cc40 con 0x5563cdd5e1b0 2024-09-21T01:56:52.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.218+0000 7ff441fb7640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cd850 tx=0x7ff43802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e451e8dfe9db6b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.218+0000 7ff4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:56:52.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.218+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4380d21d0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff4380d2370 con 0x5563cdd5e1b0 2024-09-21T01:56:52.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4380d2020 con 0x5563cdd5e1b0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff4380d70e0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff44a78f740 10 client.26935 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563ce148730 con 0x5563cdd5e1b0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.219+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4380d7280 con 0x5563cdd5e1b0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff4380dbbd0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 10 client.26935 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 20 client.26935 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd459f0 0x5563ce14ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 4 client.26935 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4380d2020 con 0x5563cdd459f0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.220+0000 7ff4417b6640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd459f0 0x5563ce14ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.221+0000 7ff4417b6640 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd459f0 0x5563ce14ba90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a113a4ef9e8876 server_cookie=f486e050b3ca1760 in_seq=0 out_seq=0 2024-09-21T01:56:52.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.221+0000 7ff4257fa640 10 client.26935 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.221+0000 7ff4257fa640 1 -- 172.21.15.29:0/703280254 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff4380d2020 con 0x5563cdd459f0 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.221+0000 7ff4257fa640 10 client.26935 handle_command_reply: tid=0 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.221+0000 7ff44a78f740 1 client.26935 shutdown 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 msgr2=0x5563ce139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cd850 tx=0x7ff43802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 shutdown_connections 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd459f0 0x5563ce14ba90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff4257fa640 0 client.26935 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:52.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd87070 0x5563ce145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd66390 0x5563ce13a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 --2- 172.21.15.29:0/703280254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd5e1b0 0x5563ce139d00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 >> 172.21.15.29:0/703280254 conn(0x5563cdd7b460 msgr2=0x5563ce131830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 shutdown_connections 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 1 -- 172.21.15.29:0/703280254 wait complete. 2024-09-21T01:56:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.222+0000 7ff44a78f740 20 client.26935 trim_cache size 0 max 16384 2024-09-21T01:56:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:51 smithi190 ceph-mon[34368]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 784 op/s 2024-09-21T01:56:52.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.248+0000 7ff44a78f740 1 Processor -- start 2024-09-21T01:56:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.248+0000 7ff44a78f740 1 -- start start 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.248+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd459f0 0x5563ce12fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff44a78f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c074f10 con 0x5563cdd5e1b0 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff44a78f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0739e0 con 0x5563cdd66390 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff44a78f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c074170 con 0x5563cdd459f0 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60004/0 (socket says 172.21.15.29:60004) 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 -- 172.21.15.29:0/1396420229 learned_addr learned my addr 172.21.15.29:0/1396420229 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff441fb7640 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd459f0 0x5563ce12fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4427b8640 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 -- 172.21.15.29:0/1396420229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd459f0 msgr2=0x5563ce12fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd459f0 0x5563ce12fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 -- 172.21.15.29:0/1396420229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 msgr2=0x5563ce136ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce136ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 -- 172.21.15.29:0/1396420229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ce130940 con 0x5563cdd5e1b0 2024-09-21T01:56:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.249+0000 7ff4417b6640 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff43c06aaa0 tx=0x7ff42c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481614e3a313c4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.250+0000 7ff4277fe640 1 -- 172.21.15.29:0/1396420229 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004be0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.250+0000 7ff4277fe640 1 -- 172.21.15.29:0/1396420229 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42c004d80 con 0x5563cdd5e1b0 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.250+0000 7ff4277fe640 1 -- 172.21.15.29:0/1396420229 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c011440 con 0x5563cdd5e1b0 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.250+0000 7ff44a78f740 1 -- 172.21.15.29:0/1396420229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 msgr2=0x5563ce145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.250+0000 7ff44a78f740 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff43c06aaa0 tx=0x7ff42c0054d0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 -- 172.21.15.29:0/1396420229 shutdown_connections 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd66390 0x5563ce136ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd5e1b0 0x5563ce145c60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 --2- 172.21.15.29:0/1396420229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd459f0 0x5563ce12fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 -- 172.21.15.29:0/1396420229 >> 172.21.15.29:0/1396420229 conn(0x5563cdd7b460 msgr2=0x5563ce12f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 -- 172.21.15.29:0/1396420229 shutdown_connections 2024-09-21T01:56:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 -- 172.21.15.29:0/1396420229 wait complete. 2024-09-21T01:56:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 -- start start 2024-09-21T01:56:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.251+0000 7ff44a78f740 1 Processor -- start 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 0x5563ce13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 0x5563ce13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 0x5563ce1af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 0x5563ce13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0712f0 con 0x5563cdd459f0 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c074840 con 0x5563cdd5e1b0 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c071960 con 0x5563cdd66390 2024-09-21T01:56:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff441fb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 0x5563ce13e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60020/0 (socket says 172.21.15.29:60020) 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff441fb7640 1 -- 172.21.15.29:0/3263262793 learned_addr learned my addr 172.21.15.29:0/3263262793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff4417b6640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 0x5563ce13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 0x5563ce1af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.252+0000 7ff44a78f740 10 client.0 _ll_register_callbacks cb 0x7ff434009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 -- 172.21.15.29:0/3263262793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 msgr2=0x5563ce1af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 0x5563ce1af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 -- 172.21.15.29:0/3263262793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 msgr2=0x5563ce13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 0x5563ce13e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 -- 172.21.15.29:0/3263262793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ce1afb80 con 0x5563cdd5e1b0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4427b8640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 0x5563ce1af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4417b6640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 0x5563ce13ed90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff42c004700 tx=0x7ff42c003940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e0205cf75b48cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c028040 con 0x5563cdd5e1b0 2024-09-21T01:56:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c004d80 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff42c01f070 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff42c0035f0 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff44a78f740 10 client.26960 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.253+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563ce1b6f60 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42c004d80 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff42c02b070 con 0x5563cdd5e1b0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 10 client.26960 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 20 client.26960 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd87070 0x5563ce1b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 4 client.26960 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563ce0a6da0 con 0x5563cdd87070 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd87070 0x5563ce1b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.254+0000 7ff4257fa640 10 client.26960 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff441fb7640 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd87070 0x5563ce1b96f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a9cf36c81f732f server_cookie=6ba7d299ff496530 in_seq=0 out_seq=0 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff4257fa640 1 -- 172.21.15.29:0/3263262793 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x5563ce0a6da0 con 0x5563cdd87070 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff4257fa640 10 client.26960 handle_command_reply: tid=0 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 client.26960 shutdown 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 msgr2=0x5563ce13ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 0x5563ce13ed90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff42c004700 tx=0x7ff42c003940 comp rx=0 tx=0).stop 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 shutdown_connections 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5563cdd87070 0x5563ce1b96f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563cdd66390 0x5563ce1af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563cdd5e1b0 0x5563ce13ed90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 --2- 172.21.15.29:0/3263262793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5563cdd459f0 0x5563ce13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 >> 172.21.15.29:0/3263262793 conn(0x5563cdd7b460 msgr2=0x5563ce12f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff4257fa640 0 client.26960 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 shutdown_connections 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 1 -- 172.21.15.29:0/3263262793 wait complete. 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.255+0000 7ff44a78f740 20 client.26960 trim_cache size 0 max 16384 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:56:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.263 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.264 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:56:52.265 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:56:52.266 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 msgr2=0x7ff4280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 0x7ff4280b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff434004a40 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 msgr2=0x7ff43c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff438006010 tx=0x7ff4380040c0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 shutdown_connections 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff4280b6510 0x7ff4280b8a20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c19bd70 0x7ff43c1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff43c108cf0 0x7ff43c19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 --2- 172.21.15.29:0/3882248465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff43c1083a0 0x7ff43c19b2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 >> 172.21.15.29:0/3882248465 conn(0x7ff43c079730 msgr2=0x7ff43c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 shutdown_connections 2024-09-21T01:56:52.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:52.256+0000 7ff44421b640 1 -- 172.21.15.29:0/3882248465 wait complete. 2024-09-21T01:56:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:51 smithi066 ceph-mon[35001]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 784 op/s 2024-09-21T01:56:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 495.0 s, 1770.4 tps, lat 55.454 ms stddev 65.494 2024-09-21T01:56:53.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:56:53.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:56:53.627 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:54.077 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:53 smithi029 ceph-mon[31228]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-21T01:56:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:53 smithi190 ceph-mon[34368]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 -- 172.21.15.29:0/3150073440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc11cba0 msgr2=0x7ff3fc11efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 --2- 172.21.15.29:0/3150073440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc11efc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff3f401acb0 tx=0x7ff3f403b920 comp rx=0 tx=0).stop 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 -- 172.21.15.29:0/3150073440 shutdown_connections 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 --2- 172.21.15.29:0/3150073440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc11efc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 --2- 172.21.15.29:0/3150073440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 --2- 172.21.15.29:0/3150073440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc111cd0 0x7ff3fc10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.235+0000 7ff402cb2640 1 -- 172.21.15.29:0/3150073440 >> 172.21.15.29:0/3150073440 conn(0x7ff3fc06e9f0 msgr2=0x7ff3fc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.236+0000 7ff402cb2640 1 -- 172.21.15.29:0/3150073440 shutdown_connections 2024-09-21T01:56:54.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.236+0000 7ff402cb2640 1 -- 172.21.15.29:0/3150073440 wait complete. 2024-09-21T01:56:54.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.237+0000 7ff402cb2640 1 Processor -- start 2024-09-21T01:56:54.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.237+0000 7ff402cb2640 1 -- start start 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc111cd0 0x7ff3fc118700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc118c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc114c50 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc114ad0 con 0x7ff3fc111cd0 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.240+0000 7ff402cb2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc114dd0 con 0x7ff3fc11cba0 2024-09-21T01:56:54.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:54.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60030/0 (socket says 172.21.15.29:60030) 2024-09-21T01:56:54.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 -- 172.21.15.29:0/2700585523 learned_addr learned my addr 172.21.15.29:0/2700585523 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:54.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff4014af640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc111cd0 0x7ff3fc118700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff4024b1640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc118c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 -- 172.21.15.29:0/2700585523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 msgr2=0x7ff3fc118c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc118c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 -- 172.21.15.29:0/2700585523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc111cd0 msgr2=0x7ff3fc118700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc111cd0 0x7ff3fc118700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.241+0000 7ff401cb0640 1 -- 172.21.15.29:0/2700585523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc1194a0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.242+0000 7ff4024b1640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc118c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:54.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.242+0000 7ff401cb0640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800e9c0 tx=0x7ff3f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:54.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.244+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800cd80 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.244+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc1accc0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.244+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1ad250 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.244+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f80035f0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.244+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8010690 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.247+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e00052c0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.249+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff3f8010830 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.250+0000 7ff3f2ffd640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 0x7ff3e80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:54.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.250+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff3f80e7530 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.251+0000 7ff4014af640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 0x7ff3e80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:54.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.251+0000 7ff4014af640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 0x7ff3e80b8b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006fd0 tx=0x7ff3ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:54.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.256+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3f80adee0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:53 smithi066 ceph-mon[35001]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-21T01:56:54.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.632+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e00058b0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.636+0000 7ff3f2ffd640 1 -- 172.21.15.29:0/2700585523 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7ff3f80b22c0 con 0x7ff3fc10dbf0 2024-09-21T01:56:54.637 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:56:54.638 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"btime":"2024-09-21T01:55:51:710037+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:55:51.255007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:56:54.638 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-09-21T01:56:54.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 msgr2=0x7ff3e80b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:54.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 0x7ff3e80b8b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006fd0 tx=0x7ff3ec008040 comp rx=0 tx=0).stop 2024-09-21T01:56:54.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 msgr2=0x7ff3fc11c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:54.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800e9c0 tx=0x7ff3f800ee80 comp rx=0 tx=0).stop 2024-09-21T01:56:54.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 shutdown_connections 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff3e80b6660 0x7ff3e80b8b70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3fc11cba0 0x7ff3fc118c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3fc111cd0 0x7ff3fc118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 --2- 172.21.15.29:0/2700585523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3fc10dbf0 0x7ff3fc11c420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 >> 172.21.15.29:0/2700585523 conn(0x7ff3fc06e9f0 msgr2=0x7ff3fc11cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 shutdown_connections 2024-09-21T01:56:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:54.640+0000 7ff402cb2640 1 -- 172.21.15.29:0/2700585523 wait complete. 2024-09-21T01:56:55.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:54 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2700585523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:55.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:54 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2700585523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:55.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:54 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2700585523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:56:55.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:56:56.207 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:55 smithi190 ceph-mon[34368]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-21T01:56:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:55 smithi066 ceph-mon[35001]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-21T01:56:56.328 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:55 smithi029 ceph-mon[31228]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.293+0000 7f28deb22640 1 -- 172.21.15.29:0/3388652040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d8102ae0 msgr2=0x7f28d8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.293+0000 7f28deb22640 1 --2- 172.21.15.29:0/3388652040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d8102ae0 0x7f28d8102ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00d910 tx=0x7f28cc02c390 comp rx=0 tx=0).stop 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.293+0000 7f28deb22640 1 -- 172.21.15.29:0/3388652040 shutdown_connections 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 --2- 172.21.15.29:0/3388652040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8108d20 0x7f28d810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 --2- 172.21.15.29:0/3388652040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d81083d0 0x7f28d81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 --2- 172.21.15.29:0/3388652040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d8102ae0 0x7f28d8102ef0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 -- 172.21.15.29:0/3388652040 >> 172.21.15.29:0/3388652040 conn(0x7f28d8079790 msgr2=0x7f28d8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 -- 172.21.15.29:0/3388652040 shutdown_connections 2024-09-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.294+0000 7f28deb22640 1 -- 172.21.15.29:0/3388652040 wait complete. 2024-09-21T01:56:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.295+0000 7f28deb22640 1 Processor -- start 2024-09-21T01:56:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.295+0000 7f28deb22640 1 -- start start 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28deb22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 0x7f28d8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28deb22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 0x7f28d81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28deb22640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d8108d20 0x7f28d819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 0x7f28d8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 0x7f28d8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60052/0 (socket says 172.21.15.29:60052) 2024-09-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28dc8be640 1 -- 172.21.15.29:0/481553145 learned_addr learned my addr 172.21.15.29:0/481553145 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28d7fff640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 0x7f28d81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.296+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8111980 con 0x7f28d8102ae0 2024-09-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8111800 con 0x7f28d81083d0 2024-09-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8111b00 con 0x7f28d8108d20 2024-09-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d8108d20 0x7f28d819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28d7fff640 1 -- 172.21.15.29:0/481553145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d8108d20 msgr2=0x7f28d819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28d7fff640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d8108d20 0x7f28d819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28d7fff640 1 -- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 msgr2=0x7f28d8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28d7fff640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 0x7f28d8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.297+0000 7f28d7fff640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d819e2a0 con 0x7f28d81083d0 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.298+0000 7f28d7fff640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 0x7f28d81988d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f28d806ab20 tx=0x7f28c400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.298+0000 7f28d5ffb640 1 -- 172.21.15.29:0/481553145 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c401d070 con 0x7f28d81083d0 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.298+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d819e590 con 0x7f28d81083d0 2024-09-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.298+0000 7f28d5ffb640 1 -- 172.21.15.29:0/481553145 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c4018440 con 0x7f28d81083d0 2024-09-21T01:56:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.298+0000 7f28d5ffb640 1 -- 172.21.15.29:0/481553145 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40165a0 con 0x7f28d81083d0 2024-09-21T01:56:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.300+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81a4cf0 con 0x7f28d81083d0 2024-09-21T01:56:57.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.302+0000 7f28d5ffb640 1 -- 172.21.15.29:0/481553145 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f28c4020070 con 0x7f28d81083d0 2024-09-21T01:56:57.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.303+0000 7f28d5ffb640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 0x7f28c80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.304+0000 7f28dc8be640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 0x7f28c80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.304+0000 7f28e5096740 1 Processor -- start 2024-09-21T01:56:57.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.304+0000 7f28d5ffb640 1 -- 172.21.15.29:0/481553145 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f28c40ee060 con 0x7f28d81083d0 2024-09-21T01:56:57.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 -- start start 2024-09-21T01:56:57.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe85995ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe85996550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28dc8be640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 0x7f28c80b8ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc02c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fe85903ea0 con 0x55fe855c5390 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.305+0000 7f28e5096740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fe858f32d0 con 0x55fe855e6070 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28e5096740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fe858f70e0 con 0x55fe855bd1b0 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40570/0 (socket says 172.21.15.29:40570) 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 -- 172.21.15.29:0/1247240764 learned_addr learned my addr 172.21.15.29:0/1247240764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28dc8be640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe85995ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe85996550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 -- 172.21.15.29:0/1247240764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 msgr2=0x55fe85995ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe85995ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 -- 172.21.15.29:0/1247240764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 msgr2=0x55fe85996550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe85996550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 -- 172.21.15.29:0/1247240764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe85651650 con 0x55fe855bd1b0 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe85996550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.306+0000 7f28d7fff640 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f28c40ad3e0 tx=0x7f28c40b0000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d66ab21497a94b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.307+0000 7f28c2ffd640 1 -- 172.21.15.29:0/1247240764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40b4ba0 con 0x55fe855bd1b0 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.307+0000 7f28c2ffd640 1 -- 172.21.15.29:0/1247240764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c40ed020 con 0x55fe855bd1b0 2024-09-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.307+0000 7f28c2ffd640 1 -- 172.21.15.29:0/1247240764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40abd20 con 0x55fe855bd1b0 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 -- 172.21.15.29:0/1247240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 msgr2=0x55fe85996010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f28c40ad3e0 tx=0x7f28c40b0000 comp rx=0 tx=0).stop 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 -- 172.21.15.29:0/1247240764 shutdown_connections 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe85996550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855bd1b0 0x55fe85996010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 --2- 172.21.15.29:0/1247240764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe85995ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.308+0000 7f28e5096740 1 -- 172.21.15.29:0/1247240764 >> 172.21.15.29:0/1247240764 conn(0x55fe855da460 msgr2=0x55fe859939c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.309+0000 7f28e5096740 1 -- 172.21.15.29:0/1247240764 shutdown_connections 2024-09-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.309+0000 7f28e5096740 1 -- 172.21.15.29:0/1247240764 wait complete. 2024-09-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.309+0000 7f28e5096740 1 -- start start 2024-09-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.310+0000 7f28e5096740 1 Processor -- start 2024-09-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.310+0000 7f28e5096740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 0x55fe859a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28e5096740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28e5096740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe859ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 0x55fe859a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 0x55fe859a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60064/0 (socket says 172.21.15.29:60064) 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40586/0 (socket says 172.21.15.29:40586) 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28dc8be640 1 -- 172.21.15.29:0/1104086221 learned_addr learned my addr 172.21.15.29:0/1104086221 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.311+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe859ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fe85754700 con 0x55fe855bd1b0 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fe8588e160 con 0x55fe855e6070 2024-09-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fe859a4250 con 0x55fe855c5390 2024-09-21T01:56:57.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28dd0bf640 1 -- 172.21.15.29:0/1104086221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 msgr2=0x55fe859a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28dd0bf640 1 -- 172.21.15.29:0/1104086221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 msgr2=0x55fe859a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 0x55fe859a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.312+0000 7f28dd0bf640 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe858bbc90 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe859ad570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d0011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72dc70730145184e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28e5096740 10 client.0 _ll_register_callbacks cb 0x55fe858a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c40ed020 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28b7fff640 10 client.26975 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28b7fff640 1 -- 172.21.15.29:0/1104086221 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d000fc50 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28b7fff640 1 -- 172.21.15.29:0/1104086221 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f28d000fdf0 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28b7fff640 1 -- 172.21.15.29:0/1104086221 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28d0004b20 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28e5096740 10 client.26975 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.313+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55fe859affb0 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28b7fff640 1 -- 172.21.15.29:0/1104086221 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f28d0004cc0 con 0x55fe855e6070 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 10 client.26975 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 10 client.26975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 20 client.26975 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe855a49f0 0x55fe859b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 4 client.26975 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe85905df0 con 0x55fe855a49f0 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.314+0000 7f28dc8be640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe855a49f0 0x55fe859b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.315+0000 7f28dc8be640 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe855a49f0 0x55fe859b3900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34d59a6d93c565a5 server_cookie=8e7ea1c5fd6572f1 in_seq=0 out_seq=0 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.315+0000 7f28b7fff640 10 client.26975 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.315+0000 7f28b7fff640 1 -- 172.21.15.29:0/1104086221 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe85905df0 con 0x55fe855a49f0 2024-09-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.316+0000 7f28b7fff640 10 client.26975 handle_command_reply: tid=0 2024-09-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.316+0000 7f28e5096740 1 client.26975 shutdown 2024-09-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 msgr2=0x55fe859ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe859ad570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d0011e60 comp rx=0 tx=0).stop 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 shutdown_connections 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe855a49f0 0x55fe859b3900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855e6070 0x55fe859ad570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 --2- 172.21.15.29:0/1104086221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855bd1b0 0x55fe859a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 >> 172.21.15.29:0/1104086221 conn(0x55fe855da460 msgr2=0x55fe85998c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28b7fff640 0 client.26975 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.318+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 shutdown_connections 2024-09-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.319+0000 7f28e5096740 1 -- 172.21.15.29:0/1104086221 wait complete. 2024-09-21T01:56:57.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.319+0000 7f28e5096740 20 client.26975 trim_cache size 0 max 16384 2024-09-21T01:56:57.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.347+0000 7f28e5096740 1 Processor -- start 2024-09-21T01:56:57.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.347+0000 7f28e5096740 1 -- start start 2024-09-21T01:56:57.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28e5096740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28e5096740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe859ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:43980/0 (socket says 172.21.15.29:43980) 2024-09-21T01:56:57.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28dc8be640 1 -- 172.21.15.29:0/4282504029 learned_addr learned my addr 172.21.15.29:0/4282504029 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28e5096740 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855e6070 0x55fe8599e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.348+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ad1f0 con 0x55fe855c5390 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81aedf0 con 0x55fe855bd1b0 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855e6070 0x55fe8599e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ad8c0 con 0x55fe855e6070 2024-09-21T01:56:57.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28d7fff640 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe859ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dc8be640 1 -- 172.21.15.29:0/4282504029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855e6070 msgr2=0x55fe8599e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dc8be640 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855e6070 0x55fe8599e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dc8be640 1 -- 172.21.15.29:0/4282504029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 msgr2=0x55fe859ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dc8be640 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe859ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.349+0000 7f28dc8be640 1 -- 172.21.15.29:0/4282504029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe85998930 con 0x55fe855bd1b0 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.350+0000 7f28dc8be640 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28cc02c980 tx=0x7f28cc03b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8b4d6ca8e397da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.350+0000 7f28c27fc640 1 -- 172.21.15.29:0/4282504029 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc059070 con 0x55fe855bd1b0 2024-09-21T01:56:57.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.350+0000 7f28c27fc640 1 -- 172.21.15.29:0/4282504029 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28cc046070 con 0x55fe855bd1b0 2024-09-21T01:56:57.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.350+0000 7f28c27fc640 1 -- 172.21.15.29:0/4282504029 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc03f420 con 0x55fe855bd1b0 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.351+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 msgr2=0x55fe859a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.351+0000 7f28e5096740 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28cc02c980 tx=0x7f28cc03b000 comp rx=0 tx=0).stop 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 shutdown_connections 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855e6070 0x55fe8599e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855c5390 0x55fe859ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 --2- 172.21.15.29:0/4282504029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a16b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 >> 172.21.15.29:0/4282504029 conn(0x55fe855da460 msgr2=0x55fe859a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 shutdown_connections 2024-09-21T01:56:57.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.352+0000 7f28e5096740 1 -- 172.21.15.29:0/4282504029 wait complete. 2024-09-21T01:56:57.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.353+0000 7f28e5096740 1 -- start start 2024-09-21T01:56:57.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.353+0000 7f28e5096740 1 Processor -- start 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855e6070 0x55fe859a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ae720 con 0x55fe855e6070 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ae050 con 0x55fe855bd1b0 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ab1d0 con 0x55fe855c5390 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28e5096740 10 client.0 _ll_register_callbacks cb 0x7f28d0009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28dc8be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:43986/0 (socket says 172.21.15.29:43986) 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.354+0000 7f28dc8be640 1 -- 172.21.15.29:0/2444937002 learned_addr learned my addr 172.21.15.29:0/2444937002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dd0bf640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855e6070 0x55fe859a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 -- 172.21.15.29:0/2444937002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 msgr2=0x55fe859a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 -- 172.21.15.29:0/2444937002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855e6070 msgr2=0x55fe859a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855e6070 0x55fe859a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 -- 172.21.15.29:0/2444937002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe859a5de0 con 0x55fe855bd1b0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28dc8be640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28cc003d80 tx=0x7f28cc003db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11598205d19afcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc059040 con 0x55fe855bd1b0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc046070 con 0x55fe855bd1b0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.355+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f28cc04e450 con 0x55fe855bd1b0 2024-09-21T01:56:57.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.356+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28cc039040 con 0x55fe855bd1b0 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.356+0000 7f28e5096740 10 client.26981 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.356+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe85a1e200 con 0x55fe855bd1b0 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28cc046070 con 0x55fe855bd1b0 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f28cc04b040 con 0x55fe855bd1b0 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 10 client.26981 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:57.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 10 client.26981 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:56:57.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 20 client.26981 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:57.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe85579da0 0x55fe85a208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:57.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 4 client.26981 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:57.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.357+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe859018a0 con 0x55fe85579da0 2024-09-21T01:56:57.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.359+0000 7f28d7fff640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe85579da0 0x55fe85a208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:57.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28d7fff640 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe85579da0 0x55fe85a208f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a93bd29c883963e server_cookie=ab7151850b7b5c7e in_seq=0 out_seq=0 2024-09-21T01:56:57.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28b7fff640 10 client.26981 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:57.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28b7fff640 1 -- 172.21.15.29:0/2444937002 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe859018a0 con 0x55fe85579da0 2024-09-21T01:56:57.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28b7fff640 10 client.26981 handle_command_reply: tid=0 2024-09-21T01:56:57.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28e5096740 1 client.26981 shutdown 2024-09-21T01:56:57.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 msgr2=0x55fe859a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.360+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28cc003d80 tx=0x7f28cc003db0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 shutdown_connections 2024-09-21T01:56:57.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55fe85579da0 0x55fe85a208f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fe855e6070 0x55fe859a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe855c5390 0x55fe859a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 --2- 172.21.15.29:0/2444937002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe855bd1b0 0x55fe859a9c80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.362+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 >> 172.21.15.29:0/2444937002 conn(0x55fe855da460 msgr2=0x55fe85996b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.363+0000 7f28b7fff640 0 client.26981 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:56:57.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.363+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 shutdown_connections 2024-09-21T01:56:57.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.364+0000 7f28e5096740 1 -- 172.21.15.29:0/2444937002 wait complete. 2024-09-21T01:56:57.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.364+0000 7f28e5096740 20 client.26981 trim_cache size 0 max 16384 2024-09-21T01:56:57.366 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:57.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.366+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 msgr2=0x7f28c80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.366+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 0x7f28c80b8ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc02c890 comp rx=0 tx=0).stop 2024-09-21T01:56:57.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.366+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 msgr2=0x7f28d81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:57.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.366+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 0x7f28d81988d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f28d806ab20 tx=0x7f28c400b850 comp rx=0 tx=0).stop 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 shutdown_connections 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f28c80b65d0 0x7f28c80b8ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d8108d20 0x7f28d819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28d81083d0 0x7f28d81988d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 --2- 172.21.15.29:0/481553145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d8102ae0 0x7f28d8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.367+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 >> 172.21.15.29:0/481553145 conn(0x7f28d8079790 msgr2=0x7f28d8106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:57.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.368+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 shutdown_connections 2024-09-21T01:56:57.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:57.368+0000 7f28deb22640 1 -- 172.21.15.29:0/481553145 wait complete. 2024-09-21T01:56:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 500.0 s, 1047.5 tps, lat 67.063 ms stddev 101.929 2024-09-21T01:56:58.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:57 smithi029 ceph-mon[31228]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 800 op/s 2024-09-21T01:56:58.152 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:56:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:57 smithi190 ceph-mon[34368]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 800 op/s 2024-09-21T01:56:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:57 smithi066 ceph-mon[35001]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 800 op/s 2024-09-21T01:56:58.412 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- 172.21.15.29:0/2593008151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554103b60 msgr2=0x7f7554103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 --2- 172.21.15.29:0/2593008151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554103b60 0x7f7554103f70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7550009900 tx=0x7f755002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- 172.21.15.29:0/2593008151 shutdown_connections 2024-09-21T01:56:58.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 --2- 172.21.15.29:0/2593008151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7554109df0 0x7f7554110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 --2- 172.21.15.29:0/2593008151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75541094a0 0x7f75541098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 --2- 172.21.15.29:0/2593008151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554103b60 0x7f7554103f70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- 172.21.15.29:0/2593008151 >> 172.21.15.29:0/2593008151 conn(0x7f755407b850 msgr2=0x7f755407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- 172.21.15.29:0/2593008151 shutdown_connections 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- 172.21.15.29:0/2593008151 wait complete. 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 Processor -- start 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.901+0000 7f755ba36640 1 -- start start 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 0x7f7554075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 0x7f7554075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7554113b00 con 0x7f7554109df0 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7554113980 con 0x7f75541094a0 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f755ba36640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7554113c80 con 0x7f7554103b60 2024-09-21T01:56:58.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60104/0 (socket says 172.21.15.29:60104) 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 -- 172.21.15.29:0/1888694101 learned_addr learned my addr 172.21.15.29:0/1888694101 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7558fd1640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 0x7f7554075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f75597d2640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 0x7f7554075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 -- 172.21.15.29:0/1888694101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 msgr2=0x7f7554075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 0x7f7554075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 -- 172.21.15.29:0/1888694101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 msgr2=0x7f7554075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 0x7f7554075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 -- 172.21.15.29:0/1888694101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541a8830 con 0x7f7554109df0 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7558fd1640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 0x7f7554075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f75597d2640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 0x7f7554075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:58.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.902+0000 7f7559fd3640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f754c00eb10 tx=0x7f754c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.903+0000 7f754a7fc640 1 -- 172.21.15.29:0/1888694101 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c004b20 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.903+0000 7f754a7fc640 1 -- 172.21.15.29:0/1888694101 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754c004cc0 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.903+0000 7f754a7fc640 1 -- 172.21.15.29:0/1888694101 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c010670 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.903+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75541a8b20 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.903+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a9010 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 Processor -- start 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f754a7fc640 1 -- 172.21.15.29:0/1888694101 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f754c005020 con 0x7f7554109df0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 -- start start 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096e5070 0x565209a8c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 0x565209a8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565209a02ea0 con 0x5652096e5070 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652099f22d0 con 0x5652096bc1b0 2024-09-21T01:56:58.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7561faa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5652099f60e0 con 0x5652096c4390 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f7558fd1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096e5070 0x565209a8c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f75597d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f75597d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44016/0 (socket says 172.21.15.29:44016) 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.906+0000 7f75597d2640 1 -- 172.21.15.29:0/3148114391 learned_addr learned my addr 172.21.15.29:0/3148114391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f7558fd1640 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 0x565209a8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 -- 172.21.15.29:0/3148114391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 msgr2=0x565209a8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 0x565209a8d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 -- 172.21.15.29:0/3148114391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096e5070 msgr2=0x565209a8c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096e5070 0x565209a8c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 -- 172.21.15.29:0/3148114391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565209750650 con 0x5652096bc1b0 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f7558fd1640 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 0x565209a8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f75597d2640 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7550009230 tx=0x7f75500042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d18406e11fcf401 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f754a7fc640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 0x7f75400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f7558fd1640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 0x7f75400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.907+0000 7f754a7fc640 1 -- 172.21.15.29:0/1888694101 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f754c0e78d0 con 0x7f7554109df0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.908+0000 7f7558fd1640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 0x7f75400b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7544009710 tx=0x7f7544005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.909+0000 7f753f7fe640 1 -- 172.21.15.29:0/3148114391 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550035030 con 0x5652096bc1b0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.909+0000 7f753f7fe640 1 -- 172.21.15.29:0/3148114391 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75500044d0 con 0x5652096bc1b0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.909+0000 7f753f7fe640 1 -- 172.21.15.29:0/3148114391 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500035f0 con 0x5652096bc1b0 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.909+0000 7f7561faa740 1 -- 172.21.15.29:0/3148114391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 msgr2=0x565209a8cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.909+0000 7f7561faa740 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7550009230 tx=0x7f75500042e0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 -- 172.21.15.29:0/3148114391 shutdown_connections 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096c4390 0x565209a8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096bc1b0 0x565209a8cbe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 --2- 172.21.15.29:0/3148114391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096e5070 0x565209a8c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 -- 172.21.15.29:0/3148114391 >> 172.21.15.29:0/3148114391 conn(0x5652096d9460 msgr2=0x565209a8a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 -- 172.21.15.29:0/3148114391 shutdown_connections 2024-09-21T01:56:58.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 -- 172.21.15.29:0/3148114391 wait complete. 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 -- start start 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.910+0000 7f7561faa740 1 Processor -- start 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096bc1b0 0x565209a97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 0x565209a982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565209853700 con 0x5652096c4390 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56520998d160 con 0x5652096e5070 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x565209a9ae20 con 0x5652096bc1b0 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7561faa740 10 client.0 _ll_register_callbacks cb 0x5652099a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7558fd1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 0x565209a982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f75597d2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096bc1b0 0x565209a97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44030/0 (socket says 172.21.15.29:44030) 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7558fd1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 0x565209a982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60128/0 (socket says 172.21.15.29:60128) 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 -- 172.21.15.29:0/2544218119 learned_addr learned my addr 172.21.15.29:0/2544218119 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 -- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096bc1b0 msgr2=0x565209a97d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096bc1b0 0x565209a97d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 -- 172.21.15.29:0/2544218119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 msgr2=0x565209a982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 0x565209a982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 -- 172.21.15.29:0/2544218119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652099bac90 con 0x5652096e5070 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f7559fd3640 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f754c00cf20 tx=0x7f754c0ad3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9422967084ff7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.911+0000 7f753cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:58.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0b2070 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f754c0b2210 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75500044d0 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f754c0a9050 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 10 client.26993 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x565209aa6b00 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75500044d0 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f754c0a93c0 con 0x5652096e5070 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 10 client.26993 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 10 client.26993 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 20 client.26993 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5652096a39f0 0x565209aaa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 4 client.26993 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565209a04df0 con 0x5652096a39f0 2024-09-21T01:56:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.912+0000 7f75597d2640 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5652096a39f0 0x565209aaa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.913+0000 7f75597d2640 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5652096a39f0 0x565209aaa000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8550b8aad80f5fc server_cookie=94b9db79190ac013 in_seq=0 out_seq=0 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.913+0000 7f753cff9640 10 client.26993 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f753cff9640 1 -- 172.21.15.29:0/2544218119 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565209a04df0 con 0x5652096a39f0 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f753cff9640 10 client.26993 handle_command_reply: tid=0 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 client.26993 shutdown 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 msgr2=0x565209aa4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f754c00cf20 tx=0x7f754c0ad3c0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 shutdown_connections 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096e5070 0x565209aa4140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096c4390 0x565209a982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5652096a39f0 0x565209aaa000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 --2- 172.21.15.29:0/2544218119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096bc1b0 0x565209a97d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f753cff9640 0 client.26993 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:58.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 >> 172.21.15.29:0/2544218119 conn(0x5652096d9460 msgr2=0x565209a8f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 shutdown_connections 2024-09-21T01:56:58.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 1 -- 172.21.15.29:0/2544218119 wait complete. 2024-09-21T01:56:58.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.914+0000 7f7561faa740 20 client.26993 trim_cache size 0 max 16384 2024-09-21T01:56:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.927+0000 7f7561faa740 1 Processor -- start 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.927+0000 7f7561faa740 1 -- start start 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209aa4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a8f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b1580 con 0x5652096bc1b0 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b3180 con 0x5652096c4390 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7561faa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b1c50 con 0x5652096e5070 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60140/0 (socket says 172.21.15.29:60140) 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 -- 172.21.15.29:0/2592778465 learned_addr learned my addr 172.21.15.29:0/2592778465 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7559fd3640 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a8f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f7558fd1640 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209aa4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 -- 172.21.15.29:0/2592778465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 msgr2=0x565209a8f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a8f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 -- 172.21.15.29:0/2592778465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 msgr2=0x565209aa4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209aa4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 -- 172.21.15.29:0/2592778465 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565209a9a4d0 con 0x5652096bc1b0 2024-09-21T01:56:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f75597d2640 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7550031440 tx=0x7f7550004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575266ece87e67ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.928+0000 7f753effd640 1 -- 172.21.15.29:0/2592778465 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550035040 con 0x5652096bc1b0 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f753effd640 1 -- 172.21.15.29:0/2592778465 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550031950 con 0x5652096bc1b0 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f753effd640 1 -- 172.21.15.29:0/2592778465 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550058990 con 0x5652096bc1b0 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 -- 172.21.15.29:0/2592778465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 msgr2=0x565209a8a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7550031440 tx=0x7f7550004600 comp rx=0 tx=0).stop 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 -- 172.21.15.29:0/2592778465 shutdown_connections 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a8f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209aa4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 --2- 172.21.15.29:0/2592778465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a8a5c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 -- 172.21.15.29:0/2592778465 >> 172.21.15.29:0/2592778465 conn(0x5652096d9460 msgr2=0x565209a89eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 -- 172.21.15.29:0/2592778465 shutdown_connections 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.929+0000 7f7561faa740 1 -- 172.21.15.29:0/2592778465 wait complete. 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 -- start start 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 Processor -- start 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a9cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b2ab0 con 0x5652096bc1b0 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b23e0 con 0x5652096c4390 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f75541af560 con 0x5652096e5070 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7561faa740 10 client.0 _ll_register_callbacks cb 0x7f754c0a9910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7558fd1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.930+0000 7f7558fd1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44046/0 (socket says 172.21.15.29:44046) 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f7558fd1640 1 -- 172.21.15.29:0/4065265464 learned_addr learned my addr 172.21.15.29:0/4065265464 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 -- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 msgr2=0x565209a9cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a9cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 -- 172.21.15.29:0/4065265464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 msgr2=0x565209a9c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 -- 172.21.15.29:0/4065265464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565209b13a10 con 0x5652096bc1b0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f7558fd1640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f75597d2640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a9c0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f755002aa00 tx=0x7f7550004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0a45573c994ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f753cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550035040 con 0x5652096bc1b0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550031950 con 0x5652096bc1b0 2024-09-21T01:56:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.931+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7550031b00 con 0x5652096bc1b0 2024-09-21T01:56:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75500049c0 con 0x5652096bc1b0 2024-09-21T01:56:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 10 client.17217 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x565209b14c90 con 0x5652096bc1b0 2024-09-21T01:56:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550031950 con 0x5652096bc1b0 2024-09-21T01:56:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f7550051440 con 0x5652096bc1b0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 10 client.17217 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 10 client.17217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 20 client.17217 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x565209678da0 0x565209b16ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 4 client.17217 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565209a008a0 con 0x565209678da0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.932+0000 7f7558fd1640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x565209678da0 0x565209b16ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f753cff9640 10 client.17217 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7558fd1640 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x565209678da0 0x565209b16ec0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=153c8b9e9ef2cec0 server_cookie=65875b219cc639d in_seq=0 out_seq=0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f753cff9640 1 -- 172.21.15.29:0/4065265464 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565209a008a0 con 0x565209678da0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f753cff9640 10 client.17217 handle_command_reply: tid=0 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 client.17217 shutdown 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 msgr2=0x565209a9c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a9c0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f755002aa00 tx=0x7f7550004450 comp rx=0 tx=0).stop 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 shutdown_connections 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x565209678da0 0x565209b16ec0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652096e5070 0x565209a9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f753cff9640 0 client.17217 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652096c4390 0x565209a9c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 --2- 172.21.15.29:0/4065265464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652096bc1b0 0x565209a9c0d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 >> 172.21.15.29:0/4065265464 conn(0x5652096d9460 msgr2=0x565209a89dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.933+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 shutdown_connections 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f7561faa740 1 -- 172.21.15.29:0/4065265464 wait complete. 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f7561faa740 20 client.17217 trim_cache size 0 max 16384 2024-09-21T01:56:58.938 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:56:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 msgr2=0x7f75400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 0x7f75400b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7544009710 tx=0x7f7544005850 comp rx=0 tx=0).stop 2024-09-21T01:56:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 msgr2=0x7f7554076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f754c00eb10 tx=0x7f754c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 shutdown_connections 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f75400b6510 0x7f75400b8a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7554109df0 0x7f7554076190 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75541094a0 0x7f7554075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 --2- 172.21.15.29:0/1888694101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554103b60 0x7f7554075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 >> 172.21.15.29:0/1888694101 conn(0x7f755407b850 msgr2=0x7f75541075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 shutdown_connections 2024-09-21T01:56:58.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:56:58.934+0000 7f755ba36640 1 -- 172.21.15.29:0/1888694101 wait complete. 2024-09-21T01:56:59.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.974+0000 7fe45189b640 1 -- 172.21.15.29:0/3449067383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 msgr2=0x7fe44c112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.974+0000 7fe45189b640 1 --2- 172.21.15.29:0/3449067383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c112780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe444009be0 tx=0x7fe44402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 -- 172.21.15.29:0/3449067383 shutdown_connections 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 --2- 172.21.15.29:0/3449067383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c112780 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 --2- 172.21.15.29:0/3449067383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10d690 0x7fe44c10fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 --2- 172.21.15.29:0/3449067383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10ac50 0x7fe44c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 -- 172.21.15.29:0/3449067383 >> 172.21.15.29:0/3449067383 conn(0x7fe44c070c70 msgr2=0x7fe44c073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 -- 172.21.15.29:0/3449067383 shutdown_connections 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 -- 172.21.15.29:0/3449067383 wait complete. 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 Processor -- start 2024-09-21T01:56:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.975+0000 7fe45189b640 1 -- start start 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10ac50 0x7fe44c1a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 0x7fe44c1a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c1a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c115900 con 0x7fe44c10d690 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c115780 con 0x7fe44c10ac50 2024-09-21T01:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe45189b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c115a80 con 0x7fe44c110270 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe443fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 0x7fe44c1a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c1a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe443fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 0x7fe44c1a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60168/0 (socket says 172.21.15.29:60168) 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe443fff640 1 -- 172.21.15.29:0/4160275202 learned_addr learned my addr 172.21.15.29:0/4160275202 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44affd640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10ac50 0x7fe44c1a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 -- 172.21.15.29:0/4160275202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10ac50 msgr2=0x7fe44c1a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10ac50 0x7fe44c1a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 -- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 msgr2=0x7fe44c1a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 0x7fe44c1a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 -- 172.21.15.29:0/4160275202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a4170 con 0x7fe44c110270 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.976+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c1a3950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe444009b30 tx=0x7fe444030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.977+0000 7fe450899640 1 -- 172.21.15.29:0/4160275202 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444041020 con 0x7fe44c110270 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.977+0000 7fe450899640 1 -- 172.21.15.29:0/4160275202 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4440318f0 con 0x7fe44c110270 2024-09-21T01:56:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.977+0000 7fe450899640 1 -- 172.21.15.29:0/4160275202 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444036040 con 0x7fe44c110270 2024-09-21T01:56:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.977+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a4400 con 0x7fe44c110270 2024-09-21T01:56:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.977+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1aa970 con 0x7fe44c110270 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.979+0000 7fe450899640 1 -- 172.21.15.29:0/4160275202 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe444044020 con 0x7fe44c110270 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 Processor -- start 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 -- start start 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf2915b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf2915c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf290cd6f0 con 0x55cf28d8a2f0 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf290bcb20 con 0x55cf28db31b0 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe45880b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cf290c0930 con 0x55cf28d924d0 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe44affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf2915b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44070/0 (socket says 172.21.15.29:44070) 2024-09-21T01:56:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 -- 172.21.15.29:0/2260323251 learned_addr learned my addr 172.21.15.29:0/2260323251 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf2915c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 -- 172.21.15.29:0/2260323251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 msgr2=0x55cf2915c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf2915c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 -- 172.21.15.29:0/2260323251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 msgr2=0x55cf2915b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf2915b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.980+0000 7fe443fff640 1 -- 172.21.15.29:0/2260323251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf29082530 con 0x55cf28db31b0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe450899640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 0x7fe4380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe44affd640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 0x7fe4380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe443fff640 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea90 tx=0x7fe43400a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34411693e95efbae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe450899640 1 -- 172.21.15.29:0/4160275202 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe444107ea0 con 0x7fe44c110270 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe441ffb640 1 -- 172.21.15.29:0/2260323251 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434017040 con 0x55cf28db31b0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe441ffb640 1 -- 172.21.15.29:0/2260323251 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43400cd40 con 0x55cf28db31b0 2024-09-21T01:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.981+0000 7fe441ffb640 1 -- 172.21.15.29:0/2260323251 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43401dc00 con 0x55cf28db31b0 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- 172.21.15.29:0/2260323251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 msgr2=0x55cf2915ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea90 tx=0x7fe43400a720 comp rx=0 tx=0).stop 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe44affd640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 0x7fe4380b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009da0 tx=0x7fe43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- 172.21.15.29:0/2260323251 shutdown_connections 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf2915c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf2915b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 --2- 172.21.15.29:0/2260323251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf2915ae50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- 172.21.15.29:0/2260323251 >> 172.21.15.29:0/2260323251 conn(0x55cf28da7300 msgr2=0x55cf291568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- 172.21.15.29:0/2260323251 shutdown_connections 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- 172.21.15.29:0/2260323251 wait complete. 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.982+0000 7fe45880b740 1 -- start start 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 Processor -- start 2024-09-21T01:56:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf29164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf29165820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf28ff4ab0 con 0x55cf28d8a2f0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf29062730 con 0x55cf28db31b0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cf2915cb30 con 0x55cf28d924d0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe45880b740 10 client.0 _ll_register_callbacks cb 0x55cf290b66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe44affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe44affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60180/0 (socket says 172.21.15.29:60180) 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe44affd640 1 -- 172.21.15.29:0/102097191 learned_addr learned my addr 172.21.15.29:0/102097191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf29165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.983+0000 7fe443fff640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf29164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 -- 172.21.15.29:0/102097191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 msgr2=0x55cf29164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf29164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 -- 172.21.15.29:0/102097191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 msgr2=0x55cf291645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 -- 172.21.15.29:0/102097191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf28e1e790 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44affd640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf29165820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe44402abe0 tx=0x7fe444003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe95b98a3552e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe42affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440b3e60 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe444031440 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43400cd40 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe4440cd7b0 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe45880b740 10 client.27011 fetch_fsmap learned FSMap version 43 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cf28e97620 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.984+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43400cd40 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fe4440b90c0 con 0x55cf28db31b0 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 10 client.27011 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:56:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 10 client.27011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 20 client.27011 populate_metadata read hostname 'smithi029' 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d71d80 0x55cf29171d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 4 client.27011 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf290cf640 con 0x55cf28d71d80 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe44affd640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d71d80 0x55cf29171d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe42affd640 10 client.27011 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.985+0000 7fe44affd640 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d71d80 0x55cf29171d10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b9888a80820c3a server_cookie=11fa85ece4f3519a in_seq=0 out_seq=0 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe42affd640 1 -- 172.21.15.29:0/102097191 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf290cf640 con 0x55cf28d71d80 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe42affd640 10 client.27011 handle_command_reply: tid=0 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 client.27011 shutdown 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 msgr2=0x55cf29165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf29165820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe44402abe0 tx=0x7fe444003f50 comp rx=0 tx=0).stop 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 shutdown_connections 2024-09-21T01:56:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d71d80 0x55cf29171d10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28db31b0 0x55cf29165820 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28d924d0 0x55cf29164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 --2- 172.21.15.29:0/102097191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf291645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe42affd640 0 client.27011 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 >> 172.21.15.29:0/102097191 conn(0x55cf28da7300 msgr2=0x55cf29154ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 shutdown_connections 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.986+0000 7fe45880b740 1 -- 172.21.15.29:0/102097191 wait complete. 2024-09-21T01:56:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:56:59.987+0000 7fe45880b740 20 client.27011 trim_cache size 0 max 16384 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.003+0000 7fe45880b740 1 Processor -- start 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.003+0000 7fe45880b740 1 -- start start 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 0x55cf291515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf29162b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b36e0 con 0x55cf28d8a2f0 2024-09-21T01:57:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b52e0 con 0x55cf28d924d0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe45880b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b3db0 con 0x55cf28db31b0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60182/0 (socket says 172.21.15.29:60182) 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 -- 172.21.15.29:0/2295531632 learned_addr learned my addr 172.21.15.29:0/2295531632 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe443fff640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 0x55cf291515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf29162b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 -- 172.21.15.29:0/2295531632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 msgr2=0x55cf29162b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf29162b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 -- 172.21.15.29:0/2295531632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 msgr2=0x55cf291515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 0x55cf291515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 -- 172.21.15.29:0/2295531632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf291627c0 con 0x55cf28d8a2f0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe443fff640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 0x55cf291515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe44affd640 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009e80 tx=0x7fe43c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daeff6a014b6f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe4417fa640 1 -- 172.21.15.29:0/2295531632 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c005020 con 0x55cf28d8a2f0 2024-09-21T01:57:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe4417fa640 1 -- 172.21.15.29:0/2295531632 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43c0047a0 con 0x55cf28d8a2f0 2024-09-21T01:57:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.004+0000 7fe4417fa640 1 -- 172.21.15.29:0/2295531632 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c02b680 con 0x55cf28d8a2f0 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 -- 172.21.15.29:0/2295531632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 msgr2=0x55cf29156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009e80 tx=0x7fe43c01b000 comp rx=0 tx=0).stop 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 -- 172.21.15.29:0/2295531632 shutdown_connections 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf29162b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d924d0 0x55cf291515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 --2- 172.21.15.29:0/2295531632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d8a2f0 0x55cf29156290 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.005+0000 7fe45880b740 1 -- 172.21.15.29:0/2295531632 >> 172.21.15.29:0/2295531632 conn(0x55cf28da7300 msgr2=0x55cf2915bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 -- 172.21.15.29:0/2295531632 shutdown_connections 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 -- 172.21.15.29:0/2295531632 wait complete. 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 -- start start 2024-09-21T01:57:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 Processor -- start 2024-09-21T01:57:00.010 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 0x55cf2916c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.006+0000 7fe45880b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf2916b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b4c10 con 0x55cf28d924d0 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b4540 con 0x55cf28d8a2f0 2024-09-21T01:57:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1b16c0 con 0x55cf28db31b0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 10 client.0 _ll_register_callbacks cb 0x7fe4440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60188/0 (socket says 172.21.15.29:60188) 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe44affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 0x55cf2916c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 -- 172.21.15.29:0/4021417638 learned_addr learned my addr 172.21.15.29:0/4021417638 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe44b7fe640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf2916b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 -- 172.21.15.29:0/4021417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 msgr2=0x55cf2916b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf2916b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 -- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 msgr2=0x55cf2916c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 0x55cf2916c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 -- 172.21.15.29:0/4021417638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2916b890 con 0x55cf28d924d0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe44affd640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 0x55cf2916c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe443fff640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea90 tx=0x7fe43400a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e61eecde5bf991 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe42affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43400e750 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe434005250 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.007+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43c0047a0 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe434036050 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 10 client.17235 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cf2916a670 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43c0047a0 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fe43400b040 con 0x55cf28d924d0 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 10 client.17235 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 20 client.17235 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d471c0 0x55cf291deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 4 client.17235 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf290cb0f0 con 0x55cf28d471c0 2024-09-21T01:57:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.008+0000 7fe44affd640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d471c0 0x55cf291deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.009+0000 7fe42affd640 10 client.17235 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:00.009+0000 7fe44affd640 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d471c0 0x55cf291deb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bee1e0a4c6e510 server_cookie=1126c383cb416367 in_seq=0 out_seq=0 2024-09-21T01:57:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:56:59 smithi029 ceph-mon[31228]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-21T01:57:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:56:59 smithi190 ceph-mon[34368]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-21T01:57:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:56:59 smithi066 ceph-mon[35001]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 -- 172.21.15.29:0/3077065697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a01083a0 msgr2=0x7f92a01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 --2- 172.21.15.29:0/3077065697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a01083a0 0x7f92a01087b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9290009900 tx=0x7f929002a410 comp rx=0 tx=0).stop 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 -- 172.21.15.29:0/3077065697 shutdown_connections 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 --2- 172.21.15.29:0/3077065697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 --2- 172.21.15.29:0/3077065697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a01083a0 0x7f92a01087b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 --2- 172.21.15.29:0/3077065697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a0102a60 0x7f92a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 -- 172.21.15.29:0/3077065697 >> 172.21.15.29:0/3077065697 conn(0x7f92a0079730 msgr2=0x7f92a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 -- 172.21.15.29:0/3077065697 shutdown_connections 2024-09-21T01:57:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.504+0000 7f92a72f8640 1 -- 172.21.15.29:0/3077065697 wait complete. 2024-09-21T01:57:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.505+0000 7f92a72f8640 1 Processor -- start 2024-09-21T01:57:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.505+0000 7f92a72f8640 1 -- start start 2024-09-21T01:57:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.505+0000 7f92a72f8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a72f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a01083a0 0x7f92a0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a72f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a72f8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0112960 con 0x7f92a0102a60 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a72f8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01127e0 con 0x7f92a01083a0 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a72f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0112ae0 con 0x7f92a0108cf0 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5094640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5094640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60214/0 (socket says 172.21.15.29:60214) 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a4893640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a01083a0 0x7f92a0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5094640 1 -- 172.21.15.29:0/135054514 learned_addr learned my addr 172.21.15.29:0/135054514 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 -- 172.21.15.29:0/135054514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a01083a0 msgr2=0x7f92a0199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a01083a0 0x7f92a0199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 -- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 msgr2=0x7f92a0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 -- 172.21.15.29:0/135054514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a019f300 con 0x7f92a0108cf0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5094640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92a5895640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a019ebd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f929800ec50 tx=0x7f929800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.506+0000 7f92967fc640 1 -- 172.21.15.29:0/135054514 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298003b70 con 0x7f92a0108cf0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.507+0000 7f92967fc640 1 -- 172.21.15.29:0/135054514 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9298003d10 con 0x7f92a0108cf0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.507+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a019f590 con 0x7f92a0108cf0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.507+0000 7f92967fc640 1 -- 172.21.15.29:0/135054514 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298010830 con 0x7f92a0108cf0 2024-09-21T01:57:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.507+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a006ae40 con 0x7f92a0108cf0 2024-09-21T01:57:00.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.509+0000 7f92ad86c740 1 Processor -- start 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.509+0000 7f92967fc640 1 -- 172.21.15.29:0/135054514 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9298003ec0 con 0x7f92a0108cf0 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 -- start start 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fe4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fe5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9f5aea0 con 0x55e5f9c1c390 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9f4a2d0 con 0x55e5f9c141b0 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92ad86c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9f4e0e0 con 0x55e5f9c3d070 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40744/0 (socket says 172.21.15.29:40744) 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 -- 172.21.15.29:0/3709257253 learned_addr learned my addr 172.21.15.29:0/3709257253 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fe5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a5094640 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fe4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 -- 172.21.15.29:0/3709257253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 msgr2=0x55e5f9fe4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fe4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 -- 172.21.15.29:0/3709257253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 msgr2=0x55e5f9fe5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fe5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.510+0000 7f92a4893640 1 -- 172.21.15.29:0/3709257253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f9ca8650 con 0x55e5f9c3d070 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f92967fc640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 0x7f928c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f92a4893640 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92a006aaa0 tx=0x7f9290005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d889ecd52e89bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f92a5094640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 0x7f928c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f928b7fe640 1 -- 172.21.15.29:0/3709257253 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035030 con 0x55e5f9c3d070 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f92967fc640 1 -- 172.21.15.29:0/135054514 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f92980e7830 con 0x7f92a0108cf0 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f928b7fe640 1 -- 172.21.15.29:0/3709257253 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9290004030 con 0x55e5f9c3d070 2024-09-21T01:57:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.511+0000 7f928b7fe640 1 -- 172.21.15.29:0/3709257253 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290003930 con 0x55e5f9c3d070 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92a5094640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 0x7f928c0b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e5f9fe3320 tx=0x7f929c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- 172.21.15.29:0/3709257253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 msgr2=0x55e5f9fe46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92a006aaa0 tx=0x7f9290005780 comp rx=0 tx=0).stop 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- 172.21.15.29:0/3709257253 shutdown_connections 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fe5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fe4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 --2- 172.21.15.29:0/3709257253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9fe46a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- 172.21.15.29:0/3709257253 >> 172.21.15.29:0/3709257253 conn(0x55e5f9c31460 msgr2=0x55e5f9fe2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- 172.21.15.29:0/3709257253 shutdown_connections 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- 172.21.15.29:0/3709257253 wait complete. 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 -- start start 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.512+0000 7f92ad86c740 1 Processor -- start 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fefd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9dab700 con 0x55e5f9c1c390 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9ee5160 con 0x55e5f9c141b0 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9ff2e20 con 0x55e5f9c3d070 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92ad86c740 10 client.0 _ll_register_callbacks cb 0x55e5f9efe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a5895640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60236/0 (socket says 172.21.15.29:60236) 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a5895640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40756/0 (socket says 172.21.15.29:40756) 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a4893640 1 -- 172.21.15.29:0/1053368472 learned_addr learned my addr 172.21.15.29:0/1053368472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.513+0000 7f92a5094640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fefd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 -- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 msgr2=0x55e5f9ffc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 -- 172.21.15.29:0/1053368472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 msgr2=0x55e5f9fefd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fefd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 -- 172.21.15.29:0/1053368472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f9f12c90 con 0x55e5f9c1c390 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a5895640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a5094640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92a4893640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f929002a930 tx=0x7f9290005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe0a4cfab477d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f9288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035070 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9290004030 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f92900043e0 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9290031b10 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92ad86c740 10 client.17247 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e5f9ffeb10 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.514+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9290004030 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f929003d450 con 0x55e5f9c1c390 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 10 client.17247 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 10 client.17247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 20 client.17247 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bfb9f0 0x55e5fa001e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 4 client.17247 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f9f5cdf0 con 0x55e5f9bfb9f0 2024-09-21T01:57:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92a5094640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bfb9f0 0x55e5fa001e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f9288ff9640 10 client.17247 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.515+0000 7f92a5094640 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bfb9f0 0x55e5fa001e70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a007bea2b8ccf3d0 server_cookie=9444062aaad3eb69 in_seq=0 out_seq=0 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f9288ff9640 1 -- 172.21.15.29:0/1053368472 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5f9f5cdf0 con 0x55e5f9bfb9f0 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f9288ff9640 10 client.17247 handle_command_reply: tid=0 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 client.17247 shutdown 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 msgr2=0x55e5f9ff0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f929002a930 tx=0x7f9290005a20 comp rx=0 tx=0).stop 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 shutdown_connections 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bfb9f0 0x55e5fa001e70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c3d070 0x55e5f9ffc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9ff0280 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f9288ff9640 0 client.17247 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:00.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 --2- 172.21.15.29:0/1053368472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c141b0 0x55e5f9fefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.516+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 >> 172.21.15.29:0/1053368472 conn(0x55e5f9c31460 msgr2=0x55e5f9fe7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.517+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 shutdown_connections 2024-09-21T01:57:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.517+0000 7f92ad86c740 1 -- 172.21.15.29:0/1053368472 wait complete. 2024-09-21T01:57:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.517+0000 7f92ad86c740 20 client.17247 trim_cache size 0 max 16384 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 Processor -- start 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 -- start start 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5f9ffc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5f9ff0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01af320 con 0x55e5f9c1c390 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b0f20 con 0x55e5f9c3d070 2024-09-21T01:57:00.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92ad86c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01af9f0 con 0x55e5f9c141b0 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60244/0 (socket says 172.21.15.29:60244) 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92a4893640 1 -- 172.21.15.29:0/2880999656 learned_addr learned my addr 172.21.15.29:0/2880999656 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92a5895640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5f9ff0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.530+0000 7f92a5094640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5f9ffc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 -- 172.21.15.29:0/2880999656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 msgr2=0x55e5f9ffc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5f9ffc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 -- 172.21.15.29:0/2880999656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 msgr2=0x55e5f9ff0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5f9ff0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 -- 172.21.15.29:0/2880999656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f9ff1000 con 0x55e5f9c1c390 2024-09-21T01:57:00.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a5895640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5f9ff0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:00.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f92a4893640 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9290006010 tx=0x7f92900097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc5de835c11034c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f928affd640 1 -- 172.21.15.29:0/2880999656 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035070 con 0x55e5f9c1c390 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f928affd640 1 -- 172.21.15.29:0/2880999656 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929004e070 con 0x55e5f9c1c390 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.531+0000 7f928affd640 1 -- 172.21.15.29:0/2880999656 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290004920 con 0x55e5f9c1c390 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- 172.21.15.29:0/2880999656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 msgr2=0x55e5f9fefaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9290006010 tx=0x7f92900097c0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- 172.21.15.29:0/2880999656 shutdown_connections 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5f9ff0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5f9fefaf0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 --2- 172.21.15.29:0/2880999656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5f9ffc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- 172.21.15.29:0/2880999656 >> 172.21.15.29:0/2880999656 conn(0x55e5f9c31460 msgr2=0x55e5f9fee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- 172.21.15.29:0/2880999656 shutdown_connections 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- 172.21.15.29:0/2880999656 wait complete. 2024-09-21T01:57:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 -- start start 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.532+0000 7f92ad86c740 1 Processor -- start 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5fa002c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5fa0032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b0850 con 0x55e5f9c1c390 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b0180 con 0x55e5f9c3d070 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ad300 con 0x55e5f9c141b0 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92ad86c740 10 client.0 _ll_register_callbacks cb 0x7f9290032c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5fa002c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40778/0 (socket says 172.21.15.29:40778) 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a4893640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5fa002c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60254/0 (socket says 172.21.15.29:60254) 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 -- 172.21.15.29:0/685740944 learned_addr learned my addr 172.21.15.29:0/685740944 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:00.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5895640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5fa0032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 -- 172.21.15.29:0/685740944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 msgr2=0x55e5fa0032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5fa0032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 -- 172.21.15.29:0/685740944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 msgr2=0x55e5fa002c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5fa002c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.533+0000 7f92a5094640 1 -- 172.21.15.29:0/685740944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fa003a40 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f92a5094640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bfd0 tx=0x7f929c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bfa45ef28243145 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f9288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c01ea50 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929004e070 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f929c01ebf0 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f929c01edf0 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f92ad86c740 10 client.27007 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e5fa06c6d0 con 0x55e5f9c141b0 2024-09-21T01:57:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.534+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929004e070 con 0x55e5f9c141b0 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f929c030510 con 0x55e5f9c141b0 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 10 client.27007 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 10 client.27007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 20 client.27007 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bd0da0 0x55e5fa06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 4 client.27007 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5f9f588a0 con 0x55e5f9bd0da0 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92a4893640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bd0da0 0x55e5fa06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f9288ff9640 10 client.27007 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f92a4893640 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bd0da0 0x55e5fa06ef80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67bcd5841dd9f6cf server_cookie=bcb9eb13310bbcea in_seq=0 out_seq=0 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f9288ff9640 1 -- 172.21.15.29:0/685740944 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5f9f588a0 con 0x55e5f9bd0da0 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.535+0000 7f9288ff9640 10 client.27007 handle_command_reply: tid=0 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 client.27007 shutdown 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 msgr2=0x55e5fa005790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bfd0 tx=0x7f929c004730 comp rx=0 tx=0).stop 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 shutdown_connections 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55e5f9bd0da0 0x55e5fa06ef80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9c3d070 0x55e5fa0032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5f9c1c390 0x55e5fa002c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f9288ff9640 0 client.27007 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 --2- 172.21.15.29:0/685740944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5f9c141b0 0x55e5fa005790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 >> 172.21.15.29:0/685740944 conn(0x55e5f9c31460 msgr2=0x55e5f9fefaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 shutdown_connections 2024-09-21T01:57:00.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 1 -- 172.21.15.29:0/685740944 wait complete. 2024-09-21T01:57:00.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92ad86c740 20 client.27007 trim_cache size 0 max 16384 2024-09-21T01:57:00.542 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:00.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 msgr2=0x7f928c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 0x7f928c0b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e5f9fe3320 tx=0x7f929c00b410 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 msgr2=0x7f92a019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.536+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a019ebd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f929800ec50 tx=0x7f929800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 shutdown_connections 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f928c0b6510 0x7f928c0b8a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0108cf0 0x7f92a019ebd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a01083a0 0x7f92a0199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 --2- 172.21.15.29:0/135054514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92a0102a60 0x7f92a0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 >> 172.21.15.29:0/135054514 conn(0x7f92a0079730 msgr2=0x7f92a0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 shutdown_connections 2024-09-21T01:57:00.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:00.537+0000 7f92a72f8640 1 -- 172.21.15.29:0/135054514 wait complete. 2024-09-21T01:57:01.944 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:57:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:01 smithi190 ceph-mon[34368]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.31k op/s 2024-09-21T01:57:02.256 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:01 smithi066 ceph-mon[35001]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.31k op/s 2024-09-21T01:57:02.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:01 smithi029 ceph-mon[31228]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.31k op/s 2024-09-21T01:57:02.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2365108475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 msgr2=0x7fb2d01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2365108475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d01087b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009900 tx=0x7fb2c402a410 comp rx=0 tx=0).stop 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2365108475 shutdown_connections 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2365108475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 0x7fb2d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2365108475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d01087b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2365108475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 0x7fb2d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2365108475 >> 172.21.15.29:0/2365108475 conn(0x7fb2d0079730 msgr2=0x7fb2d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2365108475 shutdown_connections 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.863+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2365108475 wait complete. 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.864+0000 7fb2d4d0e640 1 Processor -- start 2024-09-21T01:57:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.864+0000 7fb2d4d0e640 1 -- start start 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 0x7fb2d019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 0x7fb2d019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0112960 con 0x7fb2d01083a0 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01127e0 con 0x7fb2d0108cf0 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2d4d0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0112ae0 con 0x7fb2d0102a60 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59256/0 (socket says 172.21.15.29:59256) 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/2315387059 learned_addr learned my addr 172.21.15.29:0/2315387059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2ced9d640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 0x7fb2d019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 0x7fb2d019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/2315387059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 msgr2=0x7fb2d019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 0x7fb2d019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/2315387059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 msgr2=0x7fb2d019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 0x7fb2d019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/2315387059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d01a0510 con 0x7fb2d01083a0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.865+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006010 tx=0x7fb2c4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.866+0000 7fb2bf7fe640 1 -- 172.21.15.29:0/2315387059 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c403f030 con 0x7fb2d01083a0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.866+0000 7fb2bf7fe640 1 -- 172.21.15.29:0/2315387059 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c4003aa0 con 0x7fb2d01083a0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.866+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d01a07a0 con 0x7fb2d01083a0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.866+0000 7fb2bf7fe640 1 -- 172.21.15.29:0/2315387059 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4035070 con 0x7fb2d01083a0 2024-09-21T01:57:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.866+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01a6f60 con 0x7fb2d01083a0 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.868+0000 7fb2db282740 1 Processor -- start 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 -- start start 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2bf7fe640 1 -- 172.21.15.29:0/2315387059 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb2c4042030 con 0x7fb2d01083a0 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 0x561a44ba2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44ba3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561a44b18ea0 con 0x561a447fb070 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561a44b082d0 con 0x561a447d21b0 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2db282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a44b0c0e0 con 0x561a447da390 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59260/0 (socket says 172.21.15.29:59260) 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2ced9d640 1 -- 172.21.15.29:0/2774754128 learned_addr learned my addr 172.21.15.29:0/2774754128 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44ba3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.869+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 0x561a44ba2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 -- 172.21.15.29:0/2774754128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 msgr2=0x561a44ba3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44ba3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 -- 172.21.15.29:0/2774754128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 msgr2=0x561a44ba2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 0x561a44ba2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 -- 172.21.15.29:0/2774754128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a44866650 con 0x561a447fb070 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 0x561a44ba2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2bf7fe640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 0x7fb2b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ced9d640 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009f70 tx=0x7fb2c8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1faa45d94931576 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 0x7fb2b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2bf7fe640 1 -- 172.21.15.29:0/2315387059 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb2c41076a0 con 0x7fb2d01083a0 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2bcff9640 1 -- 172.21.15.29:0/2774754128 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8016070 con 0x561a447fb070 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.870+0000 7fb2bcff9640 1 -- 172.21.15.29:0/2774754128 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c8004d20 con 0x561a447fb070 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2bcff9640 1 -- 172.21.15.29:0/2774754128 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8012770 con 0x561a447fb070 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 -- 172.21.15.29:0/2774754128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 msgr2=0x561a44ba26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009f70 tx=0x7fb2c8010800 comp rx=0 tx=0).stop 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 0x7fb2b80b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x561a44ba1320 tx=0x7fb2c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 -- 172.21.15.29:0/2774754128 shutdown_connections 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44ba3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447d21b0 0x561a44ba2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 --2- 172.21.15.29:0/2774754128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447fb070 0x561a44ba26a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 -- 172.21.15.29:0/2774754128 >> 172.21.15.29:0/2774754128 conn(0x561a447ef460 msgr2=0x561a44ba0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 -- 172.21.15.29:0/2774754128 shutdown_connections 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.871+0000 7fb2db282740 1 -- 172.21.15.29:0/2774754128 wait complete. 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 -- start start 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 Processor -- start 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44bb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447fb070 0x561a44bbe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561a44969700 con 0x561a447d21b0 2024-09-21T01:57:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561a44aa3160 con 0x561a447fb070 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a44bb5230 con 0x561a447da390 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2ce59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2ce59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59266/0 (socket says 172.21.15.29:59266) 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2ce59c640 1 -- 172.21.15.29:0/1358546487 learned_addr learned my addr 172.21.15.29:0/1358546487 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2db282740 10 client.0 _ll_register_callbacks cb 0x561a44abc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.872+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44bb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ced9d640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447fb070 0x561a44bbe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 -- 172.21.15.29:0/1358546487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 msgr2=0x561a44bb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44bb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 -- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447fb070 msgr2=0x561a44bbe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447fb070 0x561a44bbe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 -- 172.21.15.29:0/1358546487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a44ad0c90 con 0x561a447d21b0 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44bb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb2c001ea00 tx=0x7fb2c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15a889118382f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0004020 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb2c0018030 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.873+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c8004d20 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00041c0 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 10 client.17265 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561a44bc0f90 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c8004d20 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fb2c0004910 con 0x561a447d21b0 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 10 client.17265 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:02.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 20 client.17265 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447b99f0 0x561a44bc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 4 client.17265 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a44b1adf0 con 0x561a447b99f0 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.874+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447b99f0 0x561a44bc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.875+0000 7fb2b5ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:02.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.875+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447b99f0 0x561a44bc4570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1fc64391a40e73c server_cookie=96b04a92c5212d9b in_seq=0 out_seq=0 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.875+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/1358546487 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a44b1adf0 con 0x561a447b99f0 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.875+0000 7fb2b5ffb640 10 client.17265 handle_command_reply: tid=0 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.875+0000 7fb2db282740 1 client.17265 shutdown 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 msgr2=0x561a44bb2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb2c001ea00 tx=0x7fb2c0004730 comp rx=0 tx=0).stop 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 shutdown_connections 2024-09-21T01:57:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447b99f0 0x561a44bc4570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2b5ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447fb070 0x561a44bbe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447da390 0x561a44bb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 --2- 172.21.15.29:0/1358546487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447d21b0 0x561a44bb2130 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 >> 172.21.15.29:0/1358546487 conn(0x561a447ef460 msgr2=0x561a44ba4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 shutdown_connections 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 1 -- 172.21.15.29:0/1358546487 wait complete. 2024-09-21T01:57:02.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.876+0000 7fb2db282740 20 client.17265 trim_cache size 0 max 16384 2024-09-21T01:57:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.889+0000 7fb2db282740 1 Processor -- start 2024-09-21T01:57:02.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.889+0000 7fb2db282740 1 -- start start 2024-09-21T01:57:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447ef460 0x561a44bb1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 0x561a44baf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01ad190 con 0x561a447da390 2024-09-21T01:57:02.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01aed90 con 0x561a447ef460 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2db282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01ad860 con 0x561a447d21b0 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 0x561a44baf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38338/0 (socket says 172.21.15.29:38338) 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 0x561a44baf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59276/0 (socket says 172.21.15.29:59276) 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/960623333 learned_addr learned my addr 172.21.15.29:0/960623333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447ef460 0x561a44bb1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/960623333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447ef460 msgr2=0x561a44bb1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447ef460 0x561a44bb1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/960623333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 msgr2=0x561a44baf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 0x561a44baf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.890+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/960623333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a44bb3060 con 0x561a447d21b0 2024-09-21T01:57:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.891+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cd5f0 tx=0x7fb2c403b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f0ff9f1479f2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.891+0000 7fb2b7fff640 1 -- 172.21.15.29:0/960623333 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4049040 con 0x561a447d21b0 2024-09-21T01:57:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.891+0000 7fb2b7fff640 1 -- 172.21.15.29:0/960623333 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c403b460 con 0x561a447d21b0 2024-09-21T01:57:02.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.891+0000 7fb2b7fff640 1 -- 172.21.15.29:0/960623333 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40d7260 con 0x561a447d21b0 2024-09-21T01:57:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- 172.21.15.29:0/960623333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 msgr2=0x561a44bb2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cd5f0 tx=0x7fb2c403b640 comp rx=0 tx=0).stop 2024-09-21T01:57:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- 172.21.15.29:0/960623333 shutdown_connections 2024-09-21T01:57:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447da390 0x561a44baf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb2420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 --2- 172.21.15.29:0/960623333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447ef460 0x561a44bb1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- 172.21.15.29:0/960623333 >> 172.21.15.29:0/960623333 conn(0x561a44ba7bb0 msgr2=0x561a44bb09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- 172.21.15.29:0/960623333 shutdown_connections 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- 172.21.15.29:0/960623333 wait complete. 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 -- start start 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.892+0000 7fb2db282740 1 Processor -- start 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 0x561a44bb6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 0x561a44c2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01ae6c0 con 0x561a447ef460 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01adff0 con 0x561a447da390 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01ab170 con 0x561a447d21b0 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2db282740 10 client.0 _ll_register_callbacks cb 0x7fb2c0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:02.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ce59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 0x561a44c2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 0x561a44bb6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ce59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38354/0 (socket says 172.21.15.29:38354) 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ce59c640 1 -- 172.21.15.29:0/3137596422 learned_addr learned my addr 172.21.15.29:0/3137596422 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 0x561a44c2df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59282/0 (socket says 172.21.15.29:59282) 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/3137596422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 msgr2=0x561a44bb6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/3137596422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 msgr2=0x561a44c2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 0x561a44c2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2cdd9b640 1 -- 172.21.15.29:0/3137596422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a44c2e470 con 0x561a447da390 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.893+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2cdd9b640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 0x561a44bb6db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cd5f0 tx=0x7fb2c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644a7853991b31c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:02.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4049070 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb2c40d6dd0 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a44b168a0 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb2c40cd240 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 10 client.27053 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561a44c2f560 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a44b168a0 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fb2c4109bb0 con 0x561a447da390 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 10 client.27053 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 10 client.27053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.894+0000 7fb2db282740 20 client.27053 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447fb070 0x561a44c31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2db282740 4 client.27053 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a44b13cd0 con 0x561a447fb070 2024-09-21T01:57:02.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447fb070 0x561a44c31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2b5ffb640 10 client.27053 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2ce59c640 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447fb070 0x561a44c31c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7102b9dd0fe7cec server_cookie=a0fe4b0629e87410 in_seq=0 out_seq=0 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.895+0000 7fb2b5ffb640 1 -- 172.21.15.29:0/3137596422 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a44b13cd0 con 0x561a447fb070 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2b5ffb640 10 client.27053 handle_command_reply: tid=0 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 client.27053 shutdown 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 msgr2=0x561a44bb6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 0x561a44bb6db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cd5f0 tx=0x7fb2c4009580 comp rx=0 tx=0).stop 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 shutdown_connections 2024-09-21T01:57:02.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x561a447fb070 0x561a44c31c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a447ef460 0x561a44c2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a447da390 0x561a44bb6db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 --2- 172.21.15.29:0/3137596422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a447d21b0 0x561a44bb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 >> 172.21.15.29:0/3137596422 conn(0x561a44ba7bb0 msgr2=0x561a44bb1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2b5ffb640 0 client.27053 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 shutdown_connections 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 1 -- 172.21.15.29:0/3137596422 wait complete. 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.896+0000 7fb2db282740 20 client.27053 trim_cache size 0 max 16384 2024-09-21T01:57:02.902 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 msgr2=0x7fb2b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 0x7fb2b80b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x561a44ba1320 tx=0x7fb2c0005f90 comp rx=0 tx=0).stop 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 msgr2=0x7fb2d019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006010 tx=0x7fb2c4004060 comp rx=0 tx=0).stop 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 shutdown_connections 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb2b80b6510 0x7fb2b80b8a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0108cf0 0x7fb2d019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d01083a0 0x7fb2d019ab40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 --2- 172.21.15.29:0/2315387059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d0102a60 0x7fb2d019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 >> 172.21.15.29:0/2315387059 conn(0x7fb2d0079730 msgr2=0x7fb2d01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 shutdown_connections 2024-09-21T01:57:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:02.897+0000 7fb2d4d0e640 1 -- 172.21.15.29:0/2315387059 wait complete. 2024-09-21T01:57:02.972 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 505.0 s, 2.4 tps, lat 1912.795 ms stddev 175.919 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.532+0000 7fe42affd640 1 -- 172.21.15.29:0/4021417638 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf290cb0f0 con 0x55cf28d471c0 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.532+0000 7fe42affd640 10 client.17235 handle_command_reply: tid=0 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.532+0000 7fe45880b740 1 client.17235 shutdown 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.532+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 msgr2=0x55cf2916ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea90 tx=0x7fe43400a400 comp rx=0 tx=0).stop 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 shutdown_connections 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cf28d471c0 0x55cf291deb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf28db31b0 0x55cf2916b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf28d924d0 0x55cf2916ae10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 --2- 172.21.15.29:0/4021417638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf28d8a2f0 0x55cf2916c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe42affd640 0 client.17235 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 >> 172.21.15.29:0/4021417638 conn(0x55cf28da7300 msgr2=0x55cf291515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 shutdown_connections 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 1 -- 172.21.15.29:0/4021417638 wait complete. 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.533+0000 7fe45880b740 20 client.17235 trim_cache size 0 max 16384 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 msgr2=0x7fe4380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 0x7fe4380b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009da0 tx=0x7fe43c005f90 comp rx=0 tx=0).stop 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 msgr2=0x7fe44c1a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c1a3950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe444009b30 tx=0x7fe444030d50 comp rx=0 tx=0).stop 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 shutdown_connections 2024-09-21T01:57:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.534+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4380b6510 0x7fe4380b8a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44c110270 0x7fe44c1a3950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe44c10d690 0x7fe44c1a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 --2- 172.21.15.29:0/4160275202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44c10ac50 0x7fe44c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 >> 172.21.15.29:0/4160275202 conn(0x7fe44c070c70 msgr2=0x7fe44c10f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 shutdown_connections 2024-09-21T01:57:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:03.535+0000 7fe45189b640 1 -- 172.21.15.29:0/4160275202 wait complete. 2024-09-21T01:57:03.695 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:57:04.128 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:03 smithi190 ceph-mon[34368]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2024-09-21T01:57:04.266 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:03 smithi029 ceph-mon[31228]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2024-09-21T01:57:04.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:03 smithi066 ceph-mon[35001]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 -- 172.21.15.29:0/1979034141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9944105270 msgr2=0x7f994410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 --2- 172.21.15.29:0/1979034141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9944105270 0x7f994410d8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9938009900 tx=0x7f993802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 -- 172.21.15.29:0/1979034141 shutdown_connections 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 --2- 172.21.15.29:0/1979034141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f994410ddf0 0x7f9944110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 --2- 172.21.15.29:0/1979034141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9944105270 0x7f994410d8b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 --2- 172.21.15.29:0/1979034141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 0x7f9944104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 -- 172.21.15.29:0/1979034141 >> 172.21.15.29:0/1979034141 conn(0x7f9944079730 msgr2=0x7f9944079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 -- 172.21.15.29:0/1979034141 shutdown_connections 2024-09-21T01:57:04.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.713+0000 7f994a4fe640 1 -- 172.21.15.29:0/1979034141 wait complete. 2024-09-21T01:57:04.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.714+0000 7f994a4fe640 1 Processor -- start 2024-09-21T01:57:04.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.714+0000 7f994a4fe640 1 -- start start 2024-09-21T01:57:04.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 0x7f99441982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9944105270 0x7f9944198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99441118b0 con 0x7f994410ddf0 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9944111730 con 0x7f9944104840 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f994a4fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9944111a30 con 0x7f9944105270 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59302/0 (socket says 172.21.15.29:59302) 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 -- 172.21.15.29:0/2825153715 learned_addr learned my addr 172.21.15.29:0/2825153715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9943fff640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 0x7f99441982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f99437fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9944105270 0x7f9944198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 -- 172.21.15.29:0/2825153715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9944105270 msgr2=0x7f9944198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9944105270 0x7f9944198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 -- 172.21.15.29:0/2825153715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 msgr2=0x7f99441982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.715+0000 7f9948a9b640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 0x7f99441982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.716+0000 7f9948a9b640 1 -- 172.21.15.29:0/2825153715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994419e150 con 0x7f994410ddf0 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.716+0000 7f9948a9b640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f993c00eb10 tx=0x7f993c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.716+0000 7f99417fa640 1 -- 172.21.15.29:0/2825153715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c004b20 con 0x7f994410ddf0 2024-09-21T01:57:04.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.716+0000 7f99417fa640 1 -- 172.21.15.29:0/2825153715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c004cc0 con 0x7f994410ddf0 2024-09-21T01:57:04.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.716+0000 7f99417fa640 1 -- 172.21.15.29:0/2825153715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c010670 con 0x7f994410ddf0 2024-09-21T01:57:04.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.717+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994419e440 con 0x7f994410ddf0 2024-09-21T01:57:04.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.717+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994419e900 con 0x7f994410ddf0 2024-09-21T01:57:04.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.720+0000 7f9950a72740 1 Processor -- start 2024-09-21T01:57:04.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.720+0000 7f99417fa640 1 -- 172.21.15.29:0/2825153715 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f993c005020 con 0x7f994410ddf0 2024-09-21T01:57:04.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.720+0000 7f9950a72740 1 -- start start 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcfe66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcfe7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555fcf5cea0 con 0x5555fcc1e390 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5555fcf4c2d0 con 0x5555fcc3f070 2024-09-21T01:57:04.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f9950a72740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5555fcf500e0 con 0x5555fcc161b0 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f99437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcfe7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f99437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcfe7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59306/0 (socket says 172.21.15.29:59306) 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.721+0000 7f99437fe640 1 -- 172.21.15.29:0/2962520071 learned_addr learned my addr 172.21.15.29:0/2962520071 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcfe66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99417fa640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 0x7f992c0bce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99417fa640 1 -- 172.21.15.29:0/2825153715 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f993c0e7890 con 0x7f994410ddf0 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 -- 172.21.15.29:0/2962520071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 msgr2=0x5555fcfe6be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 -- 172.21.15.29:0/2962520071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 msgr2=0x5555fcfe7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcfe7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.722+0000 7f99437fe640 1 -- 172.21.15.29:0/2962520071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fccaa650 con 0x5555fcc3f070 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.723+0000 7f99437fe640 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcfe66a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9938005f70 tx=0x7f9938005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff570df45b51fe3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.723+0000 7f9948a9b640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 0x7f992c0bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.723+0000 7f992a7fc640 1 -- 172.21.15.29:0/2962520071 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938041070 con 0x5555fcc3f070 2024-09-21T01:57:04.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.723+0000 7f992a7fc640 1 -- 172.21.15.29:0/2962520071 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99380314a0 con 0x5555fcc3f070 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.724+0000 7f992a7fc640 1 -- 172.21.15.29:0/2962520071 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938035070 con 0x5555fcc3f070 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.724+0000 7f9950a72740 1 -- 172.21.15.29:0/2962520071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 msgr2=0x5555fcfe66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.724+0000 7f9950a72740 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcfe66a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9938005f70 tx=0x7f9938005a20 comp rx=0 tx=0).stop 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.724+0000 7f9948a9b640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 0x7f992c0bce60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f993c0049d0 tx=0x7f993c00efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 -- 172.21.15.29:0/2962520071 shutdown_connections 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcfe7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 --2- 172.21.15.29:0/2962520071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcfe66a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 -- 172.21.15.29:0/2962520071 >> 172.21.15.29:0/2962520071 conn(0x5555fcc33460 msgr2=0x5555fcfe4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 -- 172.21.15.29:0/2962520071 shutdown_connections 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 -- 172.21.15.29:0/2962520071 wait complete. 2024-09-21T01:57:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.725+0000 7f9950a72740 1 -- start start 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 Processor -- start 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 0x5555fcff2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc3f070 0x5555fcffe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555fcdad700 con 0x5555fcc3f070 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5555fcee7160 con 0x5555fcc1e390 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5555fcff4e70 con 0x5555fcc161b0 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f9950a72740 10 client.0 _ll_register_callbacks cb 0x5555fcf009e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f99437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f99437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38384/0 (socket says 172.21.15.29:38384) 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.726+0000 7f99437fe640 1 -- 172.21.15.29:0/3152442554 learned_addr learned my addr 172.21.15.29:0/3152442554 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:04.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc3f070 0x5555fcffe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f9943fff640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 0x5555fcff2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 -- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 msgr2=0x5555fcff2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 0x5555fcff2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 -- 172.21.15.29:0/3152442554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc3f070 msgr2=0x5555fcffe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc3f070 0x5555fcffe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 -- 172.21.15.29:0/3152442554 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fcf14c90 con 0x5555fcc161b0 2024-09-21T01:57:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f9943fff640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 0x5555fcff2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:04.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.727+0000 7f99437fe640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9938003750 tx=0x7f9938031cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1b295b9005d12b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938035070 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9938031920 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99380314a0 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9938041040 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f9950a72740 10 client.27034 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5555fd000c20 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99380314a0 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.728+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9938059070 con 0x5555fcc161b0 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 10 client.27034 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 20 client.27034 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbfd9f0 0x5555fd003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 4 client.27034 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555fcf5edf0 con 0x5555fcbfd9f0 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9943fff640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbfd9f0 0x5555fd003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f9943fff640 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbfd9f0 0x5555fd003f80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423fc3a5c78d88e4 server_cookie=d5e5f081fe8d7779 in_seq=0 out_seq=0 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.729+0000 7f99237fe640 10 client.27034 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.730+0000 7f99237fe640 1 -- 172.21.15.29:0/3152442554 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555fcf5edf0 con 0x5555fcbfd9f0 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.730+0000 7f99237fe640 10 client.27034 handle_command_reply: tid=0 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.730+0000 7f9950a72740 1 client.27034 shutdown 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.730+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 msgr2=0x5555fcff1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.730+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9938003750 tx=0x7f9938031cc0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 shutdown_connections 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbfd9f0 0x5555fd003f80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc3f070 0x5555fcffe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc1e390 0x5555fcff2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f99237fe640 0 client.27034 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 --2- 172.21.15.29:0/3152442554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcff1e00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 >> 172.21.15.29:0/3152442554 conn(0x5555fcc33460 msgr2=0x5555fcfe9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 shutdown_connections 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.731+0000 7f9950a72740 1 -- 172.21.15.29:0/3152442554 wait complete. 2024-09-21T01:57:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.733+0000 7f9950a72740 20 client.27034 trim_cache size 0 max 16384 2024-09-21T01:57:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.754+0000 7f9950a72740 1 Processor -- start 2024-09-21T01:57:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.754+0000 7f9950a72740 1 -- start start 2024-09-21T01:57:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcffe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ad0b0 con 0x5555fcc1e390 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99441aecb0 con 0x5555fcc3f070 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9950a72740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ad780 con 0x5555fcc161b0 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9943fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f99437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9943fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59326/0 (socket says 172.21.15.29:59326) 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f99437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38398/0 (socket says 172.21.15.29:38398) 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f9943fff640 1 -- 172.21.15.29:0/3638013712 learned_addr learned my addr 172.21.15.29:0/3638013712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.755+0000 7f99437fe640 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcffe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 -- 172.21.15.29:0/3638013712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 msgr2=0x5555fcfe8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 -- 172.21.15.29:0/3638013712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 msgr2=0x5555fcffe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcffe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 -- 172.21.15.29:0/3638013712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fcff49b0 con 0x5555fcc1e390 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f99437fe640 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcffe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.756+0000 7f9943fff640 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x5555fcdad7e0 tx=0x7f993400afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dbfcdf79fec456 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.757+0000 7f9929ffb640 1 -- 172.21.15.29:0/3638013712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99340040f0 con 0x5555fcc1e390 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.757+0000 7f9929ffb640 1 -- 172.21.15.29:0/3638013712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340054d0 con 0x5555fcc1e390 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.757+0000 7f9929ffb640 1 -- 172.21.15.29:0/3638013712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934004730 con 0x5555fcc1e390 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- 172.21.15.29:0/3638013712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 msgr2=0x5555fcff3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x5555fcdad7e0 tx=0x7f993400afa0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- 172.21.15.29:0/3638013712 shutdown_connections 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fcffe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fcff3a30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 --2- 172.21.15.29:0/3638013712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fcfe8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- 172.21.15.29:0/3638013712 >> 172.21.15.29:0/3638013712 conn(0x5555fcc33460 msgr2=0x5555fcff3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- 172.21.15.29:0/3638013712 shutdown_connections 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- 172.21.15.29:0/3638013712 wait complete. 2024-09-21T01:57:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.758+0000 7f9950a72740 1 -- start start 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 Processor -- start 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fd0077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fd004d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fd0052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9943fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fd004d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9943fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fd004d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59328/0 (socket says 172.21.15.29:59328) 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9943fff640 1 -- 172.21.15.29:0/3637945273 learned_addr learned my addr 172.21.15.29:0/3637945273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f99437fe640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fd0077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ae5e0 con 0x5555fcc1e390 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99441adf10 con 0x5555fcc3f070 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ab090 con 0x5555fcc161b0 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.759+0000 7f9950a72740 10 client.0 _ll_register_callbacks cb 0x7f9938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fd0052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 -- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 msgr2=0x5555fd0052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fd0052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 -- 172.21.15.29:0/3637945273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 msgr2=0x5555fd004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fd004d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fd005a40 con 0x5555fcc161b0 2024-09-21T01:57:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99437fe640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fd0077f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9938001eb0 tx=0x7f993804e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4a27baa9c80301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99237fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993805f040 con 0x5555fcc161b0 2024-09-21T01:57:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.760+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9938031920 con 0x5555fcc161b0 2024-09-21T01:57:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.761+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99340054d0 con 0x5555fcc161b0 2024-09-21T01:57:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.762+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9938035040 con 0x5555fcc161b0 2024-09-21T01:57:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.762+0000 7f9950a72740 10 client.27040 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.762+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5555fd06e660 con 0x5555fcc161b0 2024-09-21T01:57:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.762+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9938031dc0 con 0x5555fcc161b0 2024-09-21T01:57:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9938059040 con 0x5555fcc161b0 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 10 client.27040 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 20 client.27040 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbd2da0 0x5555fd070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 4 client.27040 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.763+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f99340054d0 con 0x5555fcbd2da0 2024-09-21T01:57:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.764+0000 7f9943fff640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbd2da0 0x5555fd070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.779+0000 7f99237fe640 10 client.27040 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.779+0000 7f9943fff640 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbd2da0 0x5555fd070d20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79266791ac4e72d server_cookie=7fa3fa674e8b2397 in_seq=0 out_seq=0 2024-09-21T01:57:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.779+0000 7f99237fe640 1 -- 172.21.15.29:0/3637945273 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f99340054d0 con 0x5555fcbd2da0 2024-09-21T01:57:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.779+0000 7f99237fe640 10 client.27040 handle_command_reply: tid=0 2024-09-21T01:57:04.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f9950a72740 1 client.27040 shutdown 2024-09-21T01:57:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 msgr2=0x5555fd0077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fd0077f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9938001eb0 tx=0x7f993804e040 comp rx=0 tx=0).stop 2024-09-21T01:57:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 shutdown_connections 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555fcbd2da0 0x5555fd070d20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.784+0000 7f99237fe640 0 client.27040 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555fcc3f070 0x5555fd0052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555fcc1e390 0x5555fd004d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 --2- 172.21.15.29:0/3637945273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fcc161b0 0x5555fd0077f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 >> 172.21.15.29:0/3637945273 conn(0x5555fcc33460 msgr2=0x5555fcfe7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 shutdown_connections 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 1 -- 172.21.15.29:0/3637945273 wait complete. 2024-09-21T01:57:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.785+0000 7f9950a72740 20 client.27040 trim_cache size 0 max 16384 2024-09-21T01:57:04.788 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 msgr2=0x7f992c0bce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 0x7f992c0bce60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f993c0049d0 tx=0x7f993c00efd0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 msgr2=0x7f994419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f993c00eb10 tx=0x7f993c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 shutdown_connections 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f992c0ba950 0x7f992c0bce60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994410ddf0 0x7f994419dae0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9944105270 0x7f9944198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 --2- 172.21.15.29:0/2825153715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9944104840 0x7f99441982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:04.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.791+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 >> 172.21.15.29:0/2825153715 conn(0x7f9944079730 msgr2=0x7f9944107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:04.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.792+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 shutdown_connections 2024-09-21T01:57:04.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:04.792+0000 7f994a4fe640 1 -- 172.21.15.29:0/2825153715 wait complete. 2024-09-21T01:57:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:05 smithi029 ceph-mon[31228]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 1.33k op/s 2024-09-21T01:57:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:05 smithi190 ceph-mon[34368]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 1.33k op/s 2024-09-21T01:57:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:05 smithi066 ceph-mon[35001]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 1.33k op/s 2024-09-21T01:57:06.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:57:06.880 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:57:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 510.0 s, 2040.1 tps, lat 110.919 ms stddev 679.170 2024-09-21T01:57:08.039 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:07 smithi029 ceph-mon[31228]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.089+0000 7fbb38652640 1 -- 172.21.15.29:0/731702318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb301030f0 msgr2=0x7fbb30107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.089+0000 7fbb38652640 1 --2- 172.21.15.29:0/731702318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb301030f0 0x7fbb30107620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009930 tx=0x7fbb2002a440 comp rx=0 tx=0).stop 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 -- 172.21.15.29:0/731702318 shutdown_connections 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 --2- 172.21.15.29:0/731702318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb30107be0 0x7fbb301080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 --2- 172.21.15.29:0/731702318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb301030f0 0x7fbb30107620 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 --2- 172.21.15.29:0/731702318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301027a0 0x7fbb30102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 -- 172.21.15.29:0/731702318 >> 172.21.15.29:0/731702318 conn(0x7fbb30079730 msgr2=0x7fbb30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 -- 172.21.15.29:0/731702318 shutdown_connections 2024-09-21T01:57:08.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.090+0000 7fbb38652640 1 -- 172.21.15.29:0/731702318 wait complete. 2024-09-21T01:57:08.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.091+0000 7fbb38652640 1 Processor -- start 2024-09-21T01:57:08.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.091+0000 7fbb38652640 1 -- start start 2024-09-21T01:57:08.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.091+0000 7fbb38652640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb301027a0 0x7fbb301a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.091+0000 7fbb38652640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb38652640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 0x7fbb301a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb38652640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3011a090 con 0x7fbb30107be0 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb38652640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30119f10 con 0x7fbb301030f0 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb38652640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3011a210 con 0x7fbb301027a0 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:56398/0 (socket says 172.21.15.29:56398) 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 -- 172.21.15.29:0/2384134677 learned_addr learned my addr 172.21.15.29:0/2384134677 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:08.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb36bef640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 0x7fbb301a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 -- 172.21.15.29:0/2384134677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb301027a0 msgr2=0x7fbb301a0a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb301027a0 0x7fbb301a0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.092+0000 7fbb35bed640 1 -- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 msgr2=0x7fbb301a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb35bed640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 0x7fbb301a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb35bed640 1 -- 172.21.15.29:0/2384134677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb301a6990 con 0x7fbb301030f0 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb36bef640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 0x7fbb301a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:08.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb35bed640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb2002a960 tx=0x7fbb20030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb277fe640 1 -- 172.21.15.29:0/2384134677 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2003f070 con 0x7fbb301030f0 2024-09-21T01:57:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.093+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb301a6c20 con 0x7fbb301030f0 2024-09-21T01:57:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.094+0000 7fbb277fe640 1 -- 172.21.15.29:0/2384134677 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb20004460 con 0x7fbb301030f0 2024-09-21T01:57:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.094+0000 7fbb277fe640 1 -- 172.21.15.29:0/2384134677 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20035030 con 0x7fbb301030f0 2024-09-21T01:57:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.094+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301ad3e0 con 0x7fbb301030f0 2024-09-21T01:57:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.098+0000 7fbb3ebc6740 1 Processor -- start 2024-09-21T01:57:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.099+0000 7fbb3ebc6740 1 -- start start 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.099+0000 7fbb277fe640 1 -- 172.21.15.29:0/2384134677 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbb20042070 con 0x7fbb301030f0 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 0x5617dea74970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea74eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617de9eae60 con 0x5617de6a4490 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617de9da290 con 0x5617de6cd350 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617de9de0a0 con 0x5617de6ac670 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb277fe640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 0x7fbb1c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb36bef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb36bef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:56412/0 (socket says 172.21.15.29:56412) 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb36bef640 1 -- 172.21.15.29:0/1300972097 learned_addr learned my addr 172.21.15.29:0/1300972097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.101+0000 7fbb277fe640 1 -- 172.21.15.29:0/2384134677 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb20107af0 con 0x7fbb301030f0 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb363ee640 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 0x5617dea74970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 0x7fbb1c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 -- 172.21.15.29:0/1300972097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 msgr2=0x5617dea74eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea74eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 -- 172.21.15.29:0/1300972097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 msgr2=0x5617dea74970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 0x5617dea74970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb36bef640 1 -- 172.21.15.29:0/1300972097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617de738930 con 0x5617de6cd350 2024-09-21T01:57:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.102+0000 7fbb363ee640 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 0x5617dea74970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.103+0000 7fbb36bef640 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb28013e90 tx=0x7fbb280117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63fbb4ccb475364 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.103+0000 7fbb36bef640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 0x7fbb1c0b8aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbb28010000 tx=0x7fbb2801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.103+0000 7fbb24ff9640 1 -- 172.21.15.29:0/1300972097 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28004410 con 0x5617de6cd350 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.103+0000 7fbb24ff9640 1 -- 172.21.15.29:0/1300972097 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28022070 con 0x5617de6cd350 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.103+0000 7fbb24ff9640 1 -- 172.21.15.29:0/1300972097 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800ee20 con 0x5617de6cd350 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.104+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/1300972097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 msgr2=0x5617dea74430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.104+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb28013e90 tx=0x7fbb280117b0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/1300972097 shutdown_connections 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea74eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6a4490 0x5617dea74970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/1300972097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6cd350 0x5617dea74430 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/1300972097 >> 172.21.15.29:0/1300972097 conn(0x5617de6c1740 msgr2=0x5617dea72320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/1300972097 shutdown_connections 2024-09-21T01:57:08.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/1300972097 wait complete. 2024-09-21T01:57:08.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 -- start start 2024-09-21T01:57:08.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.105+0000 7fbb3ebc6740 1 Processor -- start 2024-09-21T01:57:08.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6cd350 0x5617dea85ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617de83b890 con 0x5617de6cd350 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5617de974d40 con 0x5617de6a4490 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617dea82bb0 con 0x5617de6ac670 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb3ebc6740 10 client.0 _ll_register_callbacks cb 0x5617de9d8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38424/0 (socket says 172.21.15.29:38424) 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 -- 172.21.15.29:0/4266622849 learned_addr learned my addr 172.21.15.29:0/4266622849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:08.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb36bef640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6cd350 0x5617dea85ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb363ee640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 -- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 msgr2=0x5617dea800f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea800f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 -- 172.21.15.29:0/4266622849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6cd350 msgr2=0x5617dea85ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6cd350 0x5617dea85ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.106+0000 7fbb35bed640 1 -- 172.21.15.29:0/4266622849 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dea86510 con 0x5617de6ac670 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.107+0000 7fbb35bed640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005820 tx=0x7fbb200c9b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26da0d2fcfc7c961 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.107+0000 7fbb19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.107+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20049030 con 0x5617de6ac670 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.107+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb200f10e0 con 0x5617de6ac670 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.107+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb28022070 con 0x5617de6ac670 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbb200f12e0 con 0x5617de6ac670 2024-09-21T01:57:08.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 10 client.27046 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5617dea8eee0 con 0x5617de6ac670 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28022070 con 0x5617de6ac670 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fbb200d71c0 con 0x5617de6ac670 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 10 client.27046 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 10 client.27046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 20 client.27046 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de68bcd0 0x5617dea92380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 4 client.27046 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617de9ecdb0 con 0x5617de68bcd0 2024-09-21T01:57:08.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.108+0000 7fbb363ee640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de68bcd0 0x5617dea92380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.109+0000 7fbb19ffb640 10 client.27046 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:08.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.109+0000 7fbb363ee640 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de68bcd0 0x5617dea92380 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6bb5e69737cb3bf server_cookie=4076a074d66747e5 in_seq=0 out_seq=0 2024-09-21T01:57:08.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb19ffb640 1 -- 172.21.15.29:0/4266622849 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617de9ecdb0 con 0x5617de68bcd0 2024-09-21T01:57:08.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb19ffb640 10 client.27046 handle_command_reply: tid=0 2024-09-21T01:57:08.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 client.27046 shutdown 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 msgr2=0x5617dea806d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005820 tx=0x7fbb200c9b90 comp rx=0 tx=0).stop 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 shutdown_connections 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de68bcd0 0x5617dea92380 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6cd350 0x5617dea85ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6ac670 0x5617dea806d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb19ffb640 0 client.27046 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/4266622849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.110+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 >> 172.21.15.29:0/4266622849 conn(0x5617de6c1740 msgr2=0x5617dea775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.111+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 shutdown_connections 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.111+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/4266622849 wait complete. 2024-09-21T01:57:08.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.111+0000 7fbb3ebc6740 20 client.27046 trim_cache size 0 max 16384 2024-09-21T01:57:08.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.138+0000 7fbb3ebc6740 1 Processor -- start 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.138+0000 7fbb3ebc6740 1 -- start start 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.138+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b5a10 con 0x5617de6ac670 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b7610 con 0x5617de6a4490 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b60e0 con 0x5617de6cd350 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:56424/0 (socket says 172.21.15.29:56424) 2024-09-21T01:57:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 -- 172.21.15.29:0/3180239276 learned_addr learned my addr 172.21.15.29:0/3180239276 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb35bed640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb36bef640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 -- 172.21.15.29:0/3180239276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 msgr2=0x5617dea72350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea72350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 -- 172.21.15.29:0/3180239276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 msgr2=0x5617dea8ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb363ee640 1 -- 172.21.15.29:0/3180239276 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dea926f0 con 0x5617de6a4490 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.139+0000 7fbb36bef640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:57:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.140+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5617dea92a90 tx=0x7fbb2c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c675bc6f9d9b816 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.140+0000 7fbb1bfff640 1 -- 172.21.15.29:0/3180239276 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0142d0 con 0x5617de6a4490 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.140+0000 7fbb1bfff640 1 -- 172.21.15.29:0/3180239276 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb2c01f030 con 0x5617de6a4490 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.140+0000 7fbb1bfff640 1 -- 172.21.15.29:0/3180239276 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c004e40 con 0x5617de6a4490 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3180239276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 msgr2=0x5617dea85e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5617dea92a90 tx=0x7fbb2c005f90 comp rx=0 tx=0).stop 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3180239276 shutdown_connections 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3180239276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea85e10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3180239276 >> 172.21.15.29:0/3180239276 conn(0x5617de6c1740 msgr2=0x5617dea71c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3180239276 shutdown_connections 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3180239276 wait complete. 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.142+0000 7fbb3ebc6740 1 -- start start 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 Processor -- start 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea8c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b6f40 con 0x5617de6ac670 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b6870 con 0x5617de6a4490 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301b39f0 con 0x5617de6cd350 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb3ebc6740 10 client.0 _ll_register_callbacks cb 0x7fbb20009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb36bef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb35bed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb36bef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38438/0 (socket says 172.21.15.29:38438) 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb36bef640 1 -- 172.21.15.29:0/3242790400 learned_addr learned my addr 172.21.15.29:0/3242790400 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.143+0000 7fbb35bed640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59374/0 (socket says 172.21.15.29:59374) 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea8c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 -- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 msgr2=0x5617dea8c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea8c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 -- 172.21.15.29:0/3242790400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 msgr2=0x5617dea8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 -- 172.21.15.29:0/3242790400 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dea87480 con 0x5617de6cd350 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb36bef640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb28005fe0 tx=0x7fbb28009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c1f753aba2e942 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.144+0000 7fbb19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.145+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2802b030 con 0x5617de6cd350 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.145+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb2c01f030 con 0x5617de6cd350 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.145+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbb2800ed60 con 0x5617de6cd350 2024-09-21T01:57:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.145+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbb2800e4d0 con 0x5617de6cd350 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 10 client.27049 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5617deb0e7d0 con 0x5617de6cd350 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28036020 con 0x5617de6cd350 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fbb28027460 con 0x5617de6cd350 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 10 client.27049 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 10 client.27049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 20 client.27049 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de661080 0x5617deb10e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 4 client.27049 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:57:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.146+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fbb2c01f030 con 0x5617de661080 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.147+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de661080 0x5617deb10e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.147+0000 7fbb19ffb640 10 client.27049 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.147+0000 7fbb363ee640 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de661080 0x5617deb10e70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d215ada1d238933 server_cookie=adc89b6c63883250 in_seq=0 out_seq=0 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.149+0000 7fbb19ffb640 1 -- 172.21.15.29:0/3242790400 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fbb2c01f030 con 0x5617de661080 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.149+0000 7fbb19ffb640 10 client.27049 handle_command_reply: tid=0 2024-09-21T01:57:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.150+0000 7fbb3ebc6740 1 client.27049 shutdown 2024-09-21T01:57:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 msgr2=0x5617dea86cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb28005fe0 tx=0x7fbb28009e00 comp rx=0 tx=0).stop 2024-09-21T01:57:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 shutdown_connections 2024-09-21T01:57:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5617de661080 0x5617deb10e70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617de6cd350 0x5617dea86cf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.152+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617de6ac670 0x5617dea8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.151+0000 7fbb19ffb640 0 client.27049 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.152+0000 7fbb3ebc6740 1 --2- 172.21.15.29:0/3242790400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5617de6a4490 0x5617dea8c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.152+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 >> 172.21.15.29:0/3242790400 conn(0x5617de6c1740 msgr2=0x5617dea75360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.153+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 shutdown_connections 2024-09-21T01:57:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.153+0000 7fbb3ebc6740 1 -- 172.21.15.29:0/3242790400 wait complete. 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.153+0000 7fbb3ebc6740 20 client.27049 trim_cache size 0 max 16384 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "399d6a80-0726-417c-8b65-467fd131cefd", 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:57:08.155 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:57:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 msgr2=0x7fbb1c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 0x7fbb1c0b8aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbb28010000 tx=0x7fbb2801f000 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 msgr2=0x7fbb301a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb2002a960 tx=0x7fbb20030650 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 shutdown_connections 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbb1c0b6590 0x7fbb1c0b8aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb30107be0 0x7fbb301a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb301030f0 0x7fbb301a0fc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 --2- 172.21.15.29:0/2384134677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb301027a0 0x7fbb301a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 >> 172.21.15.29:0/2384134677 conn(0x7fbb30079730 msgr2=0x7fbb301055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 shutdown_connections 2024-09-21T01:57:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:08.154+0000 7fbb38652640 1 -- 172.21.15.29:0/2384134677 wait complete. 2024-09-21T01:57:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:07 smithi190 ceph-mon[34368]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-09-21T01:57:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:07 smithi066 ceph-mon[35001]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-09-21T01:57:08.901 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:57:09.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:57:09.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:08 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:57:09.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:08 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:57:09.342 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:09.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:57:09.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:57:09.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:57:09.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:57:09.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:08 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:57:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 -- 172.21.15.29:0/1727944293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4108d20 msgr2=0x7fbfe410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1727944293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4108d20 0x7fbfe410f550 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009ca0 tx=0x7fbfdc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 -- 172.21.15.29:0/1727944293 shutdown_connections 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1727944293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4108d20 0x7fbfe410f550 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1727944293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1727944293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4102ae0 0x7fbfe4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.942+0000 7fbfeac27640 1 -- 172.21.15.29:0/1727944293 >> 172.21.15.29:0/1727944293 conn(0x7fbfe4079760 msgr2=0x7fbfe4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.943+0000 7fbfeac27640 1 -- 172.21.15.29:0/1727944293 shutdown_connections 2024-09-21T01:57:09.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.943+0000 7fbfeac27640 1 -- 172.21.15.29:0/1727944293 wait complete. 2024-09-21T01:57:09.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 Processor -- start 2024-09-21T01:57:09.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 -- start start 2024-09-21T01:57:09.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 0x7fbfe40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 0x7fbfe41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4112990 con 0x7fbfe4102ae0 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4112810 con 0x7fbfe41083d0 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfeac27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4112b10 con 0x7fbfe4108d20 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 0x7fbfe40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 0x7fbfe40ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59390/0 (socket says 172.21.15.29:59390) 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfe89c3640 1 -- 172.21.15.29:0/1961198185 learned_addr learned my addr 172.21.15.29:0/1961198185 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.944+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 0x7fbfe41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 -- 172.21.15.29:0/1961198185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 msgr2=0x7fbfe41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 0x7fbfe41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 -- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 msgr2=0x7fbfe40ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 0x7fbfe40ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 -- 172.21.15.29:0/1961198185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe41a86d0 con 0x7fbfe41083d0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 0x7fbfe41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe40ffbb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9c0 tx=0x7fbfd000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe1ffb640 1 -- 172.21.15.29:0/1961198185 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000ccc0 con 0x7fbfe41083d0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe1ffb640 1 -- 172.21.15.29:0/1961198185 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd00108e0 con 0x7fbfe41083d0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe41a89c0 con 0x7fbfe41083d0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfe1ffb640 1 -- 172.21.15.29:0/1961198185 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00187c0 con 0x7fbfe41083d0 2024-09-21T01:57:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.945+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a8f00 con 0x7fbfe41083d0 2024-09-21T01:57:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 Processor -- start 2024-09-21T01:57:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 -- start start 2024-09-21T01:57:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbfe1ffb640 1 -- 172.21.15.29:0/1961198185 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbfd0010430 con 0x7fbfe41083d0 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x56220381d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x56220381dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562203793e50 con 0x562203476070 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562203783280 con 0x56220344d1b0 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbff119b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562203787090 con 0x562203455390 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.950+0000 7fbfe91c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe91c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59392/0 (socket says 172.21.15.29:59392) 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe91c4640 1 -- 172.21.15.29:0/2067010040 learned_addr learned my addr 172.21.15.29:0/2067010040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x56220381d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe1ffb640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 0x7fbfd40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2067010040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 msgr2=0x56220381dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 0x7fbfd40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe1ffb640 1 -- 172.21.15.29:0/1961198185 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbfd00e7690 con 0x7fbfe41083d0 2024-09-21T01:57:09.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.951+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x56220381dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x56220381dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2067010040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 msgr2=0x56220381d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2067010040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622034e1650 con 0x56220344d1b0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x56220381d7b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009cc0 tx=0x7fbfd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6103618ec63c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 0x7fbfd40b8a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0094e0 tx=0x7fbfdc02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfceffd640 1 -- 172.21.15.29:0/2067010040 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8005220 con 0x56220344d1b0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfceffd640 1 -- 172.21.15.29:0/2067010040 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd8004c50 con 0x56220344d1b0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.952+0000 7fbfceffd640 1 -- 172.21.15.29:0/2067010040 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80197d0 con 0x56220344d1b0 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.953+0000 7fbff119b740 1 -- 172.21.15.29:0/2067010040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 msgr2=0x56220381d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.953+0000 7fbff119b740 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x56220381d7b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009cc0 tx=0x7fbfd8011810 comp rx=0 tx=0).stop 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.954+0000 7fbff119b740 1 -- 172.21.15.29:0/2067010040 shutdown_connections 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.954+0000 7fbff119b740 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x56220381dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.954+0000 7fbff119b740 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x56220381d7b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.954+0000 7fbff119b740 1 --2- 172.21.15.29:0/2067010040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x56220381d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.954+0000 7fbff119b740 1 -- 172.21.15.29:0/2067010040 >> 172.21.15.29:0/2067010040 conn(0x56220346a460 msgr2=0x56220381b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:09.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.955+0000 7fbff119b740 1 -- 172.21.15.29:0/2067010040 shutdown_connections 2024-09-21T01:57:09.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.955+0000 7fbff119b740 1 -- 172.21.15.29:0/2067010040 wait complete. 2024-09-21T01:57:09.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.955+0000 7fbff119b740 1 -- start start 2024-09-21T01:57:09.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.956+0000 7fbff119b740 1 Processor -- start 2024-09-21T01:57:09.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.956+0000 7fbff119b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203828ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:56486/0 (socket says 172.21.15.29:56486) 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x562203834d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2114512764 learned_addr learned my addr 172.21.15.29:0/2114512764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5622035e4700 con 0x562203476070 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203828ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56220371e110 con 0x56220344d1b0 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x562203834d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56220382ba50 con 0x562203455390 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 10 client.0 _ll_register_callbacks cb 0x562203737990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2114512764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 msgr2=0x562203828ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203828ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2114512764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 msgr2=0x562203834d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x562203834d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbfe89c3640 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220374bc40 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.957+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd8004c50 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbfd801b040 tx=0x7fbfd8005030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189ff9347ac44196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbfc3fff640 10 client.27098 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbfc3fff640 1 -- 172.21.15.29:0/2114512764 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8033050 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbfc3fff640 1 -- 172.21.15.29:0/2114512764 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbfd80055e0 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbfc3fff640 1 -- 172.21.15.29:0/2114512764 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfd8029070 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbff119b740 10 client.27098 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.958+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=43}) -- 0x562203837740 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbfc3fff640 1 -- 172.21.15.29:0/2114512764 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fbfd8019510 con 0x56220344d1b0 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 10 client.27098 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 10 client.27098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 20 client.27098 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:09.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5622034349f0 0x56220383aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 4 client.27098 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562203795da0 con 0x5622034349f0 2024-09-21T01:57:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.959+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5622034349f0 0x56220383aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.960+0000 7fbfc3fff640 10 client.27098 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.960+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5622034349f0 0x56220383aaa0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ff213281c03a76 server_cookie=a836ac671a094482 in_seq=0 out_seq=0 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.960+0000 7fbfc3fff640 1 -- 172.21.15.29:0/2114512764 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562203795da0 con 0x5622034349f0 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.960+0000 7fbfc3fff640 10 client.27098 handle_command_reply: tid=0 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.960+0000 7fbff119b740 1 client.27098 shutdown 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 msgr2=0x5622038289b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbfd801b040 tx=0x7fbfd8005030 comp rx=0 tx=0).stop 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 shutdown_connections 2024-09-21T01:57:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5622034349f0 0x56220383aaa0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x562203834d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203828ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 --2- 172.21.15.29:0/2114512764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56220344d1b0 0x5622038289b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbfc3fff640 0 client.27098 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 >> 172.21.15.29:0/2114512764 conn(0x56220346a460 msgr2=0x562203820450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 shutdown_connections 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 1 -- 172.21.15.29:0/2114512764 wait complete. 2024-09-21T01:57:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.961+0000 7fbff119b740 20 client.27098 trim_cache size 0 max 16384 2024-09-21T01:57:09.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.987+0000 7fbff119b740 1 Processor -- start 2024-09-21T01:57:09.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.987+0000 7fbff119b740 1 -- start start 2024-09-21T01:57:09.989 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:57:09.989 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2024-09-21T01:57:09.989 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.990 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.990 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.990 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:09 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:09.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 0x56220382a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203834cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203476070 0x56220381e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1520 con 0x56220344d1b0 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b3120 con 0x562203476070 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbff119b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1bf0 con 0x562203455390 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 0x56220382a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203834cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 0x56220382a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59408/0 (socket says 172.21.15.29:59408) 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbfe89c3640 1 -- 172.21.15.29:0/3615322376 learned_addr learned my addr 172.21.15.29:0/3615322376 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.988+0000 7fbfe91c4640 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203476070 0x56220381e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 -- 172.21.15.29:0/3615322376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203476070 msgr2=0x56220381e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203476070 0x56220381e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 -- 172.21.15.29:0/3615322376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 msgr2=0x56220382a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 0x56220382a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 -- 172.21.15.29:0/3615322376 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562203816580 con 0x562203455390 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203834cb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00e51f0 tx=0x7fbfd00e5070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de86899b2b57b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:09.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfce7fc640 1 -- 172.21.15.29:0/3615322376 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b2100 con 0x562203455390 2024-09-21T01:57:09.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfce7fc640 1 -- 172.21.15.29:0/3615322376 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd00b2310 con 0x562203455390 2024-09-21T01:57:09.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.989+0000 7fbfce7fc640 1 -- 172.21.15.29:0/3615322376 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0010bb0 con 0x562203455390 2024-09-21T01:57:09.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.990+0000 7fbff119b740 1 -- 172.21.15.29:0/3615322376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 msgr2=0x562203834cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:09.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.990+0000 7fbff119b740 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203834cb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00e51f0 tx=0x7fbfd00e5070 comp rx=0 tx=0).stop 2024-09-21T01:57:09.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 -- 172.21.15.29:0/3615322376 shutdown_connections 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203476070 0x56220381e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562203455390 0x562203834cb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 --2- 172.21.15.29:0/3615322376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56220344d1b0 0x56220382a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 -- 172.21.15.29:0/3615322376 >> 172.21.15.29:0/3615322376 conn(0x56220346a460 msgr2=0x56220381dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 -- 172.21.15.29:0/3615322376 shutdown_connections 2024-09-21T01:57:09.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.992+0000 7fbff119b740 1 -- 172.21.15.29:0/3615322376 wait complete. 2024-09-21T01:57:09.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.993+0000 7fbff119b740 1 -- start start 2024-09-21T01:57:09.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.993+0000 7fbff119b740 1 Processor -- start 2024-09-21T01:57:09.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.994+0000 7fbff119b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.994+0000 7fbff119b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 0x5622038996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.994+0000 7fbff119b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x5622038a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.994+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.994+0000 7fbfe89c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38488/0 (socket says 172.21.15.29:38488) 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 0x5622038996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 0x5622038996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:56506/0 (socket says 172.21.15.29:56506) 2024-09-21T01:57:09.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbff119b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b2a50 con 0x562203476070 2024-09-21T01:57:09.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbff119b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b2380 con 0x562203455390 2024-09-21T01:57:09.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe91c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x5622038a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:09.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe89c3640 1 -- 172.21.15.29:0/1051358817 learned_addr learned my addr 172.21.15.29:0/1051358817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:09.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41af500 con 0x56220344d1b0 2024-09-21T01:57:09.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbff119b740 10 client.0 _ll_register_callbacks cb 0x7fbfd0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:10.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe89c3640 1 -- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 msgr2=0x5622038996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:10.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.995+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 0x5622038996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfe89c3640 1 -- 172.21.15.29:0/1051358817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 msgr2=0x5622038a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:10.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x5622038a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfe89c3640 1 -- 172.21.15.29:0/1051358817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562203899fc0 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfe89c3640 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8004a80 tx=0x7fbfd8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca838139d55407d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8033040 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbfd802e440 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd00b2310 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.996+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfd802e640 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 10 client.27064 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5622038a5800 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd803d070 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fbfd80434a0 con 0x56220344d1b0 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 10 client.27064 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:10.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 20 client.27064 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562203409da0 0x5622038a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 4 client.27064 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:57:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbfd00b2310 con 0x562203409da0 2024-09-21T01:57:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.997+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562203409da0 0x5622038a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.998+0000 7fbfc3fff640 10 client.27064 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.998+0000 7fbfe3fff640 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562203409da0 0x5622038a7ec0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5d560069ff7b38 server_cookie=b1c76d4ff0984cac in_seq=0 out_seq=0 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.999+0000 7fbfc3fff640 1 -- 172.21.15.29:0/1051358817 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x7fbfd00b2310 con 0x562203409da0 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.999+0000 7fbfc3fff640 10 client.27064 handle_command_reply: tid=0 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:09.999+0000 7fbff119b740 1 client.27064 shutdown 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 msgr2=0x5622038991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8004a80 tx=0x7fbfd8005f90 comp rx=0 tx=0).stop 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 shutdown_connections 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562203409da0 0x5622038a7ec0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562203476070 0x5622038a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562203455390 0x5622038996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 --2- 172.21.15.29:0/1051358817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56220344d1b0 0x5622038991b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 >> 172.21.15.29:0/1051358817 conn(0x56220346a460 msgr2=0x56220382a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.000+0000 7fbfc3fff640 0 client.27064 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.001+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 shutdown_connections 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.001+0000 7fbff119b740 1 -- 172.21.15.29:0/1051358817 wait complete. 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.001+0000 7fbff119b740 20 client.27064 trim_cache size 0 max 16384 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:57:10.006 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "399d6a80-0726-417c-8b65-467fd131cefd", 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.007 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:57:10.008 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.010 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.011 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:57:10.012 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:57:10.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:57:10.014 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:57:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 msgr2=0x7fbfd40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 0x7fbfd40b8a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0094e0 tx=0x7fbfdc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 msgr2=0x7fbfe40ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe40ffbb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9c0 tx=0x7fbfd000ee80 comp rx=0 tx=0).stop 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 shutdown_connections 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbfd40b6510 0x7fbfd40b8a20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe4108d20 0x7fbfe41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfe41083d0 0x7fbfe40ffbb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 --2- 172.21.15.29:0/1961198185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfe4102ae0 0x7fbfe40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 >> 172.21.15.29:0/1961198185 conn(0x7fbfe4079760 msgr2=0x7fbfe4101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 shutdown_connections 2024-09-21T01:57:10.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:10.002+0000 7fbfeac27640 1 -- 172.21.15.29:0/1961198185 wait complete. 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:09 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:09 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:57:11.102 INFO:tasks.cephfs.filesystem:scrub status for tag:399d6a80-0726-417c-8b65-467fd131cefd - {'path': '/', 'tag': '399d6a80-0726-417c-8b65-467fd131cefd', 'options': 'recursive,force'} 2024-09-21T01:57:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:11 smithi190 ceph-mon[34368]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 33 MiB/s wr, 1.44k op/s 2024-09-21T01:57:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:11 smithi066 ceph-mon[35001]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 33 MiB/s wr, 1.44k op/s 2024-09-21T01:57:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:11 smithi029 ceph-mon[31228]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 33 MiB/s wr, 1.44k op/s 2024-09-21T01:57:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 515.0 s, 1468.2 tps, lat 68.468 ms stddev 76.856 2024-09-21T01:57:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:13 smithi190 ceph-mon[34368]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 17 MiB/s wr, 821 op/s 2024-09-21T01:57:14.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:13 smithi066 ceph-mon[35001]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 17 MiB/s wr, 821 op/s 2024-09-21T01:57:14.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:13 smithi029 ceph-mon[31228]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 17 MiB/s wr, 821 op/s 2024-09-21T01:57:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:15 smithi190 ceph-mon[34368]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-21T01:57:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:15 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:57:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:15 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:57:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:15 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:57:16.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:15 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:57:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:15 smithi066 ceph-mon[35001]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-21T01:57:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:15 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:57:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:15 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:57:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:15 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:57:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:15 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:57:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:15 smithi029 ceph-mon[31228]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-21T01:57:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:15 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:57:16.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:15 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:57:16.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:15 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:57:16.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:15 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 -- 172.21.15.29:0/3547746661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c10a3c0 msgr2=0x7ff82c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 --2- 172.21.15.29:0/3547746661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c10a3c0 0x7ff82c10dd70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff828009900 tx=0x7ff82802a440 comp rx=0 tx=0).stop 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 -- 172.21.15.29:0/3547746661 shutdown_connections 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 --2- 172.21.15.29:0/3547746661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c110c10 0x7ff82c113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 --2- 172.21.15.29:0/3547746661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 0x7ff82c1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 --2- 172.21.15.29:0/3547746661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c10a3c0 0x7ff82c10dd70 secure :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff828009900 tx=0x7ff82802a440 comp rx=0 tx=0).stop 2024-09-21T01:57:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.698+0000 7ff834430640 1 -- 172.21.15.29:0/3547746661 >> 172.21.15.29:0/3547746661 conn(0x7ff82c070810 msgr2=0x7ff82c072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.699+0000 7ff834430640 1 -- 172.21.15.29:0/3547746661 shutdown_connections 2024-09-21T01:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.700+0000 7ff834430640 1 -- 172.21.15.29:0/3547746661 wait complete. 2024-09-21T01:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 Processor -- start 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 -- start start 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 0x7ff82c1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 0x7ff82c1aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c116a90 con 0x7ff82c110c10 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c116910 con 0x7ff82c1a7c00 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.701+0000 7ff834430640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c116c10 con 0x7ff82c10e2b0 2024-09-21T01:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 0x7ff82c1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58934/0 (socket says 172.21.15.29:58934) 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 -- 172.21.15.29:0/3306265357 learned_addr learned my addr 172.21.15.29:0/3306265357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 0x7ff82c1aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 -- 172.21.15.29:0/3306265357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 msgr2=0x7ff82c1a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 0x7ff82c1a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.702+0000 7ff8319cb640 1 -- 172.21.15.29:0/3306265357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 msgr2=0x7ff82c1aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.703+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 0x7ff82c1aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.703+0000 7ff8319cb640 1 -- 172.21.15.29:0/3306265357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c1aa840 con 0x7ff82c110c10 2024-09-21T01:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.703+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 0x7ff82c1aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.703+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff82400ebc0 tx=0x7ff82400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.704+0000 7ff81f7fe640 1 -- 172.21.15.29:0/3306265357 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824010630 con 0x7ff82c110c10 2024-09-21T01:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.704+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c1a0fb0 con 0x7ff82c110c10 2024-09-21T01:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.704+0000 7ff81f7fe640 1 -- 172.21.15.29:0/3306265357 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff824010c80 con 0x7ff82c110c10 2024-09-21T01:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.704+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c1a15d0 con 0x7ff82c110c10 2024-09-21T01:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.704+0000 7ff81f7fe640 1 -- 172.21.15.29:0/3306265357 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824010630 con 0x7ff82c110c10 2024-09-21T01:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.709+0000 7ff83b481740 1 Processor -- start 2024-09-21T01:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.710+0000 7ff81f7fe640 1 -- 172.21.15.29:0/3306265357 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff824018800 con 0x7ff82c110c10 2024-09-21T01:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 -- start start 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea552f0 0x5558aee264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558aed986f0 con 0x5558aea7e1b0 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5558aed87b20 con 0x5558aea5d4d0 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff83b481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5558aed8b930 con 0x5558aea552f0 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff8329cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff8329cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58936/0 (socket says 172.21.15.29:58936) 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.711+0000 7ff8329cd640 1 -- 172.21.15.29:0/1456056338 learned_addr learned my addr 172.21.15.29:0/1456056338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8319cb640 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8321cc640 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea552f0 0x5558aee264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 -- 172.21.15.29:0/1456056338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea552f0 msgr2=0x5558aee264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea552f0 0x5558aee264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 -- 172.21.15.29:0/1456056338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 msgr2=0x5558aee27010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee27010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 -- 172.21.15.29:0/1456056338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558aed4d530 con 0x5558aea7e1b0 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff81f7fe640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 0x7ff8180b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8321cc640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 0x7ff8180b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff8329cd640 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff820009de0 tx=0x7ff82000e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66009a12cb05b1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.712+0000 7ff81f7fe640 1 -- 172.21.15.29:0/3306265357 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff824010430 con 0x7ff82c110c10 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.713+0000 7ff81cbfe640 1 -- 172.21.15.29:0/1456056338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820017070 con 0x5558aea7e1b0 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.713+0000 7ff81cbfe640 1 -- 172.21.15.29:0/1456056338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820013920 con 0x5558aea7e1b0 2024-09-21T01:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.713+0000 7ff81cbfe640 1 -- 172.21.15.29:0/1456056338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820012930 con 0x5558aea7e1b0 2024-09-21T01:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.714+0000 7ff8321cc640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 0x7ff8180b8e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff828034040 tx=0x7ff828031440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.714+0000 7ff83b481740 1 -- 172.21.15.29:0/1456056338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 msgr2=0x5558aee25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.714+0000 7ff83b481740 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff820009de0 tx=0x7ff82000e5a0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 -- 172.21.15.29:0/1456056338 shutdown_connections 2024-09-21T01:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee27010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea552f0 0x5558aee264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 --2- 172.21.15.29:0/1456056338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee25e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 -- 172.21.15.29:0/1456056338 >> 172.21.15.29:0/1456056338 conn(0x5558aea72300 msgr2=0x5558aee218e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.715+0000 7ff83b481740 1 -- 172.21.15.29:0/1456056338 shutdown_connections 2024-09-21T01:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.716+0000 7ff83b481740 1 -- 172.21.15.29:0/1456056338 wait complete. 2024-09-21T01:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.716+0000 7ff83b481740 1 -- start start 2024-09-21T01:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.716+0000 7ff83b481740 1 Processor -- start 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558aecbfab0 con 0x5558aea7e1b0 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5558aed2d730 con 0x5558aea552f0 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5558aee27b30 con 0x5558aea5d4d0 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff83b481740 10 client.0 _ll_register_callbacks cb 0x5558aed816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50294/0 (socket says 172.21.15.29:50294) 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 -- 172.21.15.29:0/208202114 learned_addr learned my addr 172.21.15.29:0/208202114 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8319cb640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8329cd640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 -- 172.21.15.29:0/208202114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 msgr2=0x5558aee2fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee2fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 -- 172.21.15.29:0/208202114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 msgr2=0x5558aee30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8321cc640 1 -- 172.21.15.29:0/208202114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558aeae9790 con 0x5558aea552f0 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.717+0000 7ff8329cd640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff8321cc640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff828009900 tx=0x7ff82802a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6bd9cfbba536a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828036040 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff828003890 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff820013920 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff828003a90 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff83b481740 10 client.27113 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.718+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5558aeb62620 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820013920 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff82803c040 con 0x5558aea552f0 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 10 client.27113 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 10 client.27113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 20 client.27113 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea3cd80 0x5558aee3ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 4 client.27113 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.719+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558aed9a640 con 0x5558aea3cd80 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.720+0000 7ff8319cb640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea3cd80 0x5558aee3ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.720+0000 7ff815ffb640 10 client.27113 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.720+0000 7ff8319cb640 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea3cd80 0x5558aee3ce70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebdb5389696549 server_cookie=1ce0c962e0083ee4 in_seq=0 out_seq=0 2024-09-21T01:57:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.721+0000 7ff815ffb640 1 -- 172.21.15.29:0/208202114 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558aed9a640 con 0x5558aea3cd80 2024-09-21T01:57:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.721+0000 7ff815ffb640 10 client.27113 handle_command_reply: tid=0 2024-09-21T01:57:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.721+0000 7ff83b481740 1 client.27113 shutdown 2024-09-21T01:57:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.721+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 msgr2=0x5558aee2f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.721+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff828009900 tx=0x7ff82802a440 comp rx=0 tx=0).stop 2024-09-21T01:57:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 shutdown_connections 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea3cd80 0x5558aee3ce70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff815ffb640 0 client.27113 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea7e1b0 0x5558aee30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee2fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 --2- 172.21.15.29:0/208202114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea552f0 0x5558aee2f6b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 >> 172.21.15.29:0/208202114 conn(0x5558aea72300 msgr2=0x5558aee1fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 shutdown_connections 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.723+0000 7ff83b481740 1 -- 172.21.15.29:0/208202114 wait complete. 2024-09-21T01:57:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.724+0000 7ff83b481740 20 client.27113 trim_cache size 0 max 16384 2024-09-21T01:57:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 Processor -- start 2024-09-21T01:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 -- start start 2024-09-21T01:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea3cd80 0x5558aee1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff81000e430 con 0x5558aea552f0 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff810010030 con 0x5558aea5d4d0 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.742+0000 7ff83b481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff81000eb00 con 0x5558aea3cd80 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea3cd80 0x5558aee1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50298/0 (socket says 172.21.15.29:50298) 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 -- 172.21.15.29:0/3452698122 learned_addr learned my addr 172.21.15.29:0/3452698122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 -- 172.21.15.29:0/3452698122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea3cd80 msgr2=0x5558aee1c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea3cd80 0x5558aee1c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 -- 172.21.15.29:0/3452698122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 msgr2=0x5558aee220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.743+0000 7ff8329cd640 1 -- 172.21.15.29:0/3452698122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558aee21930 con 0x5558aea5d4d0 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.744+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff820009de0 tx=0x7ff8200123d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3281da7b0c56cfa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.744+0000 7ff817fff640 1 -- 172.21.15.29:0/3452698122 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820017040 con 0x5558aea5d4d0 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.744+0000 7ff817fff640 1 -- 172.21.15.29:0/3452698122 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820025480 con 0x5558aea5d4d0 2024-09-21T01:57:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.744+0000 7ff817fff640 1 -- 172.21.15.29:0/3452698122 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200136d0 con 0x5558aea5d4d0 2024-09-21T01:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 -- 172.21.15.29:0/3452698122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 msgr2=0x5558aee225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff820009de0 tx=0x7ff8200123d0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 -- 172.21.15.29:0/3452698122 shutdown_connections 2024-09-21T01:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea5d4d0 0x5558aee225e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 --2- 172.21.15.29:0/3452698122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea3cd80 0x5558aee1c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.745+0000 7ff83b481740 1 -- 172.21.15.29:0/3452698122 >> 172.21.15.29:0/3452698122 conn(0x5558aea72300 msgr2=0x5558aee21220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.746+0000 7ff83b481740 1 -- 172.21.15.29:0/3452698122 shutdown_connections 2024-09-21T01:57:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.746+0000 7ff83b481740 1 -- 172.21.15.29:0/3452698122 wait complete. 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.746+0000 7ff83b481740 1 -- start start 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.746+0000 7ff83b481740 1 Processor -- start 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff81000f960 con 0x5558aea552f0 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff81000f290 con 0x5558aea3cd80 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff81000c410 con 0x5558aea5d4d0 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50302/0 (socket says 172.21.15.29:50302) 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 -- 172.21.15.29:0/3493438117 learned_addr learned my addr 172.21.15.29:0/3493438117 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff83b481740 10 client.0 _ll_register_callbacks cb 0x7ff828034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8329cd640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 -- 172.21.15.29:0/3493438117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 msgr2=0x5558aee36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee36640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 -- 172.21.15.29:0/3493438117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 msgr2=0x5558aee36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.747+0000 7ff8321cc640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee36100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff8321cc640 1 -- 172.21.15.29:0/3493438117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558aee36dd0 con 0x5558aea3cd80 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff8321cc640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff828031640 tx=0x7ff82802ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ce2be37c0f8428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828036040 con 0x5558aea3cd80 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff820025480 con 0x5558aea3cd80 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff828031ab0 con 0x5558aea3cd80 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff828031cb0 con 0x5558aea3cd80 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff83b481740 10 client.27119 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5558aeeb0080 con 0x5558aea3cd80 2024-09-21T01:57:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.748+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820025480 con 0x5558aea3cd80 2024-09-21T01:57:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff82803c040 con 0x5558aea3cd80 2024-09-21T01:57:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 10 client.27119 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 10 client.27119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 20 client.27119 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea7e1b0 0x5558aeeb29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 4 client.27119 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558aee20440 con 0x5558aea7e1b0 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.749+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea7e1b0 0x5558aeeb29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.750+0000 7ff8319cb640 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea7e1b0 0x5558aeeb29e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8a6fd6d7401a8c server_cookie=ddcc16ba0ca37569 in_seq=0 out_seq=0 2024-09-21T01:57:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.750+0000 7ff815ffb640 10 client.27119 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.762+0000 7ff815ffb640 1 -- 172.21.15.29:0/3493438117 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558aee20440 con 0x5558aea7e1b0 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.762+0000 7ff815ffb640 10 client.27119 handle_command_reply: tid=0 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.762+0000 7ff83b481740 1 client.27119 shutdown 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 msgr2=0x5558aee37440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff828031640 tx=0x7ff82802ab30 comp rx=0 tx=0).stop 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 shutdown_connections 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5558aea7e1b0 0x5558aeeb29e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558aea5d4d0 0x5558aee36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558aea552f0 0x5558aee36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff815ffb640 0 client.27119 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 --2- 172.21.15.29:0/3493438117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558aea3cd80 0x5558aee37440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 >> 172.21.15.29:0/3493438117 conn(0x5558aea72300 msgr2=0x5558aee21220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 shutdown_connections 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 1 -- 172.21.15.29:0/3493438117 wait complete. 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff83b481740 20 client.27119 trim_cache size 0 max 16384 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:57:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 msgr2=0x7ff8180b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 0x7ff8180b8e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff828034040 tx=0x7ff828031440 comp rx=0 tx=0).stop 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 msgr2=0x7ff82c1a76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.763+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff82400ebc0 tx=0x7ff82400c6a0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 shutdown_connections 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8180b6940 0x7ff8180b8e50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82c1a7c00 0x7ff82c1aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff82c110c10 0x7ff82c1a76c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 --2- 172.21.15.29:0/3306265357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c10e2b0 0x7ff82c1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 >> 172.21.15.29:0/3306265357 conn(0x7ff82c070810 msgr2=0x7ff82c10c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 shutdown_connections 2024-09-21T01:57:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:16.764+0000 7ff834430640 1 -- 172.21.15.29:0/3306265357 wait complete. 2024-09-21T01:57:17.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:16 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:57:17.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:16 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:57:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:16 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:57:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 520.0 s, 1598.4 tps, lat 62.141 ms stddev 74.308 2024-09-21T01:57:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:17 smithi190 ceph-mon[34368]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 787 op/s 2024-09-21T01:57:18.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:17 smithi066 ceph-mon[35001]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 787 op/s 2024-09-21T01:57:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:17 smithi029 ceph-mon[31228]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 787 op/s 2024-09-21T01:57:20.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:19 smithi029 ceph-mon[31228]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-09-21T01:57:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:19 smithi190 ceph-mon[34368]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-09-21T01:57:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:19 smithi066 ceph-mon[35001]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-09-21T01:57:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:21 smithi190 ceph-mon[34368]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-21T01:57:22.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:21 smithi066 ceph-mon[35001]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-21T01:57:22.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:21 smithi029 ceph-mon[31228]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-21T01:57:22.970 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 525.0 s, 2357.2 tps, lat 42.828 ms stddev 50.298 2024-09-21T01:57:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:23 smithi190 ceph-mon[34368]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.6 MiB/s wr, 687 op/s 2024-09-21T01:57:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:23 smithi066 ceph-mon[35001]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.6 MiB/s wr, 687 op/s 2024-09-21T01:57:24.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:23 smithi029 ceph-mon[31228]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.6 MiB/s wr, 687 op/s 2024-09-21T01:57:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:25 smithi190 ceph-mon[34368]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-21T01:57:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:25 smithi066 ceph-mon[35001]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-21T01:57:26.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:25 smithi029 ceph-mon[31228]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-21T01:57:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 530.0 s, 2097.6 tps, lat 47.198 ms stddev 53.144 2024-09-21T01:57:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:27 smithi066 ceph-mon[35001]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-21T01:57:28.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:27 smithi029 ceph-mon[31228]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-21T01:57:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:27 smithi190 ceph-mon[34368]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-21T01:57:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 -- 172.21.15.29:0/555627350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 msgr2=0x7f2d8c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 --2- 172.21.15.29:0/555627350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c106a90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2d8001a910 tx=0x7f2d8003b420 comp rx=0 tx=0).stop 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 -- 172.21.15.29:0/555627350 shutdown_connections 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 --2- 172.21.15.29:0/555627350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 --2- 172.21.15.29:0/555627350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c106a90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 --2- 172.21.15.29:0/555627350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 0x7f2d8c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.918+0000 7f2d92e07640 1 -- 172.21.15.29:0/555627350 >> 172.21.15.29:0/555627350 conn(0x7f2d8c0fdee0 msgr2=0x7f2d8c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.919+0000 7f2d92e07640 1 -- 172.21.15.29:0/555627350 shutdown_connections 2024-09-21T01:57:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.919+0000 7f2d92e07640 1 -- 172.21.15.29:0/555627350 wait complete. 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 Processor -- start 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 -- start start 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 0x7f2d8c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10b130 con 0x7f2d8c104610 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10afb0 con 0x7f2d8c06b710 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d92e07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10b2b0 con 0x7f2d8c1071d0 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33200/0 (socket says 172.21.15.29:33200) 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d8bfff640 1 -- 172.21.15.29:0/1085475584 learned_addr learned my addr 172.21.15.29:0/1085475584 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.920+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 0x7f2d8c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 -- 172.21.15.29:0/1085475584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 msgr2=0x7f2d8c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 0x7f2d8c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 -- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 msgr2=0x7f2d8c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 -- 172.21.15.29:0/1085475584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c199790 con 0x7f2d8c1071d0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.921+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c198f70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400eb20 tx=0x7f2d8400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.922+0000 7f2d89ffb640 1 -- 172.21.15.29:0/1085475584 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840039c0 con 0x7f2d8c1071d0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.922+0000 7f2d89ffb640 1 -- 172.21.15.29:0/1085475584 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d84003b60 con 0x7f2d8c1071d0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.922+0000 7f2d89ffb640 1 -- 172.21.15.29:0/1085475584 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84010670 con 0x7f2d8c1071d0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.922+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c199a80 con 0x7f2d8c1071d0 2024-09-21T01:57:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.922+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c1a0170 con 0x7f2d8c1071d0 2024-09-21T01:57:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.926+0000 7f2d89ffb640 1 -- 172.21.15.29:0/1085475584 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2d84004000 con 0x7f2d8c1071d0 2024-09-21T01:57:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.926+0000 7f2d99e4b740 1 Processor -- start 2024-09-21T01:57:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.927+0000 7f2d99e4b740 1 -- start start 2024-09-21T01:57:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.927+0000 7f2d89ffb640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 0x7f2d7c0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.927+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.927+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c9073f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.927+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90740010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c906b16f0 con 0x556c903764d0 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c906a0b20 con 0x556c9036e2f0 2024-09-21T01:57:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556c906a4930 con 0x556c903971b0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 0x7f2d7c0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d89ffb640 1 -- 172.21.15.29:0/1085475584 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d840e78b0 con 0x7f2d8c1071d0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:40056/0 (socket says 172.21.15.29:40056) 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 -- 172.21.15.29:0/1616177501 learned_addr learned my addr 172.21.15.29:0/1616177501 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90740010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c9073f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 -- 172.21.15.29:0/1616177501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 msgr2=0x556c9073f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c9073f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 -- 172.21.15.29:0/1616177501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 msgr2=0x556c90740010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90740010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 -- 172.21.15.29:0/1616177501 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c90666530 con 0x556c903971b0 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90740010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 0x7f2d7c0b8b70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d80004e80 tx=0x7f2d80005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.928+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009cf0 tx=0x7f2d78011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9868794a04465544 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.929+0000 7f2d76ffd640 1 -- 172.21.15.29:0/1616177501 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7801e070 con 0x556c903971b0 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.929+0000 7f2d76ffd640 1 -- 172.21.15.29:0/1616177501 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d78009de0 con 0x556c903971b0 2024-09-21T01:57:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.929+0000 7f2d76ffd640 1 -- 172.21.15.29:0/1616177501 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780196b0 con 0x556c903971b0 2024-09-21T01:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1616177501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 msgr2=0x556c9073ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009cf0 tx=0x7f2d78011810 comp rx=0 tx=0).stop 2024-09-21T01:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1616177501 shutdown_connections 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90740010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c9073f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1616177501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903971b0 0x556c9073ee50 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009cf0 tx=0x7f2d78011810 comp rx=0 tx=0).stop 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.930+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1616177501 >> 172.21.15.29:0/1616177501 conn(0x556c9038b300 msgr2=0x556c9073a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.931+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1616177501 shutdown_connections 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.931+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1616177501 wait complete. 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.931+0000 7f2d99e4b740 1 -- start start 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 Processor -- start 2024-09-21T01:57:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90748dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c90355d80 0x556c90749930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c905d8ab0 con 0x556c903764d0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c90646730 con 0x556c9036e2f0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556c90740b30 con 0x556c90355d80 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d99e4b740 10 client.0 _ll_register_callbacks cb 0x556c9069a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.932+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55224/0 (socket says 172.21.15.29:55224) 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 -- 172.21.15.29:0/168000621 learned_addr learned my addr 172.21.15.29:0/168000621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d913a4640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c90355d80 0x556c90749930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90748dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 -- 172.21.15.29:0/168000621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c90355d80 msgr2=0x556c90749930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c90355d80 0x556c90749930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 -- 172.21.15.29:0/168000621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 msgr2=0x556c90748dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90748dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 -- 172.21.15.29:0/168000621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c90402790 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d78012fd0 tx=0x7f2d78003ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d23c52a5a44e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7801e040 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d78009de0 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.933+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d780041e0 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2d780043e0 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 10 client.27131 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556c9047b620 con 0x556c9036e2f0 2024-09-21T01:57:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d78009de0 con 0x556c9036e2f0 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f2d78021030 con 0x556c9036e2f0 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 10 client.27131 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 10 client.27131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 20 client.27131 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.934+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c9032b1c0 0x556c90755b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.935+0000 7f2d99e4b740 4 client.27131 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.935+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c906b3640 con 0x556c9032b1c0 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.935+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c9032b1c0 0x556c90755b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.935+0000 7f2d6bfff640 10 client.27131 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.935+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c9032b1c0 0x556c90755b80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb18f0fe388b66b5 server_cookie=9bc3c8aa3649ac1b in_seq=0 out_seq=0 2024-09-21T01:57:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.936+0000 7f2d6bfff640 1 -- 172.21.15.29:0/168000621 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c906b3640 con 0x556c9032b1c0 2024-09-21T01:57:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.937+0000 7f2d6bfff640 10 client.27131 handle_command_reply: tid=0 2024-09-21T01:57:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.937+0000 7f2d99e4b740 1 client.27131 shutdown 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.938+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 msgr2=0x556c90748700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.938+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d78012fd0 tx=0x7f2d78003ff0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 shutdown_connections 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c9032b1c0 0x556c90755b80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c90355d80 0x556c90749930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c903764d0 0x556c90748dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/168000621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c9036e2f0 0x556c90748700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 >> 172.21.15.29:0/168000621 conn(0x556c9038b300 msgr2=0x556c90738ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d6bfff640 0 client.27131 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 shutdown_connections 2024-09-21T01:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 1 -- 172.21.15.29:0/168000621 wait complete. 2024-09-21T01:57:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.939+0000 7f2d99e4b740 20 client.27131 trim_cache size 0 max 16384 2024-09-21T01:57:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 Processor -- start 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 -- start start 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903764d0 0x556c90746c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1a8c10 con 0x556c9036e2f0 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33226/0 (socket says 172.21.15.29:33226) 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:55226/0 (socket says 172.21.15.29:55226) 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 -- 172.21.15.29:0/1875309962 learned_addr learned my addr 172.21.15.29:0/1875309962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1aa810 con 0x556c90355d80 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1a92e0 con 0x556c903764d0 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d913a4640 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903764d0 0x556c90746c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 -- 172.21.15.29:0/1875309962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903764d0 msgr2=0x556c90746c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903764d0 0x556c90746c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 -- 172.21.15.29:0/1875309962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 msgr2=0x556c90738ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d8bfff640 1 -- 172.21.15.29:0/1875309962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c9032b3f0 con 0x556c9036e2f0 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.960+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.961+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2d80000c00 tx=0x7f2d80058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24485174fe971356 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.961+0000 7f2d767fc640 1 -- 172.21.15.29:0/1875309962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80068070 con 0x556c9036e2f0 2024-09-21T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.961+0000 7f2d767fc640 1 -- 172.21.15.29:0/1875309962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80040d90 con 0x556c9036e2f0 2024-09-21T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.961+0000 7f2d767fc640 1 -- 172.21.15.29:0/1875309962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80046050 con 0x556c9036e2f0 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 msgr2=0x556c90750810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2d80000c00 tx=0x7f2d80058000 comp rx=0 tx=0).stop 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 shutdown_connections 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c903764d0 0x556c90746c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c9036e2f0 0x556c90750810 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/1875309962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c90355d80 0x556c90738ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 >> 172.21.15.29:0/1875309962 conn(0x556c9038b300 msgr2=0x556c907355c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 shutdown_connections 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.962+0000 7f2d99e4b740 1 -- 172.21.15.29:0/1875309962 wait complete. 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 -- start start 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 Processor -- start 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 0x556c9074d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c9074e330 0x556c9074e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1aa140 con 0x556c90355d80 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1a9a70 con 0x556c903764d0 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1a6bf0 con 0x556c9074e330 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d99e4b740 10 client.0 _ll_register_callbacks cb 0x7f2d7801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d90ba3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33228/0 (socket says 172.21.15.29:33228) 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d90ba3640 1 -- 172.21.15.29:0/3310034393 learned_addr learned my addr 172.21.15.29:0/3310034393 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.963+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 0x556c9074d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d913a4640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c9074e330 0x556c9074e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 -- 172.21.15.29:0/3310034393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c9074e330 msgr2=0x556c9074e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c9074e330 0x556c9074e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 -- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 msgr2=0x556c9074d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 0x556c9074d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 -- 172.21.15.29:0/3310034393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9074ef60 con 0x556c90355d80 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 0x556c9074d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d90ba3640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d78012fd0 tx=0x7f2d78009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0306dbb962f008 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7803e060 con 0x556c90355d80 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2d78024690 con 0x556c90355d80 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d80040d90 con 0x556c90355d80 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2d7801e040 con 0x556c90355d80 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d99e4b740 10 client.17355 fetch_fsmap learned FSMap version 43 2024-09-21T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.964+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556c90750340 con 0x556c90355d80 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80040d90 con 0x556c90355d80 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f2d78021040 con 0x556c90355d80 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 10 client.17355 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 20 client.17355 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c907c0540 0x556c907c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 4 client.17355 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c90739580 con 0x556c907c0540 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.965+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c907c0540 0x556c907c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.966+0000 7f2d6bfff640 10 client.17355 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.966+0000 7f2d8bfff640 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c907c0540 0x556c907c2960 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8693e365141ebfc8 server_cookie=422acd3987650997 in_seq=0 out_seq=0 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d6bfff640 1 -- 172.21.15.29:0/3310034393 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c90739580 con 0x556c907c0540 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d6bfff640 10 client.17355 handle_command_reply: tid=0 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 client.17355 shutdown 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 msgr2=0x556c9074d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d78012fd0 tx=0x7f2d78009e50 comp rx=0 tx=0).stop 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 shutdown_connections 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556c907c0540 0x556c907c2960 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d6bfff640 0 client.17355 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c9074e330 0x556c9074e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c903764d0 0x556c9074d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 --2- 172.21.15.29:0/3310034393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c90355d80 0x556c9074d210 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 >> 172.21.15.29:0/3310034393 conn(0x556c9038b300 msgr2=0x556c9073edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 shutdown_connections 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 1 -- 172.21.15.29:0/3310034393 wait complete. 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.977+0000 7f2d99e4b740 20 client.17355 trim_cache size 0 max 16384 2024-09-21T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:57:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 msgr2=0x7f2d7c0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 0x7f2d7c0b8b70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d80004e80 tx=0x7f2d80005d50 comp rx=0 tx=0).stop 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 msgr2=0x7f2d8c198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c198f70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400eb20 tx=0x7f2d8400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 shutdown_connections 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2d7c0b6660 0x7f2d7c0b8b70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8c1071d0 0x7f2d8c198f70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c104610 0x7f2d8c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 --2- 172.21.15.29:0/1085475584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8c06b710 0x7f2d8c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 >> 172.21.15.29:0/1085475584 conn(0x7f2d8c0fdee0 msgr2=0x7f2d8c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 shutdown_connections 2024-09-21T01:57:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:29.978+0000 7f2d92e07640 1 -- 172.21.15.29:0/1085475584 wait complete. 2024-09-21T01:57:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:29 smithi066 ceph-mon[35001]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-09-21T01:57:30.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:29 smithi029 ceph-mon[31228]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-09-21T01:57:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:29 smithi190 ceph-mon[34368]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-09-21T01:57:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:31 smithi066 ceph-mon[35001]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-21T01:57:32.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:31 smithi029 ceph-mon[31228]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-21T01:57:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:31 smithi190 ceph-mon[34368]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-21T01:57:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 535.0 s, 2291.8 tps, lat 43.942 ms stddev 51.870 2024-09-21T01:57:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:33 smithi066 ceph-mon[35001]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-09-21T01:57:34.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:33 smithi029 ceph-mon[31228]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-09-21T01:57:34.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:33 smithi190 ceph-mon[34368]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-09-21T01:57:35.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:35 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:35.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:35 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:35.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:35 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:36 smithi066 ceph-mon[35001]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-09-21T01:57:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:57:36.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:36 smithi029 ceph-mon[31228]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-09-21T01:57:36.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:36.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:36.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:57:36.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:36 smithi190 ceph-mon[34368]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-09-21T01:57:36.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:36.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:36.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:57:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 540.0 s, 2374.0 tps, lat 42.153 ms stddev 47.987 2024-09-21T01:57:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:38 smithi066 ceph-mon[35001]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-09-21T01:57:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:38 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:57:38.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:38 smithi029 ceph-mon[31228]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-09-21T01:57:38.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:38 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:57:38.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:38 smithi190 ceph-mon[34368]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-09-21T01:57:38.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:38 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:57:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:40 smithi066 ceph-mon[35001]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 558 op/s 2024-09-21T01:57:40.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:40 smithi029 ceph-mon[31228]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 558 op/s 2024-09-21T01:57:40.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:40 smithi190 ceph-mon[34368]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 558 op/s 2024-09-21T01:57:41.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:57:41.527 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:42.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.097+0000 7f4e6588d640 1 -- 172.21.15.29:0/3960343716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e60104880 msgr2=0x7f4e60104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.097+0000 7f4e6588d640 1 --2- 172.21.15.29:0/3960343716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e60104880 0x7f4e60104c90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009900 tx=0x7f4e5002a410 comp rx=0 tx=0).stop 2024-09-21T01:57:42.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 -- 172.21.15.29:0/3960343716 shutdown_connections 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 --2- 172.21.15.29:0/3960343716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e60110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 --2- 172.21.15.29:0/3960343716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e601052b0 0x7f4e6010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 --2- 172.21.15.29:0/3960343716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e60104880 0x7f4e60104c90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 -- 172.21.15.29:0/3960343716 >> 172.21.15.29:0/3960343716 conn(0x7f4e60079730 msgr2=0x7f4e60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 -- 172.21.15.29:0/3960343716 shutdown_connections 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 -- 172.21.15.29:0/3960343716 wait complete. 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.098+0000 7f4e6588d640 1 Processor -- start 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 -- start start 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 0x7f4e6019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 0x7f4e6019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60113ac0 con 0x7f4e601052b0 2024-09-21T01:57:42.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 0x7f4e6019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60113940 con 0x7f4e60104880 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e6588d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60113c40 con 0x7f4e6010de30 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 0x7f4e6019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 0x7f4e6019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47946/0 (socket says 172.21.15.29:47946) 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57556/0 (socket says 172.21.15.29:57556) 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.099+0000 7f4e5effd640 1 -- 172.21.15.29:0/423430817 learned_addr learned my addr 172.21.15.29:0/423430817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e5f7fe640 1 -- 172.21.15.29:0/423430817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 msgr2=0x7f4e6019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e5f7fe640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 0x7f4e6019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e5f7fe640 1 -- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 msgr2=0x7f4e6019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e5f7fe640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 0x7f4e6019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e5f7fe640 1 -- 172.21.15.29:0/423430817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e601a0380 con 0x7f4e6010de30 2024-09-21T01:57:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.100+0000 7f4e57fff640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 0x7f4e6019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.102+0000 7f4e5f7fe640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009fd0 tx=0x7f4e5800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.102+0000 7f4e6488b640 1 -- 172.21.15.29:0/423430817 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800ce20 con 0x7f4e6010de30 2024-09-21T01:57:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.103+0000 7f4e6488b640 1 -- 172.21.15.29:0/423430817 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e58005020 con 0x7f4e6010de30 2024-09-21T01:57:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.103+0000 7f4e6488b640 1 -- 172.21.15.29:0/423430817 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58010690 con 0x7f4e6010de30 2024-09-21T01:57:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.103+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e601a0670 con 0x7f4e6010de30 2024-09-21T01:57:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.103+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601a0b60 con 0x7f4e6010de30 2024-09-21T01:57:42.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.106+0000 7f4e6be01740 1 Processor -- start 2024-09-21T01:57:42.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.106+0000 7f4e6488b640 1 -- 172.21.15.29:0/423430817 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4e580054d0 con 0x7f4e6010de30 2024-09-21T01:57:42.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.106+0000 7f4e6be01740 1 -- start start 2024-09-21T01:57:42.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530dc070 0x558253483650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 0x558253483b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5582533f9e50 con 0x5582530b31b0 2024-09-21T01:57:42.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5582533e9280 con 0x5582530dc070 2024-09-21T01:57:42.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e6be01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5582533ed090 con 0x5582530bb390 2024-09-21T01:57:42.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:57568/0 (socket says 172.21.15.29:57568) 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e57fff640 1 -- 172.21.15.29:0/3746429089 learned_addr learned my addr 172.21.15.29:0/3746429089 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530dc070 0x558253483650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.107+0000 7f4e57fff640 1 -- 172.21.15.29:0/3746429089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530dc070 msgr2=0x558253483650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530dc070 0x558253483650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e5effd640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 0x558253483b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e57fff640 1 -- 172.21.15.29:0/3746429089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 msgr2=0x558253483b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 0x558253483b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e57fff640 1 -- 172.21.15.29:0/3746429089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558253147650 con 0x5582530bb390 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e5effd640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 0x558253483b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e6488b640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 0x7f4e4c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e48013970 tx=0x7f4e4800b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0733feb5b09734b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e55ffb640 1 -- 172.21.15.29:0/3746429089 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48004030 con 0x5582530bb390 2024-09-21T01:57:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e5effd640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 0x7f4e4c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e55ffb640 1 -- 172.21.15.29:0/3746429089 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e4800e960 con 0x5582530bb390 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e55ffb640 1 -- 172.21.15.29:0/3746429089 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48003850 con 0x5582530bb390 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.108+0000 7f4e6488b640 1 -- 172.21.15.29:0/423430817 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4e580e79c0 con 0x7f4e6010de30 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 -- 172.21.15.29:0/3746429089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 msgr2=0x5582534840d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e5effd640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 0x7f4e4c0b8b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000c00 tx=0x7f4e50005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e48013970 tx=0x7f4e4800b4c0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 -- 172.21.15.29:0/3746429089 shutdown_connections 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x5582534840d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530b31b0 0x558253483b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3746429089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530dc070 0x558253483650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 -- 172.21.15.29:0/3746429089 >> 172.21.15.29:0/3746429089 conn(0x5582530d0460 msgr2=0x558253481540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 -- 172.21.15.29:0/3746429089 shutdown_connections 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.109+0000 7f4e6be01740 1 -- 172.21.15.29:0/3746429089 wait complete. 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.110+0000 7f4e6be01740 1 -- start start 2024-09-21T01:57:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.110+0000 7f4e6be01740 1 Processor -- start 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530bb390 0x55825348b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530dc070 0x55825348b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55825324a700 con 0x5582530bb390 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558253384110 con 0x5582530b31b0 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55825349a5c0 con 0x5582530dc070 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e6be01740 10 client.0 _ll_register_callbacks cb 0x55825339d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47974/0 (socket says 172.21.15.29:47974) 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 -- 172.21.15.29:0/3977716378 learned_addr learned my addr 172.21.15.29:0/3977716378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5f7fe640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530dc070 0x55825348b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 -- 172.21.15.29:0/3977716378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530dc070 msgr2=0x55825348b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530dc070 0x55825348b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 -- 172.21.15.29:0/3977716378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530bb390 msgr2=0x55825348b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530bb390 0x55825348b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.112+0000 7f4e5effd640 1 -- 172.21.15.29:0/3977716378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582533b1c40 con 0x5582530b31b0 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.113+0000 7f4e5effd640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a930 tx=0x7f4e5002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d79d4ce9d1d7674c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.113+0000 7f4e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.113+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004d020 con 0x5582530b31b0 2024-09-21T01:57:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.113+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4e50003c70 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.113+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e4800e960 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4e50035040 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 10 client.27146 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5582534a61f0 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50059070 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4e50049480 con 0x5582530b31b0 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 10 client.27146 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 20 client.27146 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825309a9f0 0x5582534a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 4 client.27146 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.114+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582533fbda0 con 0x55825309a9f0 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.115+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825309a9f0 0x5582534a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.115+0000 7f4e3effd640 10 client.27146 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.115+0000 7f4e57fff640 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825309a9f0 0x5582534a9750 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2accad291cb6bf server_cookie=4fb37de4e34feb8c in_seq=0 out_seq=0 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.116+0000 7f4e3effd640 1 -- 172.21.15.29:0/3977716378 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582533fbda0 con 0x55825309a9f0 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.116+0000 7f4e3effd640 10 client.27146 handle_command_reply: tid=0 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.116+0000 7f4e6be01740 1 client.27146 shutdown 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 msgr2=0x55825348ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a930 tx=0x7f4e5002a410 comp rx=0 tx=0).stop 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 shutdown_connections 2024-09-21T01:57:42.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825309a9f0 0x5582534a9750 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530dc070 0x55825348b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530bb390 0x55825348b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 --2- 172.21.15.29:0/3977716378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348ac40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 >> 172.21.15.29:0/3977716378 conn(0x5582530d0460 msgr2=0x55825347fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e3effd640 0 client.27146 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 shutdown_connections 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 1 -- 172.21.15.29:0/3977716378 wait complete. 2024-09-21T01:57:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.117+0000 7f4e6be01740 20 client.27146 trim_cache size 0 max 16384 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 Processor -- start 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 -- start start 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x558253480f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253481470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ad230 con 0x5582530dc070 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601aeda0 con 0x5582530b31b0 2024-09-21T01:57:42.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e6be01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ad900 con 0x5582530bb390 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47988/0 (socket says 172.21.15.29:47988) 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x558253480f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 -- 172.21.15.29:0/2335371365 learned_addr learned my addr 172.21.15.29:0/2335371365 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.150+0000 7f4e5f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253481470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 -- 172.21.15.29:0/2335371365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 msgr2=0x558253480f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x558253480f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 -- 172.21.15.29:0/2335371365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 msgr2=0x558253481470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253481470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 -- 172.21.15.29:0/2335371365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582534a37f0 con 0x5582530b31b0 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e5effd640 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031440 tx=0x7f4e50049cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f13c115579beac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.151+0000 7f4e557fa640 1 -- 172.21.15.29:0/2335371365 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004d040 con 0x5582530b31b0 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.152+0000 7f4e557fa640 1 -- 172.21.15.29:0/2335371365 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50062070 con 0x5582530b31b0 2024-09-21T01:57:42.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.152+0000 7f4e557fa640 1 -- 172.21.15.29:0/2335371365 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e500319f0 con 0x5582530b31b0 2024-09-21T01:57:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- 172.21.15.29:0/2335371365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 msgr2=0x55825348b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031440 tx=0x7f4e50049cc0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- 172.21.15.29:0/2335371365 shutdown_connections 2024-09-21T01:57:42.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253481470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530bb390 0x558253480f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 --2- 172.21.15.29:0/2335371365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530b31b0 0x55825348b100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- 172.21.15.29:0/2335371365 >> 172.21.15.29:0/2335371365 conn(0x5582530d0460 msgr2=0x55825348a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- 172.21.15.29:0/2335371365 shutdown_connections 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- 172.21.15.29:0/2335371365 wait complete. 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.153+0000 7f4e6be01740 1 -- start start 2024-09-21T01:57:42.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 Processor -- start 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 0x55825349beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 0x55825349c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253513110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ae6d0 con 0x5582530dc070 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ae000 con 0x5582530bb390 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e6be01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ab210 con 0x5582530b31b0 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.154+0000 7f4e5f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253513110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 0x55825349beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253513110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48424/0 (socket says 172.21.15.29:48424) 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5f7fe640 1 -- 172.21.15.29:0/348424104 learned_addr learned my addr 172.21.15.29:0/348424104 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:42.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e57fff640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 0x55825349c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e6be01740 10 client.0 _ll_register_callbacks cb 0x7f4e50040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 -- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 msgr2=0x55825349c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 0x55825349c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 -- 172.21.15.29:0/348424104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 msgr2=0x558253513110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253513110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 -- 172.21.15.29:0/348424104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558253513650 con 0x5582530b31b0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e57fff640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 0x55825349c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e5effd640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 0x55825349beb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e50004770 tx=0x7f4e50009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c0b497c81eb34c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.155+0000 7f4e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004d040 con 0x5582530b31b0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4e500314a0 con 0x5582530b31b0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e50062070 con 0x5582530b31b0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4e500316e0 con 0x5582530b31b0 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e6be01740 10 client.27118 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:42.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.156+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5582535147e0 con 0x5582530b31b0 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.157+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50059040 con 0x5582530b31b0 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4e5005da50 con 0x5582530b31b0 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 10 client.27118 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 10 client.27118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 20 client.27118 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:42.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825306fda0 0x558253516e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:42.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 4 client.27118 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:57:42.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.159+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4e50062070 con 0x55825306fda0 2024-09-21T01:57:42.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.160+0000 7f4e57fff640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825306fda0 0x558253516e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:42.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.160+0000 7f4e57fff640 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825306fda0 0x558253516e30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3843cb6af4b3583 server_cookie=c416e974e81972cd in_seq=0 out_seq=0 2024-09-21T01:57:42.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.160+0000 7f4e3effd640 10 client.27118 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e3effd640 1 -- 172.21.15.29:0/348424104 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x7f4e50062070 con 0x55825306fda0 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e3effd640 10 client.27118 handle_command_reply: tid=0 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 client.27118 shutdown 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 msgr2=0x55825349beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 0x55825349beb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e50004770 tx=0x7f4e50009260 comp rx=0 tx=0).stop 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 shutdown_connections 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55825306fda0 0x558253516e30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5582530dc070 0x558253513110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582530bb390 0x55825349c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e3effd640 0 client.27118 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 --2- 172.21.15.29:0/348424104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582530b31b0 0x55825349beb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.161+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 >> 172.21.15.29:0/348424104 conn(0x5582530d0460 msgr2=0x558253480d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.162+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 shutdown_connections 2024-09-21T01:57:42.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.162+0000 7f4e6be01740 1 -- 172.21.15.29:0/348424104 wait complete. 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.162+0000 7f4e6be01740 20 client.27118 trim_cache size 0 max 16384 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:57:42.164 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:57:42.165 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.167 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:57:42.169 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:57:42.170 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 msgr2=0x7f4e4c0b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 0x7f4e4c0b8b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000c00 tx=0x7f4e50005a20 comp rx=0 tx=0).stop 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 msgr2=0x7f4e6019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009fd0 tx=0x7f4e5800ef10 comp rx=0 tx=0).stop 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 shutdown_connections 2024-09-21T01:57:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4e4c0b6640 0x7f4e4c0b8b50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e6010de30 0x7f4e6019fd10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e601052b0 0x7f4e6019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 --2- 172.21.15.29:0/423430817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60104880 0x7f4e6019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 >> 172.21.15.29:0/423430817 conn(0x7f4e60079730 msgr2=0x7f4e60107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 shutdown_connections 2024-09-21T01:57:42.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:42.164+0000 7f4e6588d640 1 -- 172.21.15.29:0/423430817 wait complete. 2024-09-21T01:57:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:42 smithi066 ceph-mon[35001]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 725 op/s 2024-09-21T01:57:42.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:42 smithi029 ceph-mon[31228]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 725 op/s 2024-09-21T01:57:42.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:42 smithi190 ceph-mon[34368]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 725 op/s 2024-09-21T01:57:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 545.0 s, 2619.0 tps, lat 38.177 ms stddev 45.232 2024-09-21T01:57:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 -- 172.21.15.29:0/2788200696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010a3c0 msgr2=0x7f35e010c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 --2- 172.21.15.29:0/2788200696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010a3c0 0x7f35e010c7e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009860 tx=0x7f35c802a380 comp rx=0 tx=0).stop 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 -- 172.21.15.29:0/2788200696 shutdown_connections 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 --2- 172.21.15.29:0/2788200696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e010f9d0 0x7f35e011ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 --2- 172.21.15.29:0/2788200696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010cdb0 0x7f35e010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 --2- 172.21.15.29:0/2788200696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010a3c0 0x7f35e010c7e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.154+0000 7f35e6256640 1 -- 172.21.15.29:0/2788200696 >> 172.21.15.29:0/2788200696 conn(0x7f35e0070810 msgr2=0x7f35e0072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.156+0000 7f35e6256640 1 -- 172.21.15.29:0/2788200696 shutdown_connections 2024-09-21T01:57:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.160+0000 7f35e6256640 1 -- 172.21.15.29:0/2788200696 wait complete. 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.160+0000 7f35e6256640 1 Processor -- start 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 -- start start 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010a3c0 0x7f35e01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e010cdb0 0x7f35e01a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0114e70 con 0x7f35e010cdb0 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0114cf0 con 0x7f35e010f9d0 2024-09-21T01:57:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.161+0000 7f35e6256640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0114ff0 con 0x7f35e010a3c0 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010a3c0 0x7f35e01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48016/0 (socket says 172.21.15.29:48016) 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 -- 172.21.15.29:0/2005400435 learned_addr learned my addr 172.21.15.29:0/2005400435 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 -- 172.21.15.29:0/2005400435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010a3c0 msgr2=0x7f35e01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010a3c0 0x7f35e01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 -- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e010cdb0 msgr2=0x7f35e01a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e010cdb0 0x7f35e01a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.164+0000 7f35dffff640 1 -- 172.21.15.29:0/2005400435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01a1fc0 con 0x7f35e010f9d0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.165+0000 7f35dffff640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f35d801fd00 tx=0x7f35d801fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.165+0000 7f35dcff9640 1 -- 172.21.15.29:0/2005400435 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d801dbd0 con 0x7f35e010f9d0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.165+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01a22b0 con 0x7f35e010f9d0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.165+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e007b800 con 0x7f35e010f9d0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.165+0000 7f35dcff9640 1 -- 172.21.15.29:0/2005400435 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d8003e20 con 0x7f35e010f9d0 2024-09-21T01:57:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.166+0000 7f35dcff9640 1 -- 172.21.15.29:0/2005400435 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8028440 con 0x7f35e010f9d0 2024-09-21T01:57:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.172+0000 7f35ed1c8740 1 Processor -- start 2024-09-21T01:57:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 -- start start 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039a72f0 0x55e103d784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d79010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e103cea6f0 con 0x55e1039af4d0 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e103cd9b20 con 0x55e1039d01b0 2024-09-21T01:57:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e103cdd930 con 0x55e1039a72f0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35dcff9640 1 -- 172.21.15.29:0/2005400435 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f35d8003970 con 0x7f35e010f9d0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35deffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35deffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48024/0 (socket says 172.21.15.29:48024) 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.173+0000 7f35deffd640 1 -- 172.21.15.29:0/3164893319 learned_addr learned my addr 172.21.15.29:0/3164893319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 -- 172.21.15.29:0/3164893319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039a72f0 msgr2=0x55e103d784c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039a72f0 0x55e103d784c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 -- 172.21.15.29:0/3164893319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 msgr2=0x55e103d79010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d79010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 -- 172.21.15.29:0/3164893319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e103c9f530 con 0x55e1039d01b0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.174+0000 7f35deffd640 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35d000cae0 tx=0x7f35d0030300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e17799cb606ec17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.176+0000 7f35d5ffb640 1 -- 172.21.15.29:0/3164893319 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0030a70 con 0x55e1039d01b0 2024-09-21T01:57:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.176+0000 7f35d5ffb640 1 -- 172.21.15.29:0/3164893319 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d0030c10 con 0x55e1039d01b0 2024-09-21T01:57:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.177+0000 7f35dcff9640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 0x7f35cc0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.177+0000 7f35dcff9640 1 -- 172.21.15.29:0/2005400435 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f35d80f7f30 con 0x7f35e010f9d0 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 -- 172.21.15.29:0/3164893319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 msgr2=0x55e103d77e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35d000cae0 tx=0x7f35d0030300 comp rx=0 tx=0).stop 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 -- 172.21.15.29:0/3164893319 shutdown_connections 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d79010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039a72f0 0x55e103d784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/3164893319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d77e50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.178+0000 7f35ed1c8740 1 -- 172.21.15.29:0/3164893319 >> 172.21.15.29:0/3164893319 conn(0x55e1039c4300 msgr2=0x55e103d738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.179+0000 7f35ed1c8740 1 -- 172.21.15.29:0/3164893319 shutdown_connections 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.181+0000 7f35df7fe640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 0x7f35cc0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.181+0000 7f35ed1c8740 1 -- 172.21.15.29:0/3164893319 wait complete. 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.182+0000 7f35ed1c8740 1 -- start start 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.182+0000 7f35ed1c8740 1 Processor -- start 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039af4d0 0x55e103d81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48464/0 (socket says 172.21.15.29:48464) 2024-09-21T01:57:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35df7fe640 1 -- 172.21.15.29:0/939370696 learned_addr learned my addr 172.21.15.29:0/939370696 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e103c11ab0 con 0x55e1039a72f0 2024-09-21T01:57:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e103c7f730 con 0x55e1039af4d0 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e103d79b80 con 0x55e1039d01b0 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35deffd640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039af4d0 0x55e103d81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35ed1c8740 10 client.0 _ll_register_callbacks cb 0x55e103cd36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 -- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039af4d0 msgr2=0x55e103d81d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039af4d0 0x55e103d81d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 -- 172.21.15.29:0/939370696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 msgr2=0x55e103d81690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.183+0000 7f35dffff640 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e103a3b790 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d0030c10 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35dffff640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d828c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f35d80bdd70 tx=0x7f35d80bdaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20aaa2ccc2db05c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35beffd640 10 client.27124 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35beffd640 1 -- 172.21.15.29:0/939370696 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80bd4d0 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35beffd640 1 -- 172.21.15.29:0/939370696 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f35d80b99c0 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35beffd640 1 -- 172.21.15.29:0/939370696 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f35d80b9be0 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35ed1c8740 10 client.27124 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.184+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55e103ab4620 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35df7fe640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35beffd640 1 -- 172.21.15.29:0/939370696 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f35d80a91c0 con 0x55e1039d01b0 2024-09-21T01:57:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 10 client.27124 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 10 client.27124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35df7fe640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 0x7f35cc0b8b60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8030480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 20 client.27124 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e10398ed80 0x55e103d8ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 4 client.27124 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e103cec640 con 0x55e10398ed80 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.185+0000 7f35df7fe640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e10398ed80 0x55e103d8ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.186+0000 7f35df7fe640 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e10398ed80 0x55e103d8ec50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a1eb6d744a7dac server_cookie=e64e9a9fcbdc0efa in_seq=0 out_seq=0 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.186+0000 7f35beffd640 10 client.27124 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.186+0000 7f35beffd640 1 -- 172.21.15.29:0/939370696 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e103cec640 con 0x55e10398ed80 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35beffd640 10 client.27124 handle_command_reply: tid=0 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 client.27124 shutdown 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 msgr2=0x55e103d828c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d828c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f35d80bdd70 tx=0x7f35d80bdaf0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 shutdown_connections 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e10398ed80 0x55e103d8ec50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d828c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039af4d0 0x55e103d81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/939370696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d81690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 >> 172.21.15.29:0/939370696 conn(0x55e1039c4300 msgr2=0x55e103d738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.187+0000 7f35beffd640 0 client.27124 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.188+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 shutdown_connections 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.188+0000 7f35ed1c8740 1 -- 172.21.15.29:0/939370696 wait complete. 2024-09-21T01:57:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.188+0000 7f35ed1c8740 20 client.27124 trim_cache size 0 max 16384 2024-09-21T01:57:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.218+0000 7f35ed1c8740 1 Processor -- start 2024-09-21T01:57:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.218+0000 7f35ed1c8740 1 -- start start 2024-09-21T01:57:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d78db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d80890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48036/0 (socket says 172.21.15.29:48036) 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 -- 172.21.15.29:0/4033018700 learned_addr learned my addr 172.21.15.29:0/4033018700 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000e460 con 0x55e1039af4d0 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000ffd0 con 0x55e1039a72f0 2024-09-21T01:57:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000eb30 con 0x55e1039d01b0 2024-09-21T01:57:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35dffff640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d80890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35deffd640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d78db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 -- 172.21.15.29:0/4033018700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 msgr2=0x55e103d80890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d80890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.219+0000 7f35df7fe640 1 -- 172.21.15.29:0/4033018700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 msgr2=0x55e103d78db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35df7fe640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d78db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35df7fe640 1 -- 172.21.15.29:0/4033018700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e103d7f6d0 con 0x55e1039a72f0 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35dffff640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d80890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35df7fe640 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f35c802a970 tx=0x7f35c803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19517d6804e5a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35d57fa640 1 -- 172.21.15.29:0/4033018700 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8057070 con 0x55e1039a72f0 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.220+0000 7f35d57fa640 1 -- 172.21.15.29:0/4033018700 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c8038040 con 0x55e1039a72f0 2024-09-21T01:57:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.221+0000 7f35d57fa640 1 -- 172.21.15.29:0/4033018700 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8037040 con 0x55e1039a72f0 2024-09-21T01:57:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 msgr2=0x55e103d8ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f35c802a970 tx=0x7f35c803f000 comp rx=0 tx=0).stop 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 shutdown_connections 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039d01b0 0x55e103d80890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039af4d0 0x55e103d78db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/4033018700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039a72f0 0x55e103d8ec10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 >> 172.21.15.29:0/4033018700 conn(0x55e1039c4300 msgr2=0x55e103d6e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 shutdown_connections 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- 172.21.15.29:0/4033018700 wait complete. 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.222+0000 7f35ed1c8740 1 -- start start 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 Processor -- start 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039af4d0 0x55e103d87aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000f900 con 0x55e1039a72f0 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000f230 con 0x55e1039d01b0 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000c440 con 0x55e1039af4d0 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35ed1c8740 10 client.0 _ll_register_callbacks cb 0x7f35d80e1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48468/0 (socket says 172.21.15.29:48468) 2024-09-21T01:57:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.223+0000 7f35df7fe640 1 -- 172.21.15.29:0/1076420624 learned_addr learned my addr 172.21.15.29:0/1076420624 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35deffd640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039af4d0 0x55e103d87aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 -- 172.21.15.29:0/1076420624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039af4d0 msgr2=0x55e103d87aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039af4d0 0x55e103d87aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 -- 172.21.15.29:0/1076420624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 msgr2=0x55e103d88de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35dffff640 1 -- 172.21.15.29:0/1076420624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e103d885a0 con 0x55e1039d01b0 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.224+0000 7f35df7fe640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35dffff640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d87fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35d80bb2e0 tx=0x7f35d80043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabe42a9f71fa398 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80bde40 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f35d80bcd40 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c8038040 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f35d80be770 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 10 client.27170 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e103d874a0 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c8038040 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f35d8004a40 con 0x55e1039d01b0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 10 client.27170 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 10 client.27170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 20 client.27170 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1039641c0 0x55e103dfbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 4 client.27170 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.225+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e103ce80f0 con 0x55e1039641c0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.226+0000 7f35df7fe640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1039641c0 0x55e103dfbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.226+0000 7f35beffd640 10 client.27170 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.226+0000 7f35df7fe640 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1039641c0 0x55e103dfbc10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4af815209fc8e6 server_cookie=628061382add6f78 in_seq=0 out_seq=0 2024-09-21T01:57:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.238+0000 7f35beffd640 1 -- 172.21.15.29:0/1076420624 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e103ce80f0 con 0x55e1039641c0 2024-09-21T01:57:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.238+0000 7f35beffd640 10 client.27170 handle_command_reply: tid=0 2024-09-21T01:57:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.238+0000 7f35ed1c8740 1 client.27170 shutdown 2024-09-21T01:57:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 msgr2=0x55e103d87fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d87fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35d80bb2e0 tx=0x7f35d80043c0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 shutdown_connections 2024-09-21T01:57:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e1039641c0 0x55e103dfbc10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1039d01b0 0x55e103d87fe0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1039af4d0 0x55e103d87aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 --2- 172.21.15.29:0/1076420624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1039a72f0 0x55e103d88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.239+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 >> 172.21.15.29:0/1076420624 conn(0x55e1039c4300 msgr2=0x55e103d6e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.241+0000 7f35beffd640 0 client.27170 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 shutdown_connections 2024-09-21T01:57:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35ed1c8740 1 -- 172.21.15.29:0/1076420624 wait complete. 2024-09-21T01:57:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35ed1c8740 20 client.27170 trim_cache size 0 max 16384 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 msgr2=0x7f35cc0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 0x7f35cc0b8b60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8030480 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 msgr2=0x7f35e01a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.242+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f35d801fd00 tx=0x7f35d801fdf0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 shutdown_connections 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f35cc0b6650 0x7f35cc0b8b60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35e010f9d0 0x7f35e01a17a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e010cdb0 0x7f35e01a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 --2- 172.21.15.29:0/2005400435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e010a3c0 0x7f35e01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 >> 172.21.15.29:0/2005400435 conn(0x7f35e0070810 msgr2=0x7f35e010b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 shutdown_connections 2024-09-21T01:57:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:43.243+0000 7f35e6256640 1 -- 172.21.15.29:0/2005400435 wait complete. 2024-09-21T01:57:43.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:57:43.458 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:57:43.901 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:44 smithi066 ceph-mon[35001]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 627 op/s 2024-09-21T01:57:44.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:44 smithi029 ceph-mon[31228]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 627 op/s 2024-09-21T01:57:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:44 smithi190 ceph-mon[34368]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 627 op/s 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 -- 172.21.15.29:0/1225959441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781092f0 msgr2=0x7f2378109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 --2- 172.21.15.29:0/1225959441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781092f0 0x7f2378109700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f236c009900 tx=0x7f236c02a410 comp rx=0 tx=0).stop 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 -- 172.21.15.29:0/1225959441 shutdown_connections 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 --2- 172.21.15.29:0/1225959441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2378109c40 0x7f2378110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 --2- 172.21.15.29:0/1225959441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781092f0 0x7f2378109700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 --2- 172.21.15.29:0/1225959441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23781039b0 0x7f2378103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 -- 172.21.15.29:0/1225959441 >> 172.21.15.29:0/1225959441 conn(0x7f23780ff400 msgr2=0x7f2378101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 -- 172.21.15.29:0/1225959441 shutdown_connections 2024-09-21T01:57:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.212+0000 7f2377fff640 1 -- 172.21.15.29:0/1225959441 wait complete. 2024-09-21T01:57:45.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.214+0000 7f2377fff640 1 Processor -- start 2024-09-21T01:57:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.215+0000 7f2377fff640 1 -- start start 2024-09-21T01:57:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.215+0000 7f2377fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.215+0000 7f2377fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23781092f0 0x7f237819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.215+0000 7f2377fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2378109c40 0x7f237819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2377fff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2378113900 con 0x7f23781039b0 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2376ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2376ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48492/0 (socket says 172.21.15.29:48492) 2024-09-21T01:57:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2376ffd640 1 -- 172.21.15.29:0/1271264454 learned_addr learned my addr 172.21.15.29:0/1271264454 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2377fff640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2378113780 con 0x7f23781092f0 2024-09-21T01:57:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.216+0000 7f2377fff640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2378113a80 con 0x7f2378109c40 2024-09-21T01:57:45.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.218+0000 7f23777fe640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2378109c40 0x7f237819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:45.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.218+0000 7f23767fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23781092f0 0x7f237819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:45.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.220+0000 7f2376ffd640 1 -- 172.21.15.29:0/1271264454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2378109c40 msgr2=0x7f237819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:45.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.220+0000 7f2376ffd640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2378109c40 0x7f237819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.220+0000 7f2376ffd640 1 -- 172.21.15.29:0/1271264454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23781092f0 msgr2=0x7f237819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:45.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.220+0000 7f2376ffd640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23781092f0 0x7f237819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.220+0000 7f2376ffd640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23781a0220 con 0x7f23781039b0 2024-09-21T01:57:45.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.225+0000 7f2376ffd640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f236800e9f0 tx=0x7f236800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:45.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.229+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236800cd80 con 0x7f23781039b0 2024-09-21T01:57:45.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.229+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23680108e0 con 0x7f23781039b0 2024-09-21T01:57:45.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.229+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368018980 con 0x7f23781039b0 2024-09-21T01:57:45.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.230+0000 7f2377fff640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23781a0510 con 0x7f23781039b0 2024-09-21T01:57:45.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.230+0000 7f2377fff640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23781a6c70 con 0x7f23781039b0 2024-09-21T01:57:45.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.231+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2378103dc0 con 0x7f23781039b0 2024-09-21T01:57:45.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.243+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2368010430 con 0x7f23781039b0 2024-09-21T01:57:45.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.244+0000 7f237c8dc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 0x7f23600b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:45.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.244+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f23680e7c20 con 0x7f23781039b0 2024-09-21T01:57:45.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.245+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23680e80c0 con 0x7f23781039b0 2024-09-21T01:57:45.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.250+0000 7f23767fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 0x7f23600b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:45.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.274+0000 7f23767fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 0x7f23600b8bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f236c005ab0 tx=0x7f236c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:45.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.628+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f237819b3e0 con 0x7f23781039b0 2024-09-21T01:57:45.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.629+0000 7f237c8dc640 1 -- 172.21.15.29:0/1271264454 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11971 (secure 0 0 0) 0x7f23680ae4e0 con 0x7f23781039b0 2024-09-21T01:57:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:57:45.630 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"btime":"2024-09-21T01:57:37:007755+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:57:36.257121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T01:57:45.630 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-09-21T01:57:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 msgr2=0x7f23600b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 0x7f23600b8bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f236c005ab0 tx=0x7f236c005a20 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 msgr2=0x7f237819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f236800e9f0 tx=0x7f236800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 shutdown_connections 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f23600b66e0 0x7f23600b8bf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2378109c40 0x7f237819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23781092f0 0x7f237819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 --2- 172.21.15.29:0/1271264454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781039b0 0x7f237819a2b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.635+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 >> 172.21.15.29:0/1271264454 conn(0x7f23780ff400 msgr2=0x7f2378101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.638+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 shutdown_connections 2024-09-21T01:57:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:45.640+0000 7f235e7fc640 1 -- 172.21.15.29:0/1271264454 wait complete. 2024-09-21T01:57:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:46 smithi066 ceph-mon[35001]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.6 MiB/s wr, 631 op/s 2024-09-21T01:57:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:46 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1271264454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:57:46.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:46 smithi029 ceph-mon[31228]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.6 MiB/s wr, 631 op/s 2024-09-21T01:57:46.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:46 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1271264454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:57:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:46 smithi190 ceph-mon[34368]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.6 MiB/s wr, 631 op/s 2024-09-21T01:57:46.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:46 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1271264454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:57:46.506 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:57:46.883 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 -- 172.21.15.29:0/1295490223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85181083a0 msgr2=0x7f85181087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 --2- 172.21.15.29:0/1295490223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85181083a0 0x7f85181087b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8500009900 tx=0x7f850002a410 comp rx=0 tx=0).stop 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 -- 172.21.15.29:0/1295490223 shutdown_connections 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 --2- 172.21.15.29:0/1295490223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8518108cf0 0x7f851810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 --2- 172.21.15.29:0/1295490223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85181083a0 0x7f85181087b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 --2- 172.21.15.29:0/1295490223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518102a60 0x7f8518102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 -- 172.21.15.29:0/1295490223 >> 172.21.15.29:0/1295490223 conn(0x7f8518079730 msgr2=0x7f8518079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 -- 172.21.15.29:0/1295490223 shutdown_connections 2024-09-21T01:57:47.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.471+0000 7f851d73f640 1 -- 172.21.15.29:0/1295490223 wait complete. 2024-09-21T01:57:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.472+0000 7f851d73f640 1 Processor -- start 2024-09-21T01:57:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.472+0000 7f851d73f640 1 -- start start 2024-09-21T01:57:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.472+0000 7f851d73f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 0x7f851819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.472+0000 7f851d73f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.472+0000 7f851d73f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518108cf0 0x7f851819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f851d73f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8518112960 con 0x7f8518102a60 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f851d73f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85181127e0 con 0x7f85181083a0 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 0x7f851819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f851d73f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8518112ae0 con 0x7f8518108cf0 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48094/0 (socket says 172.21.15.29:48094) 2024-09-21T01:57:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 0x7f851819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48510/0 (socket says 172.21.15.29:48510) 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 -- 172.21.15.29:0/1229227886 learned_addr learned my addr 172.21.15.29:0/1229227886 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f85177fe640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518108cf0 0x7f851819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 -- 172.21.15.29:0/1229227886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518108cf0 msgr2=0x7f851819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518108cf0 0x7f851819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 -- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 msgr2=0x7f851819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 0x7f851819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 -- 172.21.15.29:0/1229227886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181a03b0 con 0x7f85181083a0 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.473+0000 7f850ffff640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8500032fd0 tx=0x7f8500031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.474+0000 7f8514ff9640 1 -- 172.21.15.29:0/1229227886 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850003f030 con 0x7f85181083a0 2024-09-21T01:57:47.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.474+0000 7f8514ff9640 1 -- 172.21.15.29:0/1229227886 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8500004490 con 0x7f85181083a0 2024-09-21T01:57:47.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.474+0000 7f8514ff9640 1 -- 172.21.15.29:0/1229227886 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500035070 con 0x7f85181083a0 2024-09-21T01:57:47.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.475+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181a0640 con 0x7f85181083a0 2024-09-21T01:57:47.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.475+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85180ff550 con 0x7f85181083a0 2024-09-21T01:57:47.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.480+0000 7f8523cb3740 1 Processor -- start 2024-09-21T01:57:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.480+0000 7f8514ff9640 1 -- 172.21.15.29:0/1229227886 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8500042030 con 0x7f85181083a0 2024-09-21T01:57:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.480+0000 7f8523cb3740 1 -- start start 2024-09-21T01:57:47.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f7282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f728d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f3f69eea0 con 0x563f3f381070 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f3f68e2d0 con 0x563f3f3581b0 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8523cb3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f3f6920e0 con 0x563f3f360390 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48100/0 (socket says 172.21.15.29:48100) 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f8516ffd640 1 -- 172.21.15.29:0/2419012441 learned_addr learned my addr 172.21.15.29:0/2419012441 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.481+0000 7f85177fe640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f7282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f850ffff640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f728d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8514ff9640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 0x7f85040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 -- 172.21.15.29:0/2419012441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 msgr2=0x563f3f728d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f728d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 -- 172.21.15.29:0/2419012441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 msgr2=0x563f3f7282c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f7282c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 -- 172.21.15.29:0/2419012441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3f3ec650 con 0x563f3f3581b0 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f850ffff640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f728d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f85177fe640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 0x7f85040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8514ff9640 1 -- 172.21.15.29:0/1229227886 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8500107740 con 0x7f85181083a0 2024-09-21T01:57:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f8516ffd640 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8508010da0 tx=0x7f8508009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45780e2803f3dd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f85177fe640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 0x7f85040b8a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8510005fd0 tx=0x7f8510004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f850d7fa640 1 -- 172.21.15.29:0/2419012441 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508020070 con 0x563f3f3581b0 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f850d7fa640 1 -- 172.21.15.29:0/2419012441 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850801baf0 con 0x563f3f3581b0 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.482+0000 7f850d7fa640 1 -- 172.21.15.29:0/2419012441 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850800bb30 con 0x563f3f3581b0 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.483+0000 7f8523cb3740 1 -- 172.21.15.29:0/2419012441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 msgr2=0x563f3f728800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.483+0000 7f8523cb3740 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8508010da0 tx=0x7f8508009900 comp rx=0 tx=0).stop 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.483+0000 7f8523cb3740 1 -- 172.21.15.29:0/2419012441 shutdown_connections 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f728d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f728800 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 --2- 172.21.15.29:0/2419012441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f7282c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 -- 172.21.15.29:0/2419012441 >> 172.21.15.29:0/2419012441 conn(0x563f3f375460 msgr2=0x563f3f7261b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 -- 172.21.15.29:0/2419012441 shutdown_connections 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.484+0000 7f8523cb3740 1 -- 172.21.15.29:0/2419012441 wait complete. 2024-09-21T01:57:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.485+0000 7f8523cb3740 1 -- start start 2024-09-21T01:57:47.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.485+0000 7f8523cb3740 1 Processor -- start 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.485+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.485+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f733f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f73fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8523cb3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f3f4ef700 con 0x563f3f381070 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8523cb3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f3f629160 con 0x563f3f3581b0 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8523cb3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f3f736aa0 con 0x563f3f360390 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8523cb3740 10 client.0 _ll_register_callbacks cb 0x563f3f6429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f850ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f733f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48116/0 (socket says 172.21.15.29:48116) 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 -- 172.21.15.29:0/1000146340 learned_addr learned my addr 172.21.15.29:0/1000146340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f85177fe640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f73fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 -- 172.21.15.29:0/1000146340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 msgr2=0x563f3f733f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f733f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 -- 172.21.15.29:0/1000146340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 msgr2=0x563f3f73fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f73fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.486+0000 7f8516ffd640 1 -- 172.21.15.29:0/1000146340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3f656c90 con 0x563f3f3581b0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.487+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8508013040 tx=0x7f8508005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be0cd0417e488791 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.487+0000 7f84fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.487+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508020040 con 0x563f3f3581b0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.487+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8508004020 con 0x563f3f3581b0 2024-09-21T01:57:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.487+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850801baf0 con 0x563f3f3581b0 2024-09-21T01:57:47.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.488+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8508004220 con 0x563f3f3581b0 2024-09-21T01:57:47.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.488+0000 7f8523cb3740 10 client.27188 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:47.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.488+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563f3f742840 con 0x563f3f3581b0 2024-09-21T01:57:47.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850801baf0 con 0x563f3f3581b0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f85080035f0 con 0x563f3f3581b0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 10 client.27188 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 10 client.27188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 20 client.27188 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f33f9f0 0x563f3f745ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 4 client.27188 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f3f6a0df0 con 0x563f3f33f9f0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.489+0000 7f850ffff640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f33f9f0 0x563f3f745ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.490+0000 7f850ffff640 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f33f9f0 0x563f3f745ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acb3c0a6e78de39d server_cookie=a05029c95c90a0bb in_seq=0 out_seq=0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.490+0000 7f84fe7fc640 10 client.27188 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1000146340 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f3f6a0df0 con 0x563f3f33f9f0 2024-09-21T01:57:47.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f84fe7fc640 10 client.27188 handle_command_reply: tid=0 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 client.27188 shutdown 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 msgr2=0x563f3f733a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8508013040 tx=0x7f8508005890 comp rx=0 tx=0).stop 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 shutdown_connections 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f33f9f0 0x563f3f745ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f381070 0x563f3f73fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f733f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f84fe7fc640 0 client.27188 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1000146340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f3581b0 0x563f3f733a30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.491+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 >> 172.21.15.29:0/1000146340 conn(0x563f3f375460 msgr2=0x563f3f72b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.492+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 shutdown_connections 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.492+0000 7f8523cb3740 1 -- 172.21.15.29:0/1000146340 wait complete. 2024-09-21T01:57:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.492+0000 7f8523cb3740 20 client.27188 trim_cache size 0 max 16384 2024-09-21T01:57:47.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 Processor -- start 2024-09-21T01:57:47.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 -- start start 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f3581b0 0x563f3f73fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f72b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f85181ad240 con 0x563f3f3581b0 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85181aedb0 con 0x563f3f33f9f0 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8523cb3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85181ad910 con 0x563f3f360390 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f850ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f3581b0 0x563f3f73fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48128/0 (socket says 172.21.15.29:48128) 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f8516ffd640 1 -- 172.21.15.29:0/1454781340 learned_addr learned my addr 172.21.15.29:0/1454781340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:47.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.509+0000 7f85177fe640 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f72b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 -- 172.21.15.29:0/1454781340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 msgr2=0x563f3f726200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 -- 172.21.15.29:0/1454781340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f3581b0 msgr2=0x563f3f73fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f3581b0 0x563f3f73fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 -- 172.21.15.29:0/1454781340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3f734cf0 con 0x563f3f360390 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f85177fe640 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f72b3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f851000f8e0 tx=0x7f851001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebf91448ab7c525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f850cff9640 1 -- 172.21.15.29:0/1454781340 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000fe60 con 0x563f3f360390 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f850cff9640 1 -- 172.21.15.29:0/1454781340 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8510024070 con 0x563f3f360390 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.510+0000 7f850cff9640 1 -- 172.21.15.29:0/1454781340 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510005560 con 0x563f3f360390 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.511+0000 7f8523cb3740 1 -- 172.21.15.29:0/1454781340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 msgr2=0x563f3f72b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.511+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f72b3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f851000f8e0 tx=0x7f851001a000 comp rx=0 tx=0).stop 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.513+0000 7f8523cb3740 1 -- 172.21.15.29:0/1454781340 shutdown_connections 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.513+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f360390 0x563f3f72b3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.513+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f3581b0 0x563f3f73fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.513+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1454781340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f33f9f0 0x563f3f726200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.513+0000 7f8523cb3740 1 -- 172.21.15.29:0/1454781340 >> 172.21.15.29:0/1454781340 conn(0x563f3f375460 msgr2=0x563f3f725af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.514+0000 7f8523cb3740 1 -- 172.21.15.29:0/1454781340 shutdown_connections 2024-09-21T01:57:47.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.514+0000 7f8523cb3740 1 -- 172.21.15.29:0/1454781340 wait complete. 2024-09-21T01:57:47.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.514+0000 7f8523cb3740 1 -- start start 2024-09-21T01:57:47.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8523cb3740 1 Processor -- start 2024-09-21T01:57:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 0x563f3f737ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8523cb3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 0x563f3f7381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 0x563f3f737ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 0x563f3f737ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48554/0 (socket says 172.21.15.29:48554) 2024-09-21T01:57:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.515+0000 7f8516ffd640 1 -- 172.21.15.29:0/1763520048 learned_addr learned my addr 172.21.15.29:0/1763520048 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f360390 0x563f3f738720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f85181ae6e0 con 0x563f3f33f9f0 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 0x563f3f7381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85181ae010 con 0x563f3f360390 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f85177fe640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f360390 0x563f3f738720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85181ab220 con 0x563f3f3581b0 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f8523cb3740 10 client.0 _ll_register_callbacks cb 0x7f850801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 -- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f360390 msgr2=0x563f3f738720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f360390 0x563f3f738720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 -- 172.21.15.29:0/1763520048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 msgr2=0x563f3f737ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 0x563f3f737ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.516+0000 7f850ffff640 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3f7af370 con 0x563f3f3581b0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f850ffff640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 0x563f3f7381e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85000caaf0 tx=0x7f8500009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0cb43125e54083 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500049040 con 0x563f3f3581b0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f850003ba30 con 0x563f3f3581b0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8510024070 con 0x563f3f3581b0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f85000cd390 con 0x563f3f3581b0 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f8523cb3740 10 client.27154 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:47.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563f3f7b05b0 con 0x563f3f3581b0 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8510024070 con 0x563f3f3581b0 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.517+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f85000d6e10 con 0x563f3f3581b0 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 10 client.27154 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 10 client.27154 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 20 client.27154 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f381070 0x563f3f7b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:47.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 4 client.27154 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f3f69c8a0 con 0x563f3f381070 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f381070 0x563f3f7b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f8516ffd640 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f381070 0x563f3f7b2d70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3a9731884e48a3e server_cookie=fab6552f464fc189 in_seq=0 out_seq=0 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.518+0000 7f84fe7fc640 10 client.27154 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f84fe7fc640 1 -- 172.21.15.29:0/1763520048 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f3f69c8a0 con 0x563f3f381070 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f84fe7fc640 10 client.27154 handle_command_reply: tid=0 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 client.27154 shutdown 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 msgr2=0x563f3f7381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 0x563f3f7381e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85000caaf0 tx=0x7f8500009260 comp rx=0 tx=0).stop 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 shutdown_connections 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x563f3f381070 0x563f3f7b2d70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f3f360390 0x563f3f738720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3f3581b0 0x563f3f7381e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f84fe7fc640 0 client.27154 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 --2- 172.21.15.29:0/1763520048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f3f33f9f0 0x563f3f737ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 >> 172.21.15.29:0/1763520048 conn(0x563f3f375460 msgr2=0x563f3f72b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 shutdown_connections 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.519+0000 7f8523cb3740 1 -- 172.21.15.29:0/1763520048 wait complete. 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f8523cb3740 20 client.27154 trim_cache size 0 max 16384 2024-09-21T01:57:47.523 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 msgr2=0x7f85040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 0x7f85040b8a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8510005fd0 tx=0x7f8510004ab0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 msgr2=0x7f851819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8500032fd0 tx=0x7f8500031a20 comp rx=0 tx=0).stop 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 shutdown_connections 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f85040b6510 0x7f85040b8a20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8518108cf0 0x7f851819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85181083a0 0x7f851819a9e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 --2- 172.21.15.29:0/1229227886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8518102a60 0x7f851819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:47.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.520+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 >> 172.21.15.29:0/1229227886 conn(0x7f8518079730 msgr2=0x7f851806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:47.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.521+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 shutdown_connections 2024-09-21T01:57:47.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:47.521+0000 7f851d73f640 1 -- 172.21.15.29:0/1229227886 wait complete. 2024-09-21T01:57:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 550.0 s, 1627.4 tps, lat 61.206 ms stddev 83.728 2024-09-21T01:57:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:48 smithi029 ceph-mon[31228]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-09-21T01:57:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:48 smithi066 ceph-mon[35001]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-09-21T01:57:48.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:48 smithi190 ceph-mon[34368]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-09-21T01:57:48.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:57:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:50.191 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:50 smithi029 ceph-mon[31228]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-09-21T01:57:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:50 smithi066 ceph-mon[35001]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-09-21T01:57:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.325+0000 7fa4e54e8640 1 -- 172.21.15.29:0/3725946331 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8030d70 con 0x7fa4e0111f40 2024-09-21T01:57:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.325+0000 7fa4e874e640 1 -- 172.21.15.29:0/3725946331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0111f40 msgr2=0x7fa4e010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.325+0000 7fa4e874e640 1 --2- 172.21.15.29:0/3725946331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0111f40 0x7fa4e010a3c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009be0 tx=0x7fa4d802a6c0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 -- 172.21.15.29:0/3725946331 shutdown_connections 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 --2- 172.21.15.29:0/3725946331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 0x7fa4e011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 --2- 172.21.15.29:0/3725946331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e010a9e0 0x7fa4e010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 --2- 172.21.15.29:0/3725946331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0111f40 0x7fa4e010a3c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 -- 172.21.15.29:0/3725946331 >> 172.21.15.29:0/3725946331 conn(0x7fa4e0070ec0 msgr2=0x7fa4e00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 -- 172.21.15.29:0/3725946331 shutdown_connections 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.326+0000 7fa4e874e640 1 -- 172.21.15.29:0/3725946331 wait complete. 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 Processor -- start 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 -- start start 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e0111f40 0x7fa4e01a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 0x7fa4e01aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0122900 con 0x7fa4e011cc30 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0122780 con 0x7fa4e010a9e0 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.327+0000 7fa4e874e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0122a80 con 0x7fa4e0111f40 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e6ceb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 0x7fa4e01aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48144/0 (socket says 172.21.15.29:48144) 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 -- 172.21.15.29:0/2967412015 learned_addr learned my addr 172.21.15.29:0/2967412015 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e0111f40 0x7fa4e01a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 -- 172.21.15.29:0/2967412015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e0111f40 msgr2=0x7fa4e01a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e0111f40 0x7fa4e01a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 -- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 msgr2=0x7fa4e01aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 0x7fa4e01aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 -- 172.21.15.29:0/2967412015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e01af2d0 con 0x7fa4e010a9e0 2024-09-21T01:57:50.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.328+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8004af0 tx=0x7fa4d8003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.329+0000 7fa4d77fe640 1 -- 172.21.15.29:0/2967412015 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8040020 con 0x7fa4e010a9e0 2024-09-21T01:57:50.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.329+0000 7fa4d77fe640 1 -- 172.21.15.29:0/2967412015 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d8003e10 con 0x7fa4e010a9e0 2024-09-21T01:57:50.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.329+0000 7fa4d77fe640 1 -- 172.21.15.29:0/2967412015 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8036040 con 0x7fa4e010a9e0 2024-09-21T01:57:50.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.329+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e01af560 con 0x7fa4e010a9e0 2024-09-21T01:57:50.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.329+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01b5d20 con 0x7fa4e010a9e0 2024-09-21T01:57:50.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.332+0000 7fa4d77fe640 1 -- 172.21.15.29:0/2967412015 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa4d8043020 con 0x7fa4e010a9e0 2024-09-21T01:57:50.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.333+0000 7fa4d77fe640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 0x7fa4cc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.333+0000 7fa4d77fe640 1 -- 172.21.15.29:0/2967412015 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4d8107e50 con 0x7fa4e010a9e0 2024-09-21T01:57:50.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.333+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 0x7fa4cc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.333+0000 7fa4eecc2740 1 Processor -- start 2024-09-21T01:57:50.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 0x7fa4cc0b8b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4e011c440 tx=0x7fa4d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 -- start start 2024-09-21T01:57:50.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fc4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560512f39ea0 con 0x560512c1c070 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560512f292d0 con 0x560512bf31b0 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560512f2d0e0 con 0x560512bfb390 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e5ce9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48586/0 (socket says 172.21.15.29:48586) 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e64ea640 1 -- 172.21.15.29:0/2394188766 learned_addr learned my addr 172.21.15.29:0/2394188766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.334+0000 7fa4e5ce9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48154/0 (socket says 172.21.15.29:48154) 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e6ceb640 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fc4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/2394188766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 msgr2=0x560512fc4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fc4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/2394188766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 msgr2=0x560512fc36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/2394188766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560512c87650 con 0x560512bf31b0 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa4e006c970 tx=0x7fa4d0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed3975a6f9fe097 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.335+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/2394188766 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000ee50 con 0x560512bf31b0 2024-09-21T01:57:50.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.336+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/2394188766 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d001c8c0 con 0x560512bf31b0 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.336+0000 7fa4d4ff9640 1 -- 172.21.15.29:0/2394188766 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d001d980 con 0x560512bf31b0 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.336+0000 7fa4eecc2740 1 -- 172.21.15.29:0/2394188766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 msgr2=0x560512fc3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.336+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa4e006c970 tx=0x7fa4d0014000 comp rx=0 tx=0).stop 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.337+0000 7fa4eecc2740 1 -- 172.21.15.29:0/2394188766 shutdown_connections 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.337+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fc4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.337+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fc3be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.337+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/2394188766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.337+0000 7fa4eecc2740 1 -- 172.21.15.29:0/2394188766 >> 172.21.15.29:0/2394188766 conn(0x560512c10460 msgr2=0x560512fc1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.338+0000 7fa4eecc2740 1 -- 172.21.15.29:0/2394188766 shutdown_connections 2024-09-21T01:57:50.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.339+0000 7fa4eecc2740 1 -- 172.21.15.29:0/2394188766 wait complete. 2024-09-21T01:57:50.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.340+0000 7fa4eecc2740 1 -- start start 2024-09-21T01:57:50.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.340+0000 7fa4eecc2740 1 Processor -- start 2024-09-21T01:57:50.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fcf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fcf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48158/0 (socket says 172.21.15.29:48158) 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4e64ea640 1 -- 172.21.15.29:0/3473555822 learned_addr learned my addr 172.21.15.29:0/3473555822 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fcf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560512d8a700 con 0x560512c1c070 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560512ec4160 con 0x560512bf31b0 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560512fd1f20 con 0x560512bfb390 2024-09-21T01:57:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4e6ceb640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fcf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.341+0000 7fa4eecc2740 10 client.0 _ll_register_callbacks cb 0x560512edd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 -- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 msgr2=0x560512fcf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fcf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 -- 172.21.15.29:0/3473555822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 msgr2=0x560512fcf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fcf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560512ef1c90 con 0x560512bf31b0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80099a0 tx=0x7fa4d80cdaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338417eafc6ce312 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80d2a70 con 0x560512bf31b0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4d80c98e0 con 0x560512bf31b0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.342+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d001c8c0 con 0x560512bf31b0 2024-09-21T01:57:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa4d80d79b0 con 0x560512bf31b0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 10 client.27206 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560512fddb60 con 0x560512bf31b0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80e9360 con 0x560512bf31b0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fa4d80f51b0 con 0x560512bf31b0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 10 client.27206 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 10 client.27206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 20 client.27206 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512bda9f0 0x560512fe1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 4 client.27206 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.343+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4d001c8c0 con 0x560512bda9f0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.344+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512bda9f0 0x560512fe1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.344+0000 7fa4c9ffb640 10 client.27206 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.344+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512bda9f0 0x560512fe1020 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fccd2cf8117cf96 server_cookie=6774a59b1877e22b in_seq=0 out_seq=0 2024-09-21T01:57:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/3473555822 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa4d001c8c0 con 0x560512bda9f0 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4c9ffb640 10 client.27206 handle_command_reply: tid=0 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 client.27206 shutdown 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 msgr2=0x560512fced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80099a0 tx=0x7fa4d80cdaa0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 shutdown_connections 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512c1c070 0x560512fcf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512bda9f0 0x560512fe1020 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4c9ffb640 0 client.27206 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fcf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/3473555822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bf31b0 0x560512fced40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 >> 172.21.15.29:0/3473555822 conn(0x560512c10460 msgr2=0x560512fc46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 shutdown_connections 2024-09-21T01:57:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.345+0000 7fa4eecc2740 1 -- 172.21.15.29:0/3473555822 wait complete. 2024-09-21T01:57:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.346+0000 7fa4eecc2740 20 client.27206 trim_cache size 0 max 16384 2024-09-21T01:57:50.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 Processor -- start 2024-09-21T01:57:50.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 -- start start 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bf31b0 0x560512fc0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bfb390 0x560512fc1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01be340 con 0x560512bfb390 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01bfeb0 con 0x560512bda9f0 2024-09-21T01:57:50.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01bea10 con 0x560512bf31b0 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4e64ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48162/0 (socket says 172.21.15.29:48162) 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4e64ea640 1 -- 172.21.15.29:0/627898554 learned_addr learned my addr 172.21.15.29:0/627898554 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4e6ceb640 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bfb390 0x560512fc1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.364+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bf31b0 0x560512fc0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 -- 172.21.15.29:0/627898554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bf31b0 msgr2=0x560512fc0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bf31b0 0x560512fc0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 -- 172.21.15.29:0/627898554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bfb390 msgr2=0x560512fc1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bfb390 0x560512fc1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 -- 172.21.15.29:0/627898554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560512fd1110 con 0x560512bda9f0 2024-09-21T01:57:50.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000f80 tx=0x7fa4d8009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8671b32f0b912e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4cbfff640 1 -- 172.21.15.29:0/627898554 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80cd400 con 0x560512bda9f0 2024-09-21T01:57:50.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4cbfff640 1 -- 172.21.15.29:0/627898554 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80b91b0 con 0x560512bda9f0 2024-09-21T01:57:50.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.365+0000 7fa4cbfff640 1 -- 172.21.15.29:0/627898554 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80b50e0 con 0x560512bda9f0 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 -- 172.21.15.29:0/627898554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 msgr2=0x560512fd0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000f80 tx=0x7fa4d8009280 comp rx=0 tx=0).stop 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 -- 172.21.15.29:0/627898554 shutdown_connections 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bfb390 0x560512fc1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bf31b0 0x560512fc0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/627898554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd0260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 -- 172.21.15.29:0/627898554 >> 172.21.15.29:0/627898554 conn(0x560512c10460 msgr2=0x560512fc5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 -- 172.21.15.29:0/627898554 shutdown_connections 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.366+0000 7fa4eecc2740 1 -- 172.21.15.29:0/627898554 wait complete. 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 -- start start 2024-09-21T01:57:50.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 Processor -- start 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fd3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4eecc2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01bf7e0 con 0x560512bf31b0 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4e5ce9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4e5ce9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48606/0 (socket says 172.21.15.29:48606) 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/956403639 learned_addr learned my addr 172.21.15.29:0/956403639 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4e6ceb640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fd3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.367+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01bf110 con 0x560512bda9f0 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01bc320 con 0x560512bfb390 2024-09-21T01:57:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4eecc2740 10 client.0 _ll_register_callbacks cb 0x7fa4d80cd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:50.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 msgr2=0x560512fd3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fd3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/956403639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 msgr2=0x560512fd79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:50.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e5ce9640 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560512fd3bd0 con 0x560512bf31b0 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.368+0000 7fa4e6ceb640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fd3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4e5ce9640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00058c0 tx=0x7fa4d00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459fb9585c04e973 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d003b020 con 0x560512bf31b0 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa4d0009040 con 0x560512bf31b0 2024-09-21T01:57:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d80b91b0 con 0x560512bf31b0 2024-09-21T01:57:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa4d0004020 con 0x560512bf31b0 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4eecc2740 10 client.17418 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.369+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56051304b950 con 0x560512bf31b0 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d00041c0 con 0x560512bf31b0 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fa4d0003680 con 0x560512bf31b0 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 10 client.17418 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 10 client.17418 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 20 client.17418 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512c1c070 0x56051304de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 4 client.17418 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa4d80b91b0 con 0x560512c1c070 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512c1c070 0x56051304de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4c9ffb640 10 client.17418 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.370+0000 7fa4e64ea640 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512c1c070 0x56051304de80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=320c12c6251975de server_cookie=d2e8a52a83c7e69d in_seq=0 out_seq=0 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4c9ffb640 1 -- 172.21.15.29:0/956403639 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa4d80b91b0 con 0x560512c1c070 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4c9ffb640 10 client.17418 handle_command_reply: tid=0 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 client.17418 shutdown 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 msgr2=0x560512fd2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00058c0 tx=0x7fa4d00056e0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 shutdown_connections 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x560512c1c070 0x56051304de80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560512bfb390 0x560512fd3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560512bf31b0 0x560512fd2ec0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4c9ffb640 0 client.17418 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 --2- 172.21.15.29:0/956403639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560512bda9f0 0x560512fd79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 >> 172.21.15.29:0/956403639 conn(0x560512c10460 msgr2=0x560512fdaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 shutdown_connections 2024-09-21T01:57:50.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 1 -- 172.21.15.29:0/956403639 wait complete. 2024-09-21T01:57:50.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.371+0000 7fa4eecc2740 20 client.17418 trim_cache size 0 max 16384 2024-09-21T01:57:50.378 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 msgr2=0x7fa4cc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 0x7fa4cc0b8b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4e011c440 tx=0x7fa4d0005f90 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 msgr2=0x7fa4e01a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8004af0 tx=0x7fa4d8003c20 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 shutdown_connections 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa4cc0b6610 0x7fa4cc0b8b20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e011cc30 0x7fa4e01aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e0111f40 0x7fa4e01a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 --2- 172.21.15.29:0/2967412015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e010a9e0 0x7fa4e01a9360 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 >> 172.21.15.29:0/2967412015 conn(0x7fa4e0070ec0 msgr2=0x7fa4e011e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 shutdown_connections 2024-09-21T01:57:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:50.372+0000 7fa4e874e640 1 -- 172.21.15.29:0/2967412015 wait complete. 2024-09-21T01:57:50.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:50 smithi190 ceph-mon[34368]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-09-21T01:57:51.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:57:51.479 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 -- 172.21.15.29:0/3226796207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4111f40 msgr2=0x7fede410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 --2- 172.21.15.29:0/3226796207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4111f40 0x7fede410a3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009900 tx=0x7fedd402a440 comp rx=0 tx=0).stop 2024-09-21T01:57:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 -- 172.21.15.29:0/3226796207 shutdown_connections 2024-09-21T01:57:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 --2- 172.21.15.29:0/3226796207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede411cc30 0x7fede411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 --2- 172.21.15.29:0/3226796207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede410a9e0 0x7fede410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 --2- 172.21.15.29:0/3226796207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4111f40 0x7fede410a3c0 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009900 tx=0x7fedd402a440 comp rx=0 tx=0).stop 2024-09-21T01:57:52.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.091+0000 7fede95a9640 1 -- 172.21.15.29:0/3226796207 >> 172.21.15.29:0/3226796207 conn(0x7fede4070ec0 msgr2=0x7fede40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.093+0000 7fede95a9640 1 -- 172.21.15.29:0/3226796207 shutdown_connections 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.093+0000 7fede95a9640 1 -- 172.21.15.29:0/3226796207 wait complete. 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 Processor -- start 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 -- start start 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede410a9e0 0x7fede411a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede4113d00 0x7fede4114200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fede4122900 con 0x7fede4113d00 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fede4122780 con 0x7fede410a9e0 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.094+0000 7fede95a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fede4122a80 con 0x7fede411cc30 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.095+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.095+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49468/0 (socket says 172.21.15.29:49468) 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.095+0000 7fedda7fc640 1 -- 172.21.15.29:0/2504149502 learned_addr learned my addr 172.21.15.29:0/2504149502 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.095+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede410a9e0 0x7fede411a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.095+0000 7fede37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede4113d00 0x7fede4114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.096+0000 7fedda7fc640 1 -- 172.21.15.29:0/2504149502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede410a9e0 msgr2=0x7fede411a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.096+0000 7fedda7fc640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede410a9e0 0x7fede411a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.096+0000 7fedda7fc640 1 -- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede4113d00 msgr2=0x7fede4114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.096+0000 7fedda7fc640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede4113d00 0x7fede4114200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.096+0000 7fedda7fc640 1 -- 172.21.15.29:0/2504149502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede41193c0 con 0x7fede411cc30 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.097+0000 7fedda7fc640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d740 tx=0x7fedcc00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.097+0000 7fede0ff9640 1 -- 172.21.15.29:0/2504149502 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc017070 con 0x7fede411cc30 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.098+0000 7fede0ff9640 1 -- 172.21.15.29:0/2504149502 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedcc00de50 con 0x7fede411cc30 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.098+0000 7fede0ff9640 1 -- 172.21.15.29:0/2504149502 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc003c20 con 0x7fede411cc30 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.098+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede41196b0 con 0x7fede411cc30 2024-09-21T01:57:52.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.098+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede4119ba0 con 0x7fede411cc30 2024-09-21T01:57:52.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.106+0000 7fede0ff9640 1 -- 172.21.15.29:0/2504149502 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fedcc010460 con 0x7fede411cc30 2024-09-21T01:57:52.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.107+0000 7fede0ff9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 0x7fedd00b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.107+0000 7fede0ff9640 1 -- 172.21.15.29:0/2504149502 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fedcc0e7b70 con 0x7fede411cc30 2024-09-21T01:57:52.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.108+0000 7fedefb1d740 1 Processor -- start 2024-09-21T01:57:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.108+0000 7fedefb1d740 1 -- start start 2024-09-21T01:57:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817f41b0 0x564681bc4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564681b3aea0 con 0x56468181d070 2024-09-21T01:57:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564681b2a2d0 con 0x5646817f41b0 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fedefb1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564681b2e0e0 con 0x5646817fc390 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 0x7fedd00b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fede37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fede2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fede37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49484/0 (socket says 172.21.15.29:49484) 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.109+0000 7fede37fe640 1 -- 172.21.15.29:0/1330193473 learned_addr learned my addr 172.21.15.29:0/1330193473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 0x7fedd00b8c20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009850 tx=0x7fedd4004d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede37fe640 1 -- 172.21.15.29:0/1330193473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817f41b0 msgr2=0x564681bc4be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:57:52.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede37fe640 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817f41b0 0x564681bc4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede37fe640 1 -- 172.21.15.29:0/1330193473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 msgr2=0x564681bc46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede37fe640 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.110+0000 7fede37fe640 1 -- 172.21.15.29:0/1330193473 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564681888650 con 0x5646817fc390 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.111+0000 7fede37fe640 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=596c7125574047aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.111+0000 7fedd97fa640 1 -- 172.21.15.29:0/1330193473 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc004d20 con 0x5646817fc390 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.111+0000 7fedd97fa640 1 -- 172.21.15.29:0/1330193473 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc010e50 con 0x5646817fc390 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 -- 172.21.15.29:0/1330193473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 msgr2=0x564681bc5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedd97fa640 1 -- 172.21.15.29:0/1330193473 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc004d20 con 0x5646817fc390 2024-09-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc010740 comp rx=0 tx=0).stop 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 -- 172.21.15.29:0/1330193473 shutdown_connections 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bc5120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817f41b0 0x564681bc4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 --2- 172.21.15.29:0/1330193473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.112+0000 7fedefb1d740 1 -- 172.21.15.29:0/1330193473 >> 172.21.15.29:0/1330193473 conn(0x564681811460 msgr2=0x564681bc2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.116+0000 7fedefb1d740 1 -- 172.21.15.29:0/1330193473 shutdown_connections 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.116+0000 7fedefb1d740 1 -- 172.21.15.29:0/1330193473 wait complete. 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.117+0000 7fedefb1d740 1 -- start start 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.117+0000 7fedefb1d740 1 Processor -- start 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bcfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56468198b700 con 0x56468181d070 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564681ac5160 con 0x5646817fc390 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564681bd2ed0 con 0x5646817db9f0 2024-09-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fedefb1d740 10 client.0 _ll_register_callbacks cb 0x564681ade9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fede2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bcfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fede37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fede37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49500/0 (socket says 172.21.15.29:49500) 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.118+0000 7fede37fe640 1 -- 172.21.15.29:0/460126940 learned_addr learned my addr 172.21.15.29:0/460126940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fedda7fc640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 -- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 msgr2=0x564681bd07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 -- 172.21.15.29:0/460126940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 msgr2=0x564681bd0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 -- 172.21.15.29:0/460126940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564681af2c90 con 0x5646817fc390 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fede2ffd640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bcfd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fedd4008000 tx=0x7fedd4046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af3188f2fdb830 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.119+0000 7fedca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4040860 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fedd4040a00 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc010e50 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fedd4040c00 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 10 client.27221 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564681bdec30 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc010e50 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fedd4031440 con 0x5646817fc390 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 10 client.27221 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 10 client.27221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.120+0000 7fedefb1d740 20 client.27221 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5646817b0da0 0x564681be25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedefb1d740 4 client.27221 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564681b3cdf0 con 0x5646817b0da0 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedda7fc640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5646817b0da0 0x564681be25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedda7fc640 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5646817b0da0 0x564681be25e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce6a0f8e70a1cee9 server_cookie=8ff2d1e15d996ee2 in_seq=0 out_seq=0 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.121+0000 7fedca7fc640 10 client.27221 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.122+0000 7fedca7fc640 1 -- 172.21.15.29:0/460126940 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564681b3cdf0 con 0x5646817b0da0 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.122+0000 7fedca7fc640 10 client.27221 handle_command_reply: tid=0 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.122+0000 7fedefb1d740 1 client.27221 shutdown 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 msgr2=0x564681bcfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bcfd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fedd4008000 tx=0x7fedd4046000 comp rx=0 tx=0).stop 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 shutdown_connections 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5646817b0da0 0x564681be25e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 --2- 172.21.15.29:0/460126940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bcfd20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 >> 172.21.15.29:0/460126940 conn(0x564681811460 msgr2=0x564681bc7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedca7fc640 0 client.27221 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 shutdown_connections 2024-09-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 1 -- 172.21.15.29:0/460126940 wait complete. 2024-09-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.123+0000 7fedefb1d740 20 client.27221 trim_cache size 0 max 16384 2024-09-21T01:57:52.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 Processor -- start 2024-09-21T01:57:52.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 -- start start 2024-09-21T01:57:52.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bc5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400e460 con 0x56468181d070 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400ffd0 con 0x5646817fc390 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fedefb1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400eb30 con 0x5646817db9f0 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fede37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fede37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40400/0 (socket says 172.21.15.29:40400) 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.144+0000 7fede37fe640 1 -- 172.21.15.29:0/415025913 learned_addr learned my addr 172.21.15.29:0/415025913 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede2ffd640 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fedda7fc640 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bc5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 -- 172.21.15.29:0/415025913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 msgr2=0x564681bd06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 -- 172.21.15.29:0/415025913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 msgr2=0x564681bc5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bc5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 -- 172.21.15.29:0/415025913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564681bc3350 con 0x56468181d070 2024-09-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.145+0000 7fede37fe640 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a651ebf1ff7a10e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.146+0000 7fedd8ff9640 1 -- 172.21.15.29:0/415025913 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc03c700 con 0x56468181d070 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.146+0000 7fedd8ff9640 1 -- 172.21.15.29:0/415025913 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc04b070 con 0x56468181d070 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.146+0000 7fedd8ff9640 1 -- 172.21.15.29:0/415025913 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc046bb0 con 0x56468181d070 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 -- 172.21.15.29:0/415025913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 msgr2=0x564681bc1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc009b30 comp rx=0 tx=0).stop 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 -- 172.21.15.29:0/415025913 shutdown_connections 2024-09-21T01:57:52.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bc1bc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817fc390 0x564681bc5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 --2- 172.21.15.29:0/415025913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817db9f0 0x564681bd06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 -- 172.21.15.29:0/415025913 >> 172.21.15.29:0/415025913 conn(0x564681811460 msgr2=0x564681bc56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 -- 172.21.15.29:0/415025913 shutdown_connections 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.147+0000 7fedefb1d740 1 -- 172.21.15.29:0/415025913 wait complete. 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 -- start start 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 Processor -- start 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 0x564681bd87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400f900 con 0x56468181d070 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400f230 con 0x5646817db9f0 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400c440 con 0x5646817fc390 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fede37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedefb1d740 10 client.0 _ll_register_callbacks cb 0x7fedd40094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fede2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 0x564681bd87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bd3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49508/0 (socket says 172.21.15.29:49508) 2024-09-21T01:57:52.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.148+0000 7fedda7fc640 1 -- 172.21.15.29:0/2577929964 learned_addr learned my addr 172.21.15.29:0/2577929964 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 -- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 msgr2=0x564681bd3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bd3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 -- 172.21.15.29:0/2577929964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 msgr2=0x564681bd4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 -- 172.21.15.29:0/2577929964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564681bd46d0 con 0x5646817db9f0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede37fe640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fede2ffd640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 0x564681bd87a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedd4031a50 tx=0x7fedd40054f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300ad0eeec0b66aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fedca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.149+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4031ce0 con 0x5646817db9f0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc04b070 con 0x5646817db9f0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fedd4040660 con 0x5646817db9f0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fedd40408a0 con 0x5646817db9f0 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedefb1d740 10 client.27227 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:52.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564681c4ccd0 con 0x5646817db9f0 2024-09-21T01:57:52.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.150+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc04b070 con 0x5646817db9f0 2024-09-21T01:57:52.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.151+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fedd404cc70 con 0x5646817db9f0 2024-09-21T01:57:52.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 10 client.27227 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:52.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 10 client.27227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:57:52.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 20 client.27227 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564681c4d420 0x564681c4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 4 client.27227 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564681b388a0 con 0x564681c4d420 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.152+0000 7fedda7fc640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564681c4d420 0x564681c4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.153+0000 7fedca7fc640 10 client.27227 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedda7fc640 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564681c4d420 0x564681c4f840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=298eddd79ed214b8 server_cookie=8151740f8ff1cb2c in_seq=0 out_seq=0 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedca7fc640 1 -- 172.21.15.29:0/2577929964 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564681b388a0 con 0x564681c4d420 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedca7fc640 10 client.27227 handle_command_reply: tid=0 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 client.27227 shutdown 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 msgr2=0x564681bd87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 0x564681bd87a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedd4031a50 tx=0x7fedd40054f0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 shutdown_connections 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x564681c4d420 0x564681c4f840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468181d070 0x564681bd4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedca7fc640 0 client.27227 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646817fc390 0x564681bd3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 --2- 172.21.15.29:0/2577929964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646817db9f0 0x564681bd87a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.154+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 >> 172.21.15.29:0/2577929964 conn(0x564681811460 msgr2=0x564681bd04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 shutdown_connections 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fedefb1d740 1 -- 172.21.15.29:0/2577929964 wait complete. 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fedefb1d740 20 client.27227 trim_cache size 0 max 16384 2024-09-21T01:57:52.158 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:52.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 msgr2=0x7fedd00b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 0x7fedd00b8c20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009850 tx=0x7fedd4004d00 comp rx=0 tx=0).stop 2024-09-21T01:57:52.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 msgr2=0x7fede4113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.155+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d740 tx=0x7fedcc00dc00 comp rx=0 tx=0).stop 2024-09-21T01:57:52.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 shutdown_connections 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fedd00b6710 0x7fedd00b8c20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fede4113d00 0x7fede4114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fede411cc30 0x7fede4113740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 --2- 172.21.15.29:0/2504149502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede410a9e0 0x7fede411a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 >> 172.21.15.29:0/2504149502 conn(0x7fede4070ec0 msgr2=0x7fede411d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 shutdown_connections 2024-09-21T01:57:52.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:52.156+0000 7fede95a9640 1 -- 172.21.15.29:0/2504149502 wait complete. 2024-09-21T01:57:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:52 smithi066 ceph-mon[35001]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 768 op/s 2024-09-21T01:57:52.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:52 smithi029 ceph-mon[31228]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 768 op/s 2024-09-21T01:57:52.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:52 smithi190 ceph-mon[34368]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 768 op/s 2024-09-21T01:57:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 555.0 s, 1845.8 tps, lat 54.131 ms stddev 64.761 2024-09-21T01:57:53.235 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:57:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 -- 172.21.15.29:0/1142511897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 msgr2=0x7f117810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 --2- 172.21.15.29:0/1142511897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f117810d850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f116c009860 tx=0x7f116c02a340 comp rx=0 tx=0).stop 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 -- 172.21.15.29:0/1142511897 shutdown_connections 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 --2- 172.21.15.29:0/1142511897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117810de10 0x7f1178110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 --2- 172.21.15.29:0/1142511897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f117810d850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 --2- 172.21.15.29:0/1142511897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11781049b0 0x7f1178104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 -- 172.21.15.29:0/1142511897 >> 172.21.15.29:0/1142511897 conn(0x7f1178079730 msgr2=0x7f1178079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 -- 172.21.15.29:0/1142511897 shutdown_connections 2024-09-21T01:57:54.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.258+0000 7f117e34d640 1 -- 172.21.15.29:0/1142511897 wait complete. 2024-09-21T01:57:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.259+0000 7f117e34d640 1 Processor -- start 2024-09-21T01:57:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.259+0000 7f117e34d640 1 -- start start 2024-09-21T01:57:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f1178102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f117810de10 0x7f1178102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1178113a70 con 0x7f117810de10 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11781138f0 con 0x7f1178105300 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f117e34d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1178113bf0 con 0x7f11781049b0 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f1177fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f1177fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49532/0 (socket says 172.21.15.29:49532) 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.260+0000 7f1177fff640 1 -- 172.21.15.29:0/1000446287 learned_addr learned my addr 172.21.15.29:0/1000446287 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f11777fe640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f1178102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f117c8ea640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f117810de10 0x7f1178102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 -- 172.21.15.29:0/1000446287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 msgr2=0x7f1178102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f1178102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 -- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f117810de10 msgr2=0x7f1178102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f117810de10 0x7f1178102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 -- 172.21.15.29:0/1000446287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178103490 con 0x7f11781049b0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.261+0000 7f1177fff640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f116800d770 tx=0x7f116800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.262+0000 7f11757fa640 1 -- 172.21.15.29:0/1000446287 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168017070 con 0x7f11781049b0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.262+0000 7f11757fa640 1 -- 172.21.15.29:0/1000446287 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168009c30 con 0x7f11781049b0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.262+0000 7f11757fa640 1 -- 172.21.15.29:0/1000446287 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168003c90 con 0x7f11781049b0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.262+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11781ace80 con 0x7f11781049b0 2024-09-21T01:57:54.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.262+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11781ad2e0 con 0x7f11781049b0 2024-09-21T01:57:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.266+0000 7f11848c1740 1 Processor -- start 2024-09-21T01:57:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.267+0000 7f11848c1740 1 -- start start 2024-09-21T01:57:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa119a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa119a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55efa1110ea0 con 0x55efa0df3070 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efa11002d0 con 0x55efa0dca1b0 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11848c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efa11040e0 con 0x55efa0dd2390 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49538/0 (socket says 172.21.15.29:49538) 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f11777fe640 1 -- 172.21.15.29:0/2404176654 learned_addr learned my addr 172.21.15.29:0/2404176654 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:54.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.268+0000 7f117c8ea640 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa119a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11757fa640 1 -- 172.21.15.29:0/1000446287 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1168010460 con 0x7f11781049b0 2024-09-21T01:57:54.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f1177fff640 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa119a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11777fe640 1 -- 172.21.15.29:0/2404176654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 msgr2=0x55efa119a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11777fe640 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa119a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11777fe640 1 -- 172.21.15.29:0/2404176654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 msgr2=0x55efa119a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11777fe640 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa119a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.269+0000 7f11777fe640 1 -- 172.21.15.29:0/2404176654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa0e5e650 con 0x55efa0dd2390 2024-09-21T01:57:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.270+0000 7f11777fe640 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f116c009830 tx=0x7f116c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b03db0808332825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.270+0000 7f115e7fc640 1 -- 172.21.15.29:0/2404176654 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c041020 con 0x55efa0dd2390 2024-09-21T01:57:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.270+0000 7f115e7fc640 1 -- 172.21.15.29:0/2404176654 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f116c030450 con 0x55efa0dd2390 2024-09-21T01:57:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.271+0000 7f11757fa640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 0x7f11600b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.271+0000 7f11757fa640 1 -- 172.21.15.29:0/1000446287 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f11680e6ae0 con 0x7f11781049b0 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.271+0000 7f117c8ea640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 0x7f11600b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.271+0000 7f115e7fc640 1 -- 172.21.15.29:0/2404176654 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c037040 con 0x55efa0dd2390 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 -- 172.21.15.29:0/2404176654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 msgr2=0x55efa119ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f116c009830 tx=0x7f116c005e00 comp rx=0 tx=0).stop 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 -- 172.21.15.29:0/2404176654 shutdown_connections 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa119ad40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa119a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 --2- 172.21.15.29:0/2404176654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa119a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 -- 172.21.15.29:0/2404176654 >> 172.21.15.29:0/2404176654 conn(0x55efa0de7460 msgr2=0x55efa11981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 -- 172.21.15.29:0/2404176654 shutdown_connections 2024-09-21T01:57:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f117c8ea640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 0x7f11600b8b20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11781047b0 tx=0x7f11700095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.272+0000 7f11848c1740 1 -- 172.21.15.29:0/2404176654 wait complete. 2024-09-21T01:57:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.273+0000 7f11848c1740 1 -- start start 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.273+0000 7f11848c1740 1 Processor -- start 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55efa0f61700 con 0x55efa0df3070 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efa109b160 con 0x55efa0dca1b0 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efa11a8a90 con 0x55efa0dd2390 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.274+0000 7f11848c1740 10 client.0 _ll_register_callbacks cb 0x55efa10b49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.275+0000 7f1177fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.275+0000 7f11777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.275+0000 7f117c8ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.275+0000 7f117c8ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40430/0 (socket says 172.21.15.29:40430) 2024-09-21T01:57:54.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.275+0000 7f117c8ea640 1 -- 172.21.15.29:0/3706159472 learned_addr learned my addr 172.21.15.29:0/3706159472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 -- 172.21.15.29:0/3706159472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 msgr2=0x55efa11a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 -- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 msgr2=0x55efa11a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 -- 172.21.15.29:0/3706159472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa10c8c90 con 0x55efa0df3070 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.276+0000 7f117c8ea640 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f117001cbc0 tx=0x7f1170004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f9f0dd58889ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.277+0000 7f11577fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.277+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117002c060 con 0x55efa0df3070 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.277+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1170016030 con 0x55efa0df3070 2024-09-21T01:57:54.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.277+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c030450 con 0x55efa0df3070 2024-09-21T01:57:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.278+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f117001cdd0 con 0x55efa0df3070 2024-09-21T01:57:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.278+0000 7f11848c1740 10 client.17445 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:54.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.278+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55efa11b47a0 con 0x55efa0df3070 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f116c030450 con 0x55efa0df3070 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f117000b040 con 0x55efa0df3070 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 10 client.17445 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 10 client.17445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 20 client.17445 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0db19f0 0x55efa11b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 4 client.17445 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:54.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.279+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efa1112df0 con 0x55efa0db19f0 2024-09-21T01:57:54.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.280+0000 7f1177fff640 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0db19f0 0x55efa11b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.280+0000 7f11577fe640 10 client.17445 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.280+0000 7f1177fff640 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0db19f0 0x55efa11b81b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfce63230e66a4b3 server_cookie=afc410b3233b2e39 in_seq=0 out_seq=0 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.281+0000 7f11577fe640 1 -- 172.21.15.29:0/3706159472 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efa1112df0 con 0x55efa0db19f0 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.281+0000 7f11577fe640 10 client.17445 handle_command_reply: tid=0 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.281+0000 7f11848c1740 1 client.17445 shutdown 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 msgr2=0x55efa11a6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f117001cbc0 tx=0x7f1170004a40 comp rx=0 tx=0).stop 2024-09-21T01:57:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 shutdown_connections 2024-09-21T01:57:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0db19f0 0x55efa11b81b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0df3070 0x55efa11a6340 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 --2- 172.21.15.29:0/3706159472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 >> 172.21.15.29:0/3706159472 conn(0x55efa0de7460 msgr2=0x55efa119d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11577fe640 0 client.17445 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 shutdown_connections 2024-09-21T01:57:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 1 -- 172.21.15.29:0/3706159472 wait complete. 2024-09-21T01:57:54.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.282+0000 7f11848c1740 20 client.17445 trim_cache size 0 max 16384 2024-09-21T01:57:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:54 smithi066 ceph-mon[35001]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-21T01:57:54.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.307+0000 7f11848c1740 1 Processor -- start 2024-09-21T01:57:54.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.307+0000 7f11848c1740 1 -- start start 2024-09-21T01:57:54.308 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:54 smithi029 ceph-mon[31228]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-21T01:57:54.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.308+0000 7f11848c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.308+0000 7f1177fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.308+0000 7f1177fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40436/0 (socket says 172.21.15.29:40436) 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f1177fff640 1 -- 172.21.15.29:0/3511298292 learned_addr learned my addr 172.21.15.29:0/3511298292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0df3070 0x55efa119b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f116400e3a0 con 0x55efa0dca1b0 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f116400ff10 con 0x55efa0df3070 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f116400ea70 con 0x55efa0dd2390 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.309+0000 7f11777fe640 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 -- 172.21.15.29:0/3511298292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 msgr2=0x55efa11a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 -- 172.21.15.29:0/3511298292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0df3070 msgr2=0x55efa119b2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:54.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0df3070 0x55efa119b2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 -- 172.21.15.29:0/3511298292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa0db1c20 con 0x55efa0dca1b0 2024-09-21T01:57:54.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.310+0000 7f1177fff640 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f116800d740 tx=0x7f11680035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6899850dcb7811f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.311+0000 7f115dffb640 1 -- 172.21.15.29:0/3511298292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168020040 con 0x55efa0dca1b0 2024-09-21T01:57:54.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.311+0000 7f115dffb640 1 -- 172.21.15.29:0/3511298292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168003760 con 0x55efa0dca1b0 2024-09-21T01:57:54.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.311+0000 7f115dffb640 1 -- 172.21.15.29:0/3511298292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680b6640 con 0x55efa0dca1b0 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 msgr2=0x55efa11a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f116800d740 tx=0x7f11680035f0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 shutdown_connections 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0df3070 0x55efa119b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0dd2390 0x55efa11a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.312+0000 7f11848c1740 1 --2- 172.21.15.29:0/3511298292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dca1b0 0x55efa11a2ab0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.313+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 >> 172.21.15.29:0/3511298292 conn(0x55efa0de7460 msgr2=0x55efa11a4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.313+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 shutdown_connections 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.313+0000 7f11848c1740 1 -- 172.21.15.29:0/3511298292 wait complete. 2024-09-21T01:57:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.313+0000 7f11848c1740 1 -- start start 2024-09-21T01:57:54.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.313+0000 7f11848c1740 1 Processor -- start 2024-09-21T01:57:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dd2390 0x55efa11a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0df3070 0x55efa11aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f116400f840 con 0x55efa0dd2390 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f116400f170 con 0x55efa0dca1b0 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f116400c380 con 0x55efa0df3070 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f11848c1740 10 client.0 _ll_register_callbacks cb 0x7f117001c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f1177fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f1177fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58840/0 (socket says 172.21.15.29:58840) 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.314+0000 7f1177fff640 1 -- 172.21.15.29:0/2492383641 learned_addr learned my addr 172.21.15.29:0/2492383641 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f11777fe640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dd2390 0x55efa11a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 -- 172.21.15.29:0/2492383641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0df3070 msgr2=0x55efa11aa030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0df3070 0x55efa11aa030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 -- 172.21.15.29:0/2492383641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dd2390 msgr2=0x55efa11a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dd2390 0x55efa11a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 -- 172.21.15.29:0/2492383641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa11aa7c0 con 0x55efa0dca1b0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f1177fff640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1168009610 tx=0x7f11680ac730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2886c395ad0ab695 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f11577fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168020040 con 0x55efa0dca1b0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f11680b61b0 con 0x55efa0dca1b0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.315+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1168003760 con 0x55efa0dca1b0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.316+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1168009de0 con 0x55efa0dca1b0 2024-09-21T01:57:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.316+0000 7f11848c1740 10 client.27242 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:54.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.316+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55efa1222b90 con 0x55efa0dca1b0 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168003760 con 0x55efa0dca1b0 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f1168093b60 con 0x55efa0dca1b0 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 10 client.27242 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 10 client.27242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 20 client.27242 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0d86da0 0x55efa1225300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 4 client.27242 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.317+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efa110e8a0 con 0x55efa0d86da0 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.318+0000 7f11777fe640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0d86da0 0x55efa1225300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.319+0000 7f11777fe640 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0d86da0 0x55efa1225300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7edab09d9399b20a server_cookie=900d634dfdb1fdda in_seq=0 out_seq=0 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.319+0000 7f11577fe640 10 client.27242 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.319+0000 7f11577fe640 1 -- 172.21.15.29:0/2492383641 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efa110e8a0 con 0x55efa0d86da0 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.320+0000 7f11577fe640 10 client.27242 handle_command_reply: tid=0 2024-09-21T01:57:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.320+0000 7f11848c1740 1 client.27242 shutdown 2024-09-21T01:57:54.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.320+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 msgr2=0x55efa11ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.320+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1168009610 tx=0x7f11680ac730 comp rx=0 tx=0).stop 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 shutdown_connections 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55efa0d86da0 0x55efa1225300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efa0df3070 0x55efa11aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11577fe640 0 client.27242 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55efa0dd2390 0x55efa11a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 --2- 172.21.15.29:0/2492383641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efa0dca1b0 0x55efa11ae5e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.321+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 >> 172.21.15.29:0/2492383641 conn(0x55efa0de7460 msgr2=0x55efa119b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.322+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 shutdown_connections 2024-09-21T01:57:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.322+0000 7f11848c1740 1 -- 172.21.15.29:0/2492383641 wait complete. 2024-09-21T01:57:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.322+0000 7f11848c1740 20 client.27242 trim_cache size 0 max 16384 2024-09-21T01:57:54.324 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.323+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 msgr2=0x7f11600b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.323+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 0x7f11600b8b20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11781047b0 tx=0x7f11700095c0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.323+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 msgr2=0x7f1178104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.323+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f116800d770 tx=0x7f116800dc30 comp rx=0 tx=0).stop 2024-09-21T01:57:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 shutdown_connections 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11600b6610 0x7f11600b8b20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f117810de10 0x7f1178102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1178105300 0x7f1178102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 --2- 172.21.15.29:0/1000446287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11781049b0 0x7f1178104120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 >> 172.21.15.29:0/1000446287 conn(0x7f1178079730 msgr2=0x7f11780ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 shutdown_connections 2024-09-21T01:57:54.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:54.324+0000 7f117e34d640 1 -- 172.21.15.29:0/1000446287 wait complete. 2024-09-21T01:57:54.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:54 smithi190 ceph-mon[34368]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-21T01:57:55.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:57:55.713 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:56.168 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:56 smithi029 ceph-mon[31228]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2024-09-21T01:57:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:56 smithi066 ceph-mon[35001]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2024-09-21T01:57:56.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.313+0000 7f5327945640 1 -- 172.21.15.29:0/1915048949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 msgr2=0x7f532011f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.313+0000 7f5327945640 1 --2- 172.21.15.29:0/1915048949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f532011f140 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f531c009900 tx=0x7f531c02a440 comp rx=0 tx=0).stop 2024-09-21T01:57:56.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.314+0000 7f5327945640 1 -- 172.21.15.29:0/1915048949 shutdown_connections 2024-09-21T01:57:56.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.314+0000 7f5327945640 1 --2- 172.21.15.29:0/1915048949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f532011f140 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.314+0000 7f5327945640 1 --2- 172.21.15.29:0/1915048949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f532010a9e0 0x7f532010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.314+0000 7f5327945640 1 --2- 172.21.15.29:0/1915048949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 0x7f532010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.314+0000 7f5327945640 1 -- 172.21.15.29:0/1915048949 >> 172.21.15.29:0/1915048949 conn(0x7f5320070ec0 msgr2=0x7f53200712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.318+0000 7f5327945640 1 -- 172.21.15.29:0/1915048949 shutdown_connections 2024-09-21T01:57:56.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.321+0000 7f5327945640 1 -- 172.21.15.29:0/1915048949 wait complete. 2024-09-21T01:57:56.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.322+0000 7f5327945640 1 Processor -- start 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.324+0000 7f5327945640 1 -- start start 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 0x7f532011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f53201136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5320122960 con 0x7f5320111f40 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53201227e0 con 0x7f532011cc30 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.325+0000 7f5327945640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5320122ae0 con 0x7f5320113be0 2024-09-21T01:57:56.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49562/0 (socket says 172.21.15.29:49562) 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 -- 172.21.15.29:0/3328449868 learned_addr learned my addr 172.21.15.29:0/3328449868 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f53256e1640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 0x7f532011a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5324ee0640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f53201136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 -- 172.21.15.29:0/3328449868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 msgr2=0x7f53201136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f53201136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 -- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 msgr2=0x7f532011a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 0x7f532011a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.326+0000 7f5325ee2640 1 -- 172.21.15.29:0/3328449868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5320119230 con 0x7f5320113be0 2024-09-21T01:57:56.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.327+0000 7f5325ee2640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f531c004770 tx=0x7f531c02a940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.335+0000 7f53167fc640 1 -- 172.21.15.29:0/3328449868 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c061070 con 0x7f5320113be0 2024-09-21T01:57:56.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.335+0000 7f53167fc640 1 -- 172.21.15.29:0/3328449868 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c05db20 con 0x7f5320113be0 2024-09-21T01:57:56.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.335+0000 7f53167fc640 1 -- 172.21.15.29:0/3328449868 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c035030 con 0x7f5320113be0 2024-09-21T01:57:56.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.335+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201194c0 con 0x7f5320113be0 2024-09-21T01:57:56.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.335+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201199a0 con 0x7f5320113be0 2024-09-21T01:57:56.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.339+0000 7f53167fc640 1 -- 172.21.15.29:0/3328449868 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f531c064070 con 0x7f5320113be0 2024-09-21T01:57:56.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.341+0000 7f532deb9740 1 Processor -- start 2024-09-21T01:57:56.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.341+0000 7f53167fc640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 0x7f530c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.342+0000 7f53167fc640 1 -- 172.21.15.29:0/3328449868 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f531c12a880 con 0x7f5320113be0 2024-09-21T01:57:56.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 -- start start 2024-09-21T01:57:56.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9407be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9408120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5559d937dea0 con 0x5559d90371b0 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5559d936d2d0 con 0x5559d903f390 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f532deb9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5559d93710e0 con 0x5559d9060070 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.345+0000 7f53256e1640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 0x7f530c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49564/0 (socket says 172.21.15.29:49564) 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 -- 172.21.15.29:0/4072555562 learned_addr learned my addr 172.21.15.29:0/4072555562 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f5325ee2640 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9408120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 0x7f530c0b8c20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5310011630 tx=0x7f5310011a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 -- 172.21.15.29:0/4072555562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 msgr2=0x5559d9408120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9408120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 -- 172.21.15.29:0/4072555562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 msgr2=0x5559d9407be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:56.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9407be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.346+0000 7f53256e1640 1 -- 172.21.15.29:0/4072555562 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d90cb650 con 0x5559d9060070 2024-09-21T01:57:56.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.347+0000 7f53256e1640 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5310004340 tx=0x7f531001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6934caabecfa4de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.350+0000 7f530b7fe640 1 -- 172.21.15.29:0/4072555562 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531001a040 con 0x5559d9060070 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.350+0000 7f530b7fe640 1 -- 172.21.15.29:0/4072555562 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310018070 con 0x5559d9060070 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 -- 172.21.15.29:0/4072555562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 msgr2=0x5559d94076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5310004340 tx=0x7f531001d000 comp rx=0 tx=0).stop 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 -- 172.21.15.29:0/4072555562 shutdown_connections 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9408120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9407be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 --2- 172.21.15.29:0/4072555562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94076a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.352+0000 7f532deb9740 1 -- 172.21.15.29:0/4072555562 >> 172.21.15.29:0/4072555562 conn(0x5559d9054460 msgr2=0x5559d9405590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.355+0000 7f532deb9740 1 -- 172.21.15.29:0/4072555562 shutdown_connections 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.355+0000 7f532deb9740 1 -- 172.21.15.29:0/4072555562 wait complete. 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.355+0000 7f532deb9740 1 -- start start 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.355+0000 7f532deb9740 1 Processor -- start 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d903f390 0x5559d9413120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9413660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5559d91ce700 con 0x5559d9060070 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5559d9308160 con 0x5559d90371b0 2024-09-21T01:57:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5559d9415e70 con 0x5559d903f390 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.356+0000 7f532deb9740 10 client.0 _ll_register_callbacks cb 0x5559d93219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58876/0 (socket says 172.21.15.29:58876) 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 -- 172.21.15.29:0/629099672 learned_addr learned my addr 172.21.15.29:0/629099672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 -- 172.21.15.29:0/629099672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d903f390 msgr2=0x5559d9413120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d903f390 0x5559d9413120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 -- 172.21.15.29:0/629099672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 msgr2=0x5559d9413660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9413660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.357+0000 7f53256e1640 1 -- 172.21.15.29:0/629099672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d9335c90 con 0x5559d90371b0 2024-09-21T01:57:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.358+0000 7f53256e1640 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5310006010 tx=0x7f5310009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f71ff570ca3292 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f5308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:56.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531001a050 con 0x5559d90371b0 2024-09-21T01:57:56.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5310018070 con 0x5559d90371b0 2024-09-21T01:57:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5310004ea0 con 0x5559d90371b0 2024-09-21T01:57:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f53100050a0 con 0x5559d90371b0 2024-09-21T01:57:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f532deb9740 10 client.27251 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.359+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559d9443a90 con 0x5559d90371b0 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.360+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310018070 con 0x5559d90371b0 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.361+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f5310003b60 con 0x5559d90371b0 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 10 client.27251 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 10 client.27251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 20 client.27251 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d901e9f0 0x5559d9447450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 4 client.27251 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:56.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559d937fdf0 con 0x5559d901e9f0 2024-09-21T01:57:56.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.362+0000 7f5324ee0640 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d901e9f0 0x5559d9447450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.364+0000 7f5324ee0640 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d901e9f0 0x5559d9447450 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b066c20092fd17c7 server_cookie=3b64da0fc6bdf8d9 in_seq=0 out_seq=0 2024-09-21T01:57:56.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.364+0000 7f5308ff9640 10 client.27251 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.365+0000 7f5308ff9640 1 -- 172.21.15.29:0/629099672 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559d937fdf0 con 0x5559d901e9f0 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.365+0000 7f5308ff9640 10 client.27251 handle_command_reply: tid=0 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.365+0000 7f532deb9740 1 client.27251 shutdown 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 msgr2=0x5559d9412be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5310006010 tx=0x7f5310009c50 comp rx=0 tx=0).stop 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 shutdown_connections 2024-09-21T01:57:56.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d901e9f0 0x5559d9447450 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9413660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d903f390 0x5559d9413120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 --2- 172.21.15.29:0/629099672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d90371b0 0x5559d9412be0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 >> 172.21.15.29:0/629099672 conn(0x5559d9054460 msgr2=0x5559d940a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.366+0000 7f5308ff9640 0 client.27251 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.367+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 shutdown_connections 2024-09-21T01:57:56.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.367+0000 7f532deb9740 1 -- 172.21.15.29:0/629099672 wait complete. 2024-09-21T01:57:56.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.367+0000 7f532deb9740 20 client.27251 trim_cache size 0 max 16384 2024-09-21T01:57:56.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.394+0000 7f532deb9740 1 Processor -- start 2024-09-21T01:57:56.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.394+0000 7f532deb9740 1 -- start start 2024-09-21T01:57:56.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d90371b0 0x5559d9409090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9404bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f530400e2d0 con 0x5559d9060070 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f530400fe40 con 0x5559d903f390 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f532deb9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f530400e9a0 con 0x5559d90371b0 2024-09-21T01:57:56.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f5324ee0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f5324ee0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58890/0 (socket says 172.21.15.29:58890) 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.395+0000 7f5324ee0640 1 -- 172.21.15.29:0/3433277468 learned_addr learned my addr 172.21.15.29:0/3433277468 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5325ee2640 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9404bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f53256e1640 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d90371b0 0x5559d9409090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5324ee0640 1 -- 172.21.15.29:0/3433277468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d90371b0 msgr2=0x5559d9409090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5324ee0640 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d90371b0 0x5559d9409090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5324ee0640 1 -- 172.21.15.29:0/3433277468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 msgr2=0x5559d9404bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5324ee0640 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9404bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.396+0000 7f5324ee0640 1 -- 172.21.15.29:0/3433277468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d9413ea0 con 0x5559d903f390 2024-09-21T01:57:56.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.397+0000 7f5324ee0640 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5318005fd0 tx=0x7f531800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8dd8ee2d032ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.397+0000 7f530affd640 1 -- 172.21.15.29:0/3433277468 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800fdd0 con 0x5559d903f390 2024-09-21T01:57:56.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.397+0000 7f530affd640 1 -- 172.21.15.29:0/3433277468 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531800e860 con 0x5559d903f390 2024-09-21T01:57:56.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.397+0000 7f530affd640 1 -- 172.21.15.29:0/3433277468 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800d7a0 con 0x5559d903f390 2024-09-21T01:57:56.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 -- 172.21.15.29:0/3433277468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 msgr2=0x5559d9414810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5318005fd0 tx=0x7f531800f7f0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 -- 172.21.15.29:0/3433277468 shutdown_connections 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d9060070 0x5559d9404bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9414810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 --2- 172.21.15.29:0/3433277468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d90371b0 0x5559d9409090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 -- 172.21.15.29:0/3433277468 >> 172.21.15.29:0/3433277468 conn(0x5559d9054460 msgr2=0x5559d9414100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.399+0000 7f532deb9740 1 -- 172.21.15.29:0/3433277468 shutdown_connections 2024-09-21T01:57:56.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.400+0000 7f532deb9740 1 -- 172.21.15.29:0/3433277468 wait complete. 2024-09-21T01:57:56.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.400+0000 7f532deb9740 1 -- start start 2024-09-21T01:57:56.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.401+0000 7f532deb9740 1 Processor -- start 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9416590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9416ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f530400f770 con 0x5559d90371b0 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f530400f0a0 con 0x5559d903f390 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f5325ee2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f5325ee2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49588/0 (socket says 172.21.15.29:49588) 2024-09-21T01:57:56.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f5325ee2640 1 -- 172.21.15.29:0/174868178 learned_addr learned my addr 172.21.15.29:0/174868178 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.402+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f530400c360 con 0x5559d9060070 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 -- 172.21.15.29:0/1979678696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a6590 msgr2=0x7f41680aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416dffb640 1 -- 172.21.15.29:0/1979678696 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170098420 con 0x7f41680a6590 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 --2- 172.21.15.29:0/1979678696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a6590 0x7f41680aaf00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4170066480 tx=0x7f417008f840 comp rx=0 tx=0).stop 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 -- 172.21.15.29:0/1979678696 shutdown_connections 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 --2- 172.21.15.29:0/1979678696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a6590 0x7f41680aaf00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 --2- 172.21.15.29:0/1979678696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a5bf0 0x7f41680a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 --2- 172.21.15.29:0/1979678696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41680a4d00 0x7f41680a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f416ffff640 1 -- 172.21.15.29:0/1979678696 >> 172.21.15.29:0/1979678696 conn(0x7f41680a0490 msgr2=0x7f41680a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f532deb9740 10 client.0 _ll_register_callbacks cb 0x7f531c003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:56.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5324ee0640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9416ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f53256e1640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9416590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5325ee2640 1 -- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 msgr2=0x5559d9416ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5325ee2640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9416ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5325ee2640 1 -- 172.21.15.29:0/174868178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 msgr2=0x5559d9416590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5325ee2640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9416590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.403+0000 7f5325ee2640 1 -- 172.21.15.29:0/174868178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d94172e0 con 0x5559d9060070 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f5325ee2640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f531c0ecf90 tx=0x7f531c0f01f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30b52450c62e262 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f5308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c06b040 con 0x5559d9060070 2024-09-21T01:57:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f531c05d670 con 0x5559d9060070 2024-09-21T01:57:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531800e860 con 0x5559d9060070 2024-09-21T01:57:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.404+0000 7f416ffff640 1 -- 172.21.15.29:0/1979678696 shutdown_connections 2024-09-21T01:57:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f416ffff640 1 -- 172.21.15.29:0/1979678696 wait complete. 2024-09-21T01:57:56.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f531c009530 con 0x5559d9060070 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f532deb9740 10 client.27220 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559d94b1360 con 0x5559d9060070 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c0f04d0 con 0x5559d9060070 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f531c0ec310 con 0x5559d9060070 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 10 client.27220 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 10 client.27220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.405+0000 7f416ffff640 1 Processor -- start 2024-09-21T01:57:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 20 client.27220 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d8ff3da0 0x5559d94b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 4 client.27220 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:57:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f531800e860 con 0x5559d8ff3da0 2024-09-21T01:57:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f416ffff640 1 -- start start 2024-09-21T01:57:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41680a5bf0 0x7f4168144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.406+0000 7f53256e1640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d8ff3da0 0x5559d94b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a6590 0x7f4168144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41680b8f60 con 0x7f41680a4d00 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41680b8de0 con 0x7f41680a6590 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41680b90e0 con 0x7f41680a5bf0 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40532/0 (socket says 172.21.15.29:40532) 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416effd640 1 -- 172.21.15.29:0/811040194 learned_addr learned my addr 172.21.15.29:0/811040194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f416f7fe640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a6590 0x7f4168144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 -- 172.21.15.29:0/811040194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41680a5bf0 msgr2=0x7f4168144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41680a5bf0 0x7f4168144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 -- 172.21.15.29:0/811040194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a6590 msgr2=0x7f4168144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a6590 0x7f4168144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 -- 172.21.15.29:0/811040194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4168145530 con 0x7f41680a4d00 2024-09-21T01:57:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f53256e1640 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d8ff3da0 0x5559d94b3af0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c421ee640ecf6e94 server_cookie=8a0822558fee0658 in_seq=0 out_seq=0 2024-09-21T01:57:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.407+0000 7f5308ff9640 10 client.27220 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f5308ff9640 1 -- 172.21.15.29:0/174868178 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f531800e860 con 0x5559d8ff3da0 2024-09-21T01:57:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f5308ff9640 10 client.27220 handle_command_reply: tid=0 2024-09-21T01:57:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f532deb9740 1 client.27220 shutdown 2024-09-21T01:57:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f416effd640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f416400d950 tx=0x7f416400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f4157fff640 1 -- 172.21.15.29:0/811040194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164017070 con 0x7f41680a4d00 2024-09-21T01:57:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f4157fff640 1 -- 172.21.15.29:0/811040194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164009c30 con 0x7f41680a4d00 2024-09-21T01:57:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f4157fff640 1 -- 172.21.15.29:0/811040194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400f620 con 0x7f41680a4d00 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 msgr2=0x5559d94afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.408+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f531c0ecf90 tx=0x7f531c0f01f0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 shutdown_connections 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5559d8ff3da0 0x5559d94b3af0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f5308ff9640 0 client.27220 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559d9060070 0x5559d94afe00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559d903f390 0x5559d9416ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 --2- 172.21.15.29:0/174868178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5559d90371b0 0x5559d9416590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 >> 172.21.15.29:0/174868178 conn(0x5559d9054460 msgr2=0x5559d941e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.415 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:57:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.409+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 shutdown_connections 2024-09-21T01:57:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.410+0000 7f532deb9740 1 -- 172.21.15.29:0/174868178 wait complete. 2024-09-21T01:57:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.410+0000 7f532deb9740 20 client.27220 trim_cache size 0 max 16384 2024-09-21T01:57:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.411+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4168145820 con 0x7f41680a4d00 2024-09-21T01:57:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.411+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416814bf10 con 0x7f41680a4d00 2024-09-21T01:57:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 msgr2=0x7f530c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 0x7f530c0b8c20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5310011630 tx=0x7f5310011a30 comp rx=0 tx=0).stop 2024-09-21T01:57:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 msgr2=0x7f5320114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f531c004770 tx=0x7f531c02a940 comp rx=0 tx=0).stop 2024-09-21T01:57:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 Processor -- start 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 shutdown_connections 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f530c0b6710 0x7f530c0b8c20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5320113be0 0x7f5320114010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532011cc30 0x7f53201136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 --2- 172.21.15.29:0/3328449868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5320111f40 0x7f532011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 >> 172.21.15.29:0/3328449868 conn(0x7f5320070ec0 msgr2=0x7f5320071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 -- start start 2024-09-21T01:57:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fea1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fea16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c2fe136f0 con 0x563c2fad84d0 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563c2fe02b20 con 0x563c2fad02f0 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.415+0000 7f417c63e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563c2fe06930 con 0x563c2faf91b0 2024-09-21T01:57:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.416+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 shutdown_connections 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.416+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.416+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40538/0 (socket says 172.21.15.29:40538) 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.416+0000 7f416e7fc640 1 -- 172.21.15.29:0/3100229318 learned_addr learned my addr 172.21.15.29:0/3100229318 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.416+0000 7f416f7fe640 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fea1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f5327945640 1 -- 172.21.15.29:0/3328449868 wait complete. 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 -- 172.21.15.29:0/3100229318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 msgr2=0x563c2fea1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fea1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 -- 172.21.15.29:0/3100229318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 msgr2=0x563c2fea16b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fea16b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 -- 172.21.15.29:0/3100229318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2fdc8530 con 0x563c2fad84d0 2024-09-21T01:57:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f4157fff640 1 -- 172.21.15.29:0/811040194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4164010460 con 0x7f41680a4d00 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.417+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c40 tx=0x7f416000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786fc106006042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.418+0000 7f41557fa640 1 -- 172.21.15.29:0/3100229318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160005020 con 0x563c2fad84d0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.418+0000 7f4157fff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 0x7f41580b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.418+0000 7f416f7fe640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 0x7f41580b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.418+0000 7f4157fff640 1 -- 172.21.15.29:0/811040194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f41640e6c20 con 0x7f41680a4d00 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.419+0000 7f41557fa640 1 -- 172.21.15.29:0/3100229318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41600047a0 con 0x563c2fad84d0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.419+0000 7f41557fa640 1 -- 172.21.15.29:0/3100229318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160010690 con 0x563c2fad84d0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f416f7fe640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 0x7f41580b8c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4170052860 tx=0x7f4170069620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 -- 172.21.15.29:0/3100229318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 msgr2=0x563c2fea2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c40 tx=0x7f416000e720 comp rx=0 tx=0).stop 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 -- 172.21.15.29:0/3100229318 shutdown_connections 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fea2200 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fea16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 --2- 172.21.15.29:0/3100229318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fea1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.420+0000 7f417c63e740 1 -- 172.21.15.29:0/3100229318 >> 172.21.15.29:0/3100229318 conn(0x563c2faed300 msgr2=0x563c2fe9cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.421+0000 7f417c63e740 1 -- 172.21.15.29:0/3100229318 shutdown_connections 2024-09-21T01:57:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.421+0000 7f417c63e740 1 -- 172.21.15.29:0/3100229318 wait complete. 2024-09-21T01:57:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.421+0000 7f417c63e740 1 -- start start 2024-09-21T01:57:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 Processor -- start 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2faf91b0 0x563c2febcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c2fd3aab0 con 0x563c2fad02f0 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49620/0 (socket says 172.21.15.29:49620) 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f416e7fc640 1 -- 172.21.15.29:0/369895895 learned_addr learned my addr 172.21.15.29:0/369895895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563c2fda8730 con 0x563c2faf91b0 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563c2fea2d70 con 0x563c2fad84d0 2024-09-21T01:57:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f416effd640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.422+0000 7f416f7fe640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2faf91b0 0x563c2febcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f417c63e740 10 client.0 _ll_register_callbacks cb 0x563c2fdfc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 -- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2faf91b0 msgr2=0x563c2febcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2faf91b0 0x563c2febcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 -- 172.21.15.29:0/369895895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad02f0 msgr2=0x563c2febb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2fb64790 con 0x563c2fad84d0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41600047a0 con 0x563c2fad84d0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f416e7fc640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c10 tx=0x7f4160005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605dddc3e39fd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f4152ff5640 10 client.27226 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f4152ff5640 1 -- 172.21.15.29:0/369895895 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160026020 con 0x563c2fad84d0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f4152ff5640 1 -- 172.21.15.29:0/369895895 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4160005200 con 0x563c2fad84d0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f4152ff5640 1 -- 172.21.15.29:0/369895895 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f416001d040 con 0x563c2fad84d0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.423+0000 7f417c63e740 10 client.27226 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=44}) -- 0x563c2fbdd620 con 0x563c2fad84d0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f4152ff5640 1 -- 172.21.15.29:0/369895895 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4160018430 con 0x563c2fad84d0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 10 client.27226 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 10 client.27226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 20 client.27226 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fab7d80 0x563c2fec8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 4 client.27226 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2fe15640 con 0x563c2fab7d80 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.424+0000 7f416effd640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fab7d80 0x563c2fec8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.425+0000 7f416effd640 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fab7d80 0x563c2fec8af0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea735e4c4e9de897 server_cookie=8de415363acb20b2 in_seq=0 out_seq=0 2024-09-21T01:57:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.425+0000 7f4152ff5640 10 client.27226 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f4152ff5640 1 -- 172.21.15.29:0/369895895 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c2fe15640 con 0x563c2fab7d80 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f4152ff5640 10 client.27226 handle_command_reply: tid=0 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 client.27226 shutdown 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 msgr2=0x563c2febbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c10 tx=0x7f4160005db0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 shutdown_connections 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fab7d80 0x563c2fec8af0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2faf91b0 0x563c2febcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbf90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 --2- 172.21.15.29:0/369895895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 >> 172.21.15.29:0/369895895 conn(0x563c2faed300 msgr2=0x563c2fe9b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f4152ff5640 0 client.27226 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.426+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 shutdown_connections 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.427+0000 7f417c63e740 1 -- 172.21.15.29:0/369895895 wait complete. 2024-09-21T01:57:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.427+0000 7f417c63e740 20 client.27226 trim_cache size 0 max 16384 2024-09-21T01:57:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 Processor -- start 2024-09-21T01:57:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 -- start start 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2faf91b0 0x563c2febaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f416814c310 con 0x563c2faf91b0 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f416814de80 con 0x563c2fad02f0 2024-09-21T01:57:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f417c63e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f416814c9e0 con 0x563c2fad84d0 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f416effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49628/0 (socket says 172.21.15.29:49628) 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f416e7fc640 1 -- 172.21.15.29:0/1258458381 learned_addr learned my addr 172.21.15.29:0/1258458381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.451+0000 7f416f7fe640 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2faf91b0 0x563c2febaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 -- 172.21.15.29:0/1258458381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 msgr2=0x563c2febbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 -- 172.21.15.29:0/1258458381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2faf91b0 msgr2=0x563c2febaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2faf91b0 0x563c2febaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 -- 172.21.15.29:0/1258458381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2febd060 con 0x563c2fad02f0 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416e7fc640 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f416effd640 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41640acd70 tx=0x7f4164010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd8fa6c74e043db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f4154ff9640 1 -- 172.21.15.29:0/1258458381 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640b18c0 con 0x563c2fad02f0 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f4154ff9640 1 -- 172.21.15.29:0/1258458381 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41640b0040 con 0x563c2fad02f0 2024-09-21T01:57:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.452+0000 7f4154ff9640 1 -- 172.21.15.29:0/1258458381 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640d00b0 con 0x563c2fad02f0 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.453+0000 7f417c63e740 1 -- 172.21.15.29:0/1258458381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 msgr2=0x563c2febb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41640acd70 tx=0x7f4164010cb0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 -- 172.21.15.29:0/1258458381 shutdown_connections 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2faf91b0 0x563c2febaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2fad84d0 0x563c2febbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 --2- 172.21.15.29:0/1258458381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2febb8d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.454+0000 7f417c63e740 1 -- 172.21.15.29:0/1258458381 >> 172.21.15.29:0/1258458381 conn(0x563c2faed300 msgr2=0x563c2fe9c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.455+0000 7f417c63e740 1 -- 172.21.15.29:0/1258458381 shutdown_connections 2024-09-21T01:57:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.455+0000 7f417c63e740 1 -- 172.21.15.29:0/1258458381 wait complete. 2024-09-21T01:57:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 -- start start 2024-09-21T01:57:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 Processor -- start 2024-09-21T01:57:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fec0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fec1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f416814d7b0 con 0x563c2fad84d0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f416814d0e0 con 0x563c2fad02f0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f417c63e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41680b3400 con 0x563c2faf91b0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f416effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fec0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.456+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40564/0 (socket says 172.21.15.29:40564) 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 -- 172.21.15.29:0/3966311432 learned_addr learned my addr 172.21.15.29:0/3966311432 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f417c63e740 10 client.0 _ll_register_callbacks cb 0x7f4160009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416f7fe640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fec1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 -- 172.21.15.29:0/3966311432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 msgr2=0x563c2fec1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fec1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 -- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 msgr2=0x563c2fec0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fec0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416e7fc640 1 -- 172.21.15.29:0/3966311432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2fec2500 con 0x563c2fad84d0 2024-09-21T01:57:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416f7fe640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fec1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.457+0000 7f416effd640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fec0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.458+0000 7f416e7fc640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c40 tx=0x7f4160009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f8f60809de961f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.458+0000 7f4152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.458+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160026040 con 0x563c2fad84d0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.458+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4160010630 con 0x563c2fad84d0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.459+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41640b0040 con 0x563c2fad84d0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.459+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4160044050 con 0x563c2fad84d0 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.459+0000 7f417c63e740 10 client.17481 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.459+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563c2fec3860 con 0x563c2fad84d0 2024-09-21T01:57:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160039030 con 0x563c2fad84d0 2024-09-21T01:57:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f416003e780 con 0x563c2fad84d0 2024-09-21T01:57:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 10 client.17481 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 20 client.17481 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fa8d1c0 0x563c2ff35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 4 client.17481 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f41640b0040 con 0x563c2fa8d1c0 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.460+0000 7f416effd640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fa8d1c0 0x563c2ff35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.461+0000 7f4152ff5640 10 client.17481 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.461+0000 7f416effd640 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fa8d1c0 0x563c2ff35900 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7367079b14d6854 server_cookie=ebe9cb5354a444c3 in_seq=0 out_seq=0 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.473+0000 7f4152ff5640 1 -- 172.21.15.29:0/3966311432 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f41640b0040 con 0x563c2fa8d1c0 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f4152ff5640 10 client.17481 handle_command_reply: tid=0 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 client.17481 shutdown 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 msgr2=0x563c2fec1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c40 tx=0x7f4160009720 comp rx=0 tx=0).stop 2024-09-21T01:57:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 shutdown_connections 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563c2fa8d1c0 0x563c2ff35900 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2faf91b0 0x563c2fec1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c2fad84d0 0x563c2fec1230 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 --2- 172.21.15.29:0/3966311432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563c2fad02f0 0x563c2fec0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 >> 172.21.15.29:0/3966311432 conn(0x563c2faed300 msgr2=0x563c2fea50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f4152ff5640 0 client.17481 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 shutdown_connections 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 1 -- 172.21.15.29:0/3966311432 wait complete. 2024-09-21T01:57:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.474+0000 7f417c63e740 20 client.17481 trim_cache size 0 max 16384 2024-09-21T01:57:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:57:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:57:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:57:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:57:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 msgr2=0x7f41580b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 0x7f41580b8c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4170052860 tx=0x7f4170069620 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 msgr2=0x7f4168143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f416400d950 tx=0x7f416400de10 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 shutdown_connections 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41580b6710 0x7f41580b8c20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41680a6590 0x7f4168144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41680a5bf0 0x7f4168144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.476+0000 7f416ffff640 1 --2- 172.21.15.29:0/811040194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41680a4d00 0x7f4168143af0 secure :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f416400d950 tx=0x7f416400de10 comp rx=0 tx=0).stop 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.477+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 >> 172.21.15.29:0/811040194 conn(0x7f41680a0490 msgr2=0x7f41680a6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.477+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 shutdown_connections 2024-09-21T01:57:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:57:56.477+0000 7f416ffff640 1 -- 172.21.15.29:0/811040194 wait complete. 2024-09-21T01:57:56.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:56 smithi190 ceph-mon[34368]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2024-09-21T01:57:57.242 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:57 smithi029 ceph-mon[31228]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 791 op/s 2024-09-21T01:57:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:57 smithi190 ceph-mon[34368]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 791 op/s 2024-09-21T01:57:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:57 smithi066 ceph-mon[35001]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 791 op/s 2024-09-21T01:57:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 560.0 s, 2641.2 tps, lat 38.171 ms stddev 47.027 2024-09-21T01:57:58.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:57:58.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:57:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:59.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.263+0000 7f5eb808a640 1 -- 172.21.15.29:0/2767354035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0109e10 msgr2=0x7f5eb0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.263+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2767354035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0109e10 0x7f5eb0110640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009ca0 tx=0x7f5ea802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.265+0000 7f5eb808a640 1 -- 172.21.15.29:0/2767354035 shutdown_connections 2024-09-21T01:57:59.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.265+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2767354035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0109e10 0x7f5eb0110640 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.265+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2767354035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb01094c0 0x7f5eb01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.265+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2767354035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb0103b80 0x7f5eb0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.265+0000 7f5eb808a640 1 -- 172.21.15.29:0/2767354035 >> 172.21.15.29:0/2767354035 conn(0x7f5eb00ff610 msgr2=0x7f5eb0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.266+0000 7f5eb808a640 1 -- 172.21.15.29:0/2767354035 shutdown_connections 2024-09-21T01:57:59.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.266+0000 7f5eb808a640 1 -- 172.21.15.29:0/2767354035 wait complete. 2024-09-21T01:57:59.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.267+0000 7f5eb808a640 1 Processor -- start 2024-09-21T01:57:59.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.267+0000 7f5eb808a640 1 -- start start 2024-09-21T01:57:59.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.267+0000 7f5eb808a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0103b80 0x7f5eb019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.267+0000 7f5eb808a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb01094c0 0x7f5eb019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb808a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb808a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0113ad0 con 0x7f5eb0103b80 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb808a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0113950 con 0x7f5eb0109e10 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb808a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0113c50 con 0x7f5eb01094c0 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58964/0 (socket says 172.21.15.29:58964) 2024-09-21T01:57:59.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 -- 172.21.15.29:0/2086851652 learned_addr learned my addr 172.21.15.29:0/2086851652 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0103b80 0x7f5eb019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 -- 172.21.15.29:0/2086851652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb01094c0 msgr2=0x7f5eb019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb01094c0 0x7f5eb019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 -- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0103b80 msgr2=0x7f5eb019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0103b80 0x7f5eb019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.268+0000 7f5eb6627640 1 -- 172.21.15.29:0/2086851652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb01a0390 con 0x7f5eb0109e10 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5eb6627640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8005cc0 tx=0x7f5ea8031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5ea6ffd640 1 -- 172.21.15.29:0/2086851652 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003b70 con 0x7f5eb0109e10 2024-09-21T01:57:59.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5ea6ffd640 1 -- 172.21.15.29:0/2086851652 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ea80318f0 con 0x7f5eb0109e10 2024-09-21T01:57:59.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01a0620 con 0x7f5eb0109e10 2024-09-21T01:57:59.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5ea6ffd640 1 -- 172.21.15.29:0/2086851652 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea802f950 con 0x7f5eb0109e10 2024-09-21T01:57:59.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.269+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01a6de0 con 0x7f5eb0109e10 2024-09-21T01:57:59.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.274+0000 7f5ebe5fe740 1 Processor -- start 2024-09-21T01:57:59.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.274+0000 7f5ea6ffd640 1 -- 172.21.15.29:0/2086851652 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5ea802fb30 con 0x7f5eb0109e10 2024-09-21T01:57:59.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 -- start start 2024-09-21T01:57:59.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e300430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 0x55bd2e300eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e276e60 con 0x55bd2df59350 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e266290 con 0x55bd2df30490 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e26a0a0 con 0x55bd2df38670 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5eb5625640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e300430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.275+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5625640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e300430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40578/0 (socket says 172.21.15.29:40578) 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58972/0 (socket says 172.21.15.29:58972) 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5625640 1 -- 172.21.15.29:0/2153197429 learned_addr learned my addr 172.21.15.29:0/2153197429 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5625640 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 0x55bd2e300eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 -- 172.21.15.29:0/2153197429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 msgr2=0x55bd2e300eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 0x55bd2e300eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 -- 172.21.15.29:0/2153197429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 msgr2=0x55bd2e300430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e300430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5e26640 1 -- 172.21.15.29:0/2153197429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2dfc4930 con 0x55bd2df30490 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5625640 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 0x55bd2e300eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5ea6ffd640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 0x7f5e9c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5eb5625640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 0x7f5e9c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.276+0000 7f5ea6ffd640 1 -- 172.21.15.29:0/2086851652 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5ea8108750 con 0x7f5eb0109e10 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.277+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009b70 tx=0x7f5eac0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ff193b07a8de61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.277+0000 7f5e9bfff640 1 -- 172.21.15.29:0/2153197429 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac009e60 con 0x55bd2df30490 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.277+0000 7f5e9bfff640 1 -- 172.21.15.29:0/2153197429 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eac012900 con 0x55bd2df30490 2024-09-21T01:57:59.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.277+0000 7f5e9bfff640 1 -- 172.21.15.29:0/2153197429 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00fae0 con 0x55bd2df30490 2024-09-21T01:57:59.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.278+0000 7f5eb5625640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 0x7f5e9c0b8b20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0012560 tx=0x7f5ea0014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.278+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/2153197429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 msgr2=0x55bd2e300970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.278+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009b70 tx=0x7f5eac0117b0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.279+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/2153197429 shutdown_connections 2024-09-21T01:57:59.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.279+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df38670 0x55bd2e300eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.279+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e300970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.279+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/2153197429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e300430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.279+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/2153197429 >> 172.21.15.29:0/2153197429 conn(0x55bd2df4d740 msgr2=0x55bd2e2fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.283+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/2153197429 shutdown_connections 2024-09-21T01:57:59.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.283+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/2153197429 wait complete. 2024-09-21T01:57:59.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.283+0000 7f5ebe5fe740 1 -- start start 2024-09-21T01:57:59.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.284+0000 7f5ebe5fe740 1 Processor -- start 2024-09-21T01:57:59.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e3103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e31c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e0c7890 con 0x55bd2df59350 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e200d40 con 0x55bd2df38670 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bd2e312f40 con 0x55bd2df30490 2024-09-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.285+0000 7f5ebe5fe740 10 client.0 _ll_register_callbacks cb 0x55bd2e264c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:59.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e3103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58984/0 (socket says 172.21.15.29:58984) 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 -- 172.21.15.29:0/1096478678 learned_addr learned my addr 172.21.15.29:0/1096478678 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 -- 172.21.15.29:0/1096478678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 msgr2=0x55bd2e3103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e3103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 -- 172.21.15.29:0/1096478678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 msgr2=0x55bd2e31c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e31c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.289+0000 7f5eb5625640 1 -- 172.21.15.29:0/1096478678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2e31c8b0 con 0x55bd2df38670 2024-09-21T01:57:59.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.290+0000 7f5eb5625640 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0003870 tx=0x7f5ea0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a7b6e0b5106851 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.290+0000 7f5e997fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:59.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.290+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000fc60 con 0x55bd2df38670 2024-09-21T01:57:59.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.290+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5ea000fe00 con 0x55bd2df38670 2024-09-21T01:57:59.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.293+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eac012900 con 0x55bd2df38670 2024-09-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.297+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5ea0021420 con 0x55bd2df38670 2024-09-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.301+0000 7f5ebe5fe740 10 client.27281 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.301+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bd2e31f220 con 0x55bd2df38670 2024-09-21T01:57:59.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.302+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ea0021a70 con 0x55bd2df38670 2024-09-21T01:57:59.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f5ea00336c0 con 0x55bd2df38670 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5ebe5fe740 10 client.27281 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5ebe5fe740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5ebe5fe740 20 client.27281 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2df17cd0 0x55bd2e3226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.303+0000 7f5ebe5fe740 4 client.27281 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.304+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5eac012900 con 0x55bd2df17cd0 2024-09-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.320+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2df17cd0 0x55bd2e3226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.330+0000 7f5e997fa640 10 client.27281 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:59.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.330+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2df17cd0 0x55bd2e3226a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f2261f09819049 server_cookie=3aceb7f047e1fca4 in_seq=0 out_seq=0 2024-09-21T01:57:59.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.331+0000 7f5e997fa640 1 -- 172.21.15.29:0/1096478678 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5eac012900 con 0x55bd2df17cd0 2024-09-21T01:57:59.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.331+0000 7f5e997fa640 10 client.27281 handle_command_reply: tid=0 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.332+0000 7f5ebe5fe740 1 client.27281 shutdown 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 msgr2=0x55bd2e3108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0003870 tx=0x7f5ea0009770 comp rx=0 tx=0).stop 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 shutdown_connections 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2df17cd0 0x55bd2e3226a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df59350 0x55bd2e31c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df38670 0x55bd2e3108f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/1096478678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e3103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.333+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 >> 172.21.15.29:0/1096478678 conn(0x55bd2df4d740 msgr2=0x55bd2e2fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.334+0000 7f5e997fa640 0 client.27281 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.334+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 shutdown_connections 2024-09-21T01:57:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.334+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/1096478678 wait complete. 2024-09-21T01:57:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.334+0000 7f5ebe5fe740 20 client.27281 trim_cache size 0 max 16384 2024-09-21T01:57:59.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.370+0000 7f5ebe5fe740 1 Processor -- start 2024-09-21T01:57:59.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.370+0000 7f5ebe5fe740 1 -- start start 2024-09-21T01:57:59.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.371+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.371+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e30b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.371+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.371+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df59350 0x55bd2e30ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.371+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58996/0 (socket says 172.21.15.29:58996) 2024-09-21T01:57:59.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.372+0000 7f5eb5e26640 1 -- 172.21.15.29:0/4278104147 learned_addr learned my addr 172.21.15.29:0/4278104147 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:59.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.372+0000 7f5eb5625640 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e30b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.372+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01acf80 con 0x55bd2df38670 2024-09-21T01:57:59.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.372+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01aeaf0 con 0x55bd2df30490 2024-09-21T01:57:59.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.372+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01ad650 con 0x55bd2df59350 2024-09-21T01:57:59.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb6627640 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df59350 0x55bd2e30ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 -- 172.21.15.29:0/4278104147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df59350 msgr2=0x55bd2e30ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df59350 0x55bd2e30ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 -- 172.21.15.29:0/4278104147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 msgr2=0x55bd2e30b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e30b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 -- 172.21.15.29:0/4278104147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2df17f00 con 0x55bd2df30490 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.373+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000c00 tx=0x7f5eac00f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9fba9d78ad57ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.374+0000 7f5e9b7fe640 1 -- 172.21.15.29:0/4278104147 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00f440 con 0x55bd2df30490 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.374+0000 7f5e9b7fe640 1 -- 172.21.15.29:0/4278104147 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eac009d80 con 0x55bd2df30490 2024-09-21T01:57:59.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.374+0000 7f5e9b7fe640 1 -- 172.21.15.29:0/4278104147 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00fa40 con 0x55bd2df30490 2024-09-21T01:57:59.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.375+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 msgr2=0x55bd2e30b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.375+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000c00 tx=0x7f5eac00f680 comp rx=0 tx=0).stop 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.376+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 shutdown_connections 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.376+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df59350 0x55bd2e30ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.376+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e30b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.376+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4278104147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df30490 0x55bd2e30b330 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.376+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 >> 172.21.15.29:0/4278104147 conn(0x55bd2df4d740 msgr2=0x55bd2e3103b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.377+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 shutdown_connections 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.377+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4278104147 wait complete. 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.377+0000 7f5ebe5fe740 1 -- start start 2024-09-21T01:57:59.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.377+0000 7f5ebe5fe740 1 Processor -- start 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.378+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e30a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.378+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e318270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5ebe5fe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 0x55bd2e3187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e30a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5eb5e26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e30a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49690/0 (socket says 172.21.15.29:49690) 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5ebe5fe740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01ae420 con 0x55bd2df38670 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01add50 con 0x55bd2df59350 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5eb5e26640 1 -- 172.21.15.29:0/4076557647 learned_addr learned my addr 172.21.15.29:0/4076557647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5eb6627640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 0x55bd2e3187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01ab010 con 0x55bd2df30490 2024-09-21T01:57:59.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.379+0000 7f5ebe5fe740 10 client.0 _ll_register_callbacks cb 0x7f5ea0009f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:57:59.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 -- 172.21.15.29:0/4076557647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 msgr2=0x55bd2e30a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e30a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 -- 172.21.15.29:0/4076557647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 msgr2=0x55bd2e318270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e318270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2e318cf0 con 0x55bd2df59350 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5eb6627640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 0x55bd2e3187b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea8003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62e6b584a5ccdae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5e997fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.380+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80cfdc0 con 0x55bd2df59350 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.381+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5ea8004b40 con 0x55bd2df59350 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.381+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eac009d80 con 0x55bd2df59350 2024-09-21T01:57:59.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.381+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5ea80ce3a0 con 0x55bd2df59350 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.381+0000 7f5ebe5fe740 10 client.27287 fetch_fsmap learned FSMap version 44 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.381+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bd2e30e000 con 0x55bd2df59350 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ea8036030 con 0x55bd2df59350 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f5ea80ca1e0 con 0x55bd2df59350 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 10 client.27287 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 10 client.27287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 20 client.27287 populate_metadata read hostname 'smithi029' 2024-09-21T01:57:59.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2deed080 0x55bd2e393d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:59.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 4 client.27287 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:57:59.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5eac009d80 con 0x55bd2deed080 2024-09-21T01:57:59.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.382+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2deed080 0x55bd2e393d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:59.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.384+0000 7f5eb5e26640 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2deed080 0x55bd2e393d70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e16586481df736 server_cookie=76e21a9c10fb4748 in_seq=0 out_seq=0 2024-09-21T01:57:59.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.384+0000 7f5e997fa640 10 client.27287 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:59.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.386+0000 7f5e997fa640 1 -- 172.21.15.29:0/4076557647 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5eac009d80 con 0x55bd2deed080 2024-09-21T01:57:59.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.386+0000 7f5e997fa640 10 client.27287 handle_command_reply: tid=0 2024-09-21T01:57:59.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 client.27287 shutdown 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 msgr2=0x55bd2e3187b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 0x55bd2e3187b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea8003f90 comp rx=0 tx=0).stop 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 shutdown_connections 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bd2deed080 0x55bd2e393d70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5e997fa640 0 client.27287 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd2df59350 0x55bd2e3187b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd2df38670 0x55bd2e318270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 --2- 172.21.15.29:0/4076557647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bd2df30490 0x55bd2e30a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.387+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 >> 172.21.15.29:0/4076557647 conn(0x55bd2df4d740 msgr2=0x55bd2e310390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.388+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 shutdown_connections 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.388+0000 7f5ebe5fe740 1 -- 172.21.15.29:0/4076557647 wait complete. 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.388+0000 7f5ebe5fe740 20 client.27287 trim_cache size 0 max 16384 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:57:59.389 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:57:59.390 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "f7547296-cdb5-408c-b734-10250cb2b816", 2024-09-21T01:57:59.390 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:57:59.390 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 msgr2=0x7f5e9c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 0x7f5e9c0b8b20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0012560 tx=0x7f5ea0014040 comp rx=0 tx=0).stop 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 msgr2=0x7f5eb019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8005cc0 tx=0x7f5ea8031700 comp rx=0 tx=0).stop 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 shutdown_connections 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5e9c0b6610 0x7f5e9c0b8b20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5eb0109e10 0x7f5eb019fc60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb01094c0 0x7f5eb019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 --2- 172.21.15.29:0/2086851652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5eb0103b80 0x7f5eb019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:59.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.389+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 >> 172.21.15.29:0/2086851652 conn(0x7f5eb00ff610 msgr2=0x7f5eb0106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:59.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.390+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 shutdown_connections 2024-09-21T01:57:59.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:57:59.390+0000 7f5eb808a640 1 -- 172.21.15.29:0/2086851652 wait complete. 2024-09-21T01:57:59.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:57:59 smithi190 ceph-mon[34368]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 604 op/s 2024-09-21T01:57:59.777 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:57:59 smithi029 ceph-mon[31228]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 604 op/s 2024-09-21T01:57:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:57:59 smithi066 ceph-mon[35001]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 604 op/s 2024-09-21T01:58:00.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:58:00.540 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.694 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:00.695 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:00.695 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:00.695 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.695 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.695 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:00 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:58:00.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:00 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:58:00.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:00.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:00.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:00.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:00 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.210+0000 7f420533f640 1 -- 172.21.15.29:0/458042122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 msgr2=0x7f420010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.210+0000 7f420533f640 1 --2- 172.21.15.29:0/458042122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420010adf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009be0 tx=0x7f41f802a710 comp rx=0 tx=0).stop 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.212+0000 7f420533f640 1 -- 172.21.15.29:0/458042122 shutdown_connections 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.212+0000 7f420533f640 1 --2- 172.21.15.29:0/458042122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f420011cc30 0x7f420011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.212+0000 7f420533f640 1 --2- 172.21.15.29:0/458042122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420010adf0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.212+0000 7f420533f640 1 --2- 172.21.15.29:0/458042122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4200111f40 0x7f420010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.212+0000 7f420533f640 1 -- 172.21.15.29:0/458042122 >> 172.21.15.29:0/458042122 conn(0x7f4200070ec0 msgr2=0x7f42000712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.214+0000 7f420533f640 1 -- 172.21.15.29:0/458042122 shutdown_connections 2024-09-21T01:58:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.218+0000 7f420533f640 1 -- 172.21.15.29:0/458042122 wait complete. 2024-09-21T01:58:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.220+0000 7f420533f640 1 Processor -- start 2024-09-21T01:58:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.220+0000 7f420533f640 1 -- start start 2024-09-21T01:58:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.221+0000 7f420533f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420011c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.221+0000 7f420533f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4200111f40 0x7f4200118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.221+0000 7f420533f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.222+0000 7f420533f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4200114c10 con 0x7f420010a9e0 2024-09-21T01:58:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.222+0000 7f420533f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4200114a90 con 0x7f4200111f40 2024-09-21T01:58:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.222+0000 7f420533f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4200114d90 con 0x7f420011cc30 2024-09-21T01:58:01.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.223+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.223+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49702/0 (socket says 172.21.15.29:49702) 2024-09-21T01:58:01.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.223+0000 7f41ff7fe640 1 -- 172.21.15.29:0/3948421171 learned_addr learned my addr 172.21.15.29:0/3948421171 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:01.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.223+0000 7f41feffd640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420011c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4200111f40 0x7f4200118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41ff7fe640 1 -- 172.21.15.29:0/3948421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4200111f40 msgr2=0x7f4200118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4200111f40 0x7f4200118730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41ff7fe640 1 -- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 msgr2=0x7f420011c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420011c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.224+0000 7f41ff7fe640 1 -- 172.21.15.29:0/3948421171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001194f0 con 0x7f420011cc30 2024-09-21T01:58:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.225+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41f001e920 tx=0x7f41f001ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.225+0000 7f41e7fff640 1 -- 172.21.15.29:0/3948421171 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0028070 con 0x7f420011cc30 2024-09-21T01:58:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.225+0000 7f41e7fff640 1 -- 172.21.15.29:0/3948421171 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f0005020 con 0x7f420011cc30 2024-09-21T01:58:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.225+0000 7f41e7fff640 1 -- 172.21.15.29:0/3948421171 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004d50 con 0x7f420011cc30 2024-09-21T01:58:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.227+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001ace90 con 0x7f420011cc30 2024-09-21T01:58:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.227+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001ad290 con 0x7f420011cc30 2024-09-21T01:58:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.237+0000 7f420b8b3740 1 Processor -- start 2024-09-21T01:58:01.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.242+0000 7f41e7fff640 1 -- 172.21.15.29:0/3948421171 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f41f001ab00 con 0x7f420011cc30 2024-09-21T01:58:01.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.244+0000 7f420b8b3740 1 -- start start 2024-09-21T01:58:01.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.245+0000 7f41e7fff640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 0x7f41e80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.245+0000 7f41e7fff640 1 -- 172.21.15.29:0/3948421171 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f41f00f7f30 con 0x7f420011cc30 2024-09-21T01:58:01.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.246+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44941270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.246+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445711b0 0x559b449417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f420b8b3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559b448b7e50 con 0x559b445711b0 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f420b8b3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559b448a7280 con 0x559b44579390 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f420b8b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b448ab090 con 0x559b4459a070 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f41feffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44941270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59032/0 (socket says 172.21.15.29:59032) 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f41ff7fe640 1 -- 172.21.15.29:0/1847534016 learned_addr learned my addr 172.21.15.29:0/1847534016 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.247+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445711b0 0x559b449417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.248+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 0x7f41e80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.248+0000 7f41ff7fe640 1 -- 172.21.15.29:0/1847534016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 msgr2=0x559b44941270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.248+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44941270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.248+0000 7f41ff7fe640 1 -- 172.21.15.29:0/1847534016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445711b0 msgr2=0x559b449417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.248+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445711b0 0x559b449417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.249+0000 7f41ff7fe640 1 -- 172.21.15.29:0/1847534016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b44605650 con 0x559b44579390 2024-09-21T01:58:01.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.249+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41f00bb340 tx=0x7f41f00bd840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea837804906ec17d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.250+0000 7f41e57fa640 1 -- 172.21.15.29:0/1847534016 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00be7d0 con 0x559b44579390 2024-09-21T01:58:01.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.250+0000 7f41e57fa640 1 -- 172.21.15.29:0/1847534016 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f00c2bb0 con 0x559b44579390 2024-09-21T01:58:01.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.250+0000 7f41e57fa640 1 -- 172.21.15.29:0/1847534016 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00b9c80 con 0x559b44579390 2024-09-21T01:58:01.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 -- 172.21.15.29:0/1847534016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 msgr2=0x559b44941cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41f00bb340 tx=0x7f41f00bd840 comp rx=0 tx=0).stop 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 0x7f41e80b8ae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f80 tx=0x7f41f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 -- 172.21.15.29:0/1847534016 shutdown_connections 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b44941cf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445711b0 0x559b449417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1847534016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44941270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 -- 172.21.15.29:0/1847534016 >> 172.21.15.29:0/1847534016 conn(0x559b4458e460 msgr2=0x559b4493f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 -- 172.21.15.29:0/1847534016 shutdown_connections 2024-09-21T01:58:01.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.251+0000 7f420b8b3740 1 -- 172.21.15.29:0/1847534016 wait complete. 2024-09-21T01:58:01.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.252+0000 7f420b8b3740 1 -- start start 2024-09-21T01:58:01.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.252+0000 7f420b8b3740 1 Processor -- start 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b4494ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b44579390 0x559b4494cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44958d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f41feffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b4494ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f41feffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b4494ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59044/0 (socket says 172.21.15.29:59044) 2024-09-21T01:58:01.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f41feffd640 1 -- 172.21.15.29:0/341767113 learned_addr learned my addr 172.21.15.29:0/341767113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:01.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.253+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559b44708700 con 0x559b44579390 2024-09-21T01:58:01.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44958d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559b44842110 con 0x559b445711b0 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b4494faa0 con 0x559b4459a070 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f420b8b3740 10 client.0 _ll_register_callbacks cb 0x559b4485b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f41ff7fe640 1 -- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 msgr2=0x559b4494ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b4494ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.254+0000 7f41ff7fe640 1 -- 172.21.15.29:0/341767113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b44579390 msgr2=0x559b4494cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.255+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b44579390 0x559b4494cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.255+0000 7f41ff7fe640 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4486fc40 con 0x559b4459a070 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.255+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00c2bb0 con 0x559b4459a070 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.255+0000 7f41e2ff5640 10 client.27256 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:01.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.255+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44958d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41f00be060 tx=0x7f41f0005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6839d0c3b6be595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.256+0000 7f41e2ff5640 1 -- 172.21.15.29:0/341767113 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00e5020 con 0x559b4459a070 2024-09-21T01:58:01.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.256+0000 7f41e2ff5640 1 -- 172.21.15.29:0/341767113 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f41f00a9380 con 0x559b4459a070 2024-09-21T01:58:01.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.256+0000 7f41e2ff5640 1 -- 172.21.15.29:0/341767113 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f41f00e5020 con 0x559b4459a070 2024-09-21T01:58:01.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.257+0000 7f420b8b3740 10 client.27256 fetch_fsmap learned FSMap version 44 2024-09-21T01:58:01.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.257+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=44}) -- 0x559b4495b720 con 0x559b4459a070 2024-09-21T01:58:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f41e2ff5640 1 -- 172.21.15.29:0/341767113 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f41f001f040 con 0x559b4459a070 2024-09-21T01:58:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f420b8b3740 10 client.27256 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:58:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f420b8b3740 10 client.27256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:01.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f420b8b3740 20 client.27256 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:01.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b445589f0 0x559b4495ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.258+0000 7f420b8b3740 4 client.27256 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:01.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.259+0000 7f41feffd640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b445589f0 0x559b4495ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.259+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f41f00c2bb0 con 0x559b445589f0 2024-09-21T01:58:01.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.260+0000 7f41e2ff5640 10 client.27256 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:01.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.260+0000 7f41feffd640 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b445589f0 0x559b4495ea80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b5db5573cee4af server_cookie=80acba19152ae5c2 in_seq=0 out_seq=0 2024-09-21T01:58:01.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.261+0000 7f41e2ff5640 1 -- 172.21.15.29:0/341767113 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f41f00c2bb0 con 0x559b445589f0 2024-09-21T01:58:01.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.261+0000 7f41e2ff5640 10 client.27256 handle_command_reply: tid=0 2024-09-21T01:58:01.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.261+0000 7f420b8b3740 1 client.27256 shutdown 2024-09-21T01:58:01.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.262+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 msgr2=0x559b44958d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.262+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44958d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41f00be060 tx=0x7f41f0005700 comp rx=0 tx=0).stop 2024-09-21T01:58:01.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.262+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 shutdown_connections 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.262+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b445589f0 0x559b4495ea80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.262+0000 7f41e2ff5640 0 client.27256 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b4459a070 0x559b44958d90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b44579390 0x559b4494cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 --2- 172.21.15.29:0/341767113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b4494ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 >> 172.21.15.29:0/341767113 conn(0x559b4458e460 msgr2=0x559b44944420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 shutdown_connections 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 1 -- 172.21.15.29:0/341767113 wait complete. 2024-09-21T01:58:01.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.263+0000 7f420b8b3740 20 client.27256 trim_cache size 0 max 16384 2024-09-21T01:58:01.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.294+0000 7f420b8b3740 1 Processor -- start 2024-09-21T01:58:01.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.294+0000 7f420b8b3740 1 -- start start 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b44944a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b44579390 0x559b44949c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b5cc0 con 0x559b445589f0 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b7830 con 0x559b445711b0 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f420b8b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b6390 con 0x559b44579390 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40654/0 (socket says 172.21.15.29:40654) 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 -- 172.21.15.29:0/1374033281 learned_addr learned my addr 172.21.15.29:0/1374033281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b44944a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b44579390 0x559b44949c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 -- 172.21.15.29:0/1374033281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b44579390 msgr2=0x559b44949c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b44579390 0x559b44949c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 -- 172.21.15.29:0/1374033281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 msgr2=0x559b44944a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b44944a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.295+0000 7f41feffd640 1 -- 172.21.15.29:0/1374033281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b44944fc0 con 0x559b445589f0 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.296+0000 7f41feffd640 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f41f40107b0 tx=0x7f41f4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914cd5fed1b9b94c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.297+0000 7f41e4ff9640 1 -- 172.21.15.29:0/1374033281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004050 con 0x559b445589f0 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.297+0000 7f41e4ff9640 1 -- 172.21.15.29:0/1374033281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f00c2bb0 con 0x559b445589f0 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.297+0000 7f41e4ff9640 1 -- 172.21.15.29:0/1374033281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f40052c0 con 0x559b445589f0 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- 172.21.15.29:0/1374033281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 msgr2=0x559b4494b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f41f40107b0 tx=0x7f41f4014040 comp rx=0 tx=0).stop 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- 172.21.15.29:0/1374033281 shutdown_connections 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b44579390 0x559b44949c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b445711b0 0x559b44944a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 --2- 172.21.15.29:0/1374033281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4494b900 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- 172.21.15.29:0/1374033281 >> 172.21.15.29:0/1374033281 conn(0x559b4458e460 msgr2=0x559b44944370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- 172.21.15.29:0/1374033281 shutdown_connections 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- 172.21.15.29:0/1374033281 wait complete. 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 -- start start 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.298+0000 7f420b8b3740 1 Processor -- start 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b445711b0 0x559b44960340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b449c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b7160 con 0x559b445589f0 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b6a90 con 0x559b44579390 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b3c40 con 0x559b445711b0 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f420b8b3740 10 client.0 _ll_register_callbacks cb 0x7f41f00a8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40666/0 (socket says 172.21.15.29:40666) 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 -- 172.21.15.29:0/3364919266 learned_addr learned my addr 172.21.15.29:0/3364919266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b445711b0 0x559b44960340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41ff7fe640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b449c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 -- 172.21.15.29:0/3364919266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b445711b0 msgr2=0x559b44960340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b445711b0 0x559b44960340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 -- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 msgr2=0x559b449c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b449c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.299+0000 7f41feffd640 1 -- 172.21.15.29:0/3364919266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b44960c30 con 0x559b445589f0 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.301+0000 7f41feffd640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009dd0 tx=0x7f41f40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c24e9b4064515bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.301+0000 7f41e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:58:01.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.301+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4034060 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.301+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f41f400f8a0 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.301+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00c2bb0 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f41f4004310 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 10 client.17508 fetch_fsmap learned FSMap version 44 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559b449c9800 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f4005020 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f41f4023070 con 0x559b445589f0 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 10 client.17508 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 20 client.17508 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b4459a070 0x559b449cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:01.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 4 client.17508 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.302+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f41f00c2bb0 con 0x559b4459a070 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.303+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b4459a070 0x559b449cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.304+0000 7f41fe7fc640 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b4459a070 0x559b449cbe00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2879f8551d55f463 server_cookie=fb0e986b8a3ae53b in_seq=0 out_seq=0 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.304+0000 7f41e2ff5640 10 client.17508 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f41e2ff5640 1 -- 172.21.15.29:0/3364919266 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x7f41f00c2bb0 con 0x559b4459a070 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f41e2ff5640 10 client.17508 handle_command_reply: tid=0 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 client.17508 shutdown 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 msgr2=0x559b4495fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009dd0 tx=0x7f41f40040e0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 shutdown_connections 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559b4459a070 0x559b449cbe00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559b44579390 0x559b449c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b445711b0 0x559b44960340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 --2- 172.21.15.29:0/3364919266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559b445589f0 0x559b4495fe00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 >> 172.21.15.29:0/3364919266 conn(0x559b4458e460 msgr2=0x559b4494dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f41e2ff5640 0 client.17508 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 shutdown_connections 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 1 -- 172.21.15.29:0/3364919266 wait complete. 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.305+0000 7f420b8b3740 20 client.17508 trim_cache size 0 max 16384 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:58:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "f7547296-cdb5-408c-b734-10250cb2b816", 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:58:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:01.318 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 msgr2=0x7f41e80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 0x7f41e80b8ae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f80 tx=0x7f41f8005fd0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 msgr2=0x7f4200118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41f001e920 tx=0x7f41f001ede0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 shutdown_connections 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f41e80b65d0 0x7f41e80b8ae0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f420011cc30 0x7f4200118cd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4200111f40 0x7f4200118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 --2- 172.21.15.29:0/3948421171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f420010a9e0 0x7f420011c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 >> 172.21.15.29:0/3948421171 conn(0x7f4200070ec0 msgr2=0x7f420010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.306+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 shutdown_connections 2024-09-21T01:58:01.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:01.307+0000 7f420533f640 1 -- 172.21.15.29:0/3948421171 wait complete. 2024-09-21T01:58:01.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:01 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:01.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:01 smithi029 ceph-mon[31228]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.6 MiB/s wr, 827 op/s 2024-09-21T01:58:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:01 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:01.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:01 smithi190 ceph-mon[34368]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.6 MiB/s wr, 827 op/s 2024-09-21T01:58:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:01 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:01.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:01 smithi066 ceph-mon[35001]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.6 MiB/s wr, 827 op/s 2024-09-21T01:58:02.169 INFO:tasks.cephfs.filesystem:scrub status for tag:f7547296-cdb5-408c-b734-10250cb2b816 - {'path': '/', 'tag': 'f7547296-cdb5-408c-b734-10250cb2b816', 'options': 'recursive,force'} 2024-09-21T01:58:02.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:02 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:02.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:02 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:02.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:02 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 565.0 s, 2243.8 tps, lat 44.266 ms stddev 47.457 2024-09-21T01:58:03.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:03 smithi190 ceph-mon[34368]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 709 op/s 2024-09-21T01:58:03.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:03 smithi066 ceph-mon[35001]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 709 op/s 2024-09-21T01:58:03.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:03 smithi029 ceph-mon[31228]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 709 op/s 2024-09-21T01:58:05.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:05 smithi190 ceph-mon[34368]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.4 MiB/s wr, 726 op/s 2024-09-21T01:58:05.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:58:05.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:58:05.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:05 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:58:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:05 smithi066 ceph-mon[35001]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.4 MiB/s wr, 726 op/s 2024-09-21T01:58:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:58:05.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:58:05.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:05 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:58:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:05 smithi029 ceph-mon[31228]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.4 MiB/s wr, 726 op/s 2024-09-21T01:58:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:58:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:58:05.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:05 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:58:06.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:06 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:58:06.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:06 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:58:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:06 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:58:06.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:06 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:58:06.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:06 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:58:06.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:06 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:58:07.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:07 smithi190 ceph-mon[34368]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-09-21T01:58:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:07 smithi066 ceph-mon[35001]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-09-21T01:58:07.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:07 smithi029 ceph-mon[31228]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-09-21T01:58:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 570.0 s, 2454.8 tps, lat 40.646 ms stddev 55.356 2024-09-21T01:58:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.616+0000 7fabec430640 1 -- 172.21.15.29:0/4052670243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 msgr2=0x7fabe410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.616+0000 7fabec430640 1 --2- 172.21.15.29:0/4052670243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe410a130 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009900 tx=0x7fabe002a410 comp rx=0 tx=0).stop 2024-09-21T01:58:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 -- 172.21.15.29:0/4052670243 shutdown_connections 2024-09-21T01:58:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 --2- 172.21.15.29:0/4052670243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410b6e0 0x7fabe4112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 --2- 172.21.15.29:0/4052670243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410ad40 0x7fabe410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 --2- 172.21.15.29:0/4052670243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe410a130 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 -- 172.21.15.29:0/4052670243 >> 172.21.15.29:0/4052670243 conn(0x7fabe4079010 msgr2=0x7fabe407b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 -- 172.21.15.29:0/4052670243 shutdown_connections 2024-09-21T01:58:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 -- 172.21.15.29:0/4052670243 wait complete. 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 Processor -- start 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.617+0000 7fabec430640 1 -- start start 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410ad40 0x7fabe41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410b6e0 0x7fabe41a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4115900 con 0x7fabe4109cc0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4115780 con 0x7fabe410b6e0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabec430640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4115a80 con 0x7fabe410ad40 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55142/0 (socket says 172.21.15.29:55142) 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 -- 172.21.15.29:0/3007722779 learned_addr learned my addr 172.21.15.29:0/3007722779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410ad40 0x7fabe41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410b6e0 0x7fabe41a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 -- 172.21.15.29:0/3007722779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410ad40 msgr2=0x7fabe41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410ad40 0x7fabe41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 -- 172.21.15.29:0/3007722779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410b6e0 msgr2=0x7fabe41a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410b6e0 0x7fabe41a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.618+0000 7fabea1cc640 1 -- 172.21.15.29:0/3007722779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe41a1f90 con 0x7fabe4109cc0 2024-09-21T01:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fabe0006010 tx=0x7fabe0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabdb7fe640 1 -- 172.21.15.29:0/3007722779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe003f030 con 0x7fabe4109cc0 2024-09-21T01:58:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabdb7fe640 1 -- 172.21.15.29:0/3007722779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe0003aa0 con 0x7fabe4109cc0 2024-09-21T01:58:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe41a2220 con 0x7fabe4109cc0 2024-09-21T01:58:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabdb7fe640 1 -- 172.21.15.29:0/3007722779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0035070 con 0x7fabe4109cc0 2024-09-21T01:58:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.619+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe41a8970 con 0x7fabe4109cc0 2024-09-21T01:58:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.623+0000 7fabf3481740 1 Processor -- start 2024-09-21T01:58:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.623+0000 7fabf3481740 1 -- start start 2024-09-21T01:58:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.623+0000 7fabf3481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f58e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabf3481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabf3481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabf3481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2ecb6f0 con 0x55bdd2bb11b0 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabf3481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2ebab20 con 0x55bdd2b904d0 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabf3481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2ebe930 con 0x55bdd2b882f0 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabea9cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f58e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabe99cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabe99cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48614/0 (socket says 172.21.15.29:48614) 2024-09-21T01:58:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.624+0000 7fabe99cb640 1 -- 172.21.15.29:0/3414657817 learned_addr learned my addr 172.21.15.29:0/3414657817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 -- 172.21.15.29:0/3414657817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 msgr2=0x55bdd2f594b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f594b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 -- 172.21.15.29:0/3414657817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2bb11b0 msgr2=0x55bdd2f58e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f58e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 -- 172.21.15.29:0/3414657817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd2e80530 con 0x55bdd2b904d0 2024-09-21T01:58:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fabd400ac20 tx=0x7fabd400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2112a6301a6f3f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabdb7fe640 1 -- 172.21.15.29:0/3007722779 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fabe0042030 con 0x7fabe4109cc0 2024-09-21T01:58:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabd8bfe640 1 -- 172.21.15.29:0/3414657817 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd401d070 con 0x55bdd2b904d0 2024-09-21T01:58:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabd8bfe640 1 -- 172.21.15.29:0/3414657817 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd4018af0 con 0x55bdd2b904d0 2024-09-21T01:58:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.625+0000 7fabd8bfe640 1 -- 172.21.15.29:0/3414657817 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4010b40 con 0x55bdd2b904d0 2024-09-21T01:58:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabdb7fe640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 0x7fabd00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabdb7fe640 1 -- 172.21.15.29:0/3007722779 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fabe01077d0 con 0x7fabe4109cc0 2024-09-21T01:58:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 0x7fabd00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 -- 172.21.15.29:0/3414657817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 msgr2=0x55bdd2f5a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fabd400ac20 tx=0x7fabd400f790 comp rx=0 tx=0).stop 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 -- 172.21.15.29:0/3414657817 shutdown_connections 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b904d0 0x55bdd2f5a000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 --2- 172.21.15.29:0/3414657817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f58e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.626+0000 7fabf3481740 1 -- 172.21.15.29:0/3414657817 >> 172.21.15.29:0/3414657817 conn(0x55bdd2ba5300 msgr2=0x55bdd2f548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabf3481740 1 -- 172.21.15.29:0/3414657817 shutdown_connections 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabf3481740 1 -- 172.21.15.29:0/3414657817 wait complete. 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 0x7fabd00b8aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fabdc005fd0 tx=0x7fabdc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabf3481740 1 -- start start 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabf3481740 1 Processor -- start 2024-09-21T01:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.627+0000 7fabf3481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f63860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2df2ab0 con 0x55bdd2b904d0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2e60730 con 0x55bdd2b882f0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bdd2f5ab20 con 0x55bdd2bb11b0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabf3481740 10 client.0 _ll_register_callbacks cb 0x55bdd2eb46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:48622/0 (socket says 172.21.15.29:48622) 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 -- 172.21.15.29:0/3625844175 learned_addr learned my addr 172.21.15.29:0/3625844175 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 -- 172.21.15.29:0/3625844175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 msgr2=0x55bdd2f63860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f63860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 -- 172.21.15.29:0/3625844175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 msgr2=0x55bdd2f62cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f62cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea1cc640 1 -- 172.21.15.29:0/3625844175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd2c1c790 con 0x55bdd2b882f0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.628+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f63860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fabe00cd720 tx=0x7fabe00c99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7014cae90b9a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0049040 con 0x55bdd2b882f0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd4018af0 con 0x55bdd2b882f0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fabe00d6f00 con 0x55bdd2b882f0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fabe003b460 con 0x55bdd2b882f0 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabf3481740 10 client.27311 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bdd2c95620 con 0x55bdd2b882f0 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.629+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe00cd530 con 0x55bdd2b882f0 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fabe00f2070 con 0x55bdd2b882f0 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 10 client.27311 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 20 client.27311 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2b6fd80 0x55bdd2f6feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 4 client.27311 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabd4018af0 con 0x55bdd2b6fd80 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.630+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2b6fd80 0x55bdd2f6feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.631+0000 7fabcdffb640 10 client.27311 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.631+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2b6fd80 0x55bdd2f6feb0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6a09caefc564 server_cookie=344558011814ac34 in_seq=0 out_seq=0 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.631+0000 7fabcdffb640 1 -- 172.21.15.29:0/3625844175 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fabd4018af0 con 0x55bdd2b6fd80 2024-09-21T01:58:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.631+0000 7fabcdffb640 10 client.27311 handle_command_reply: tid=0 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.631+0000 7fabf3481740 1 client.27311 shutdown 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 msgr2=0x55bdd2f62630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fabe00cd720 tx=0x7fabe00c99f0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 shutdown_connections 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2b6fd80 0x55bdd2f6feb0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2bb11b0 0x55bdd2f63860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabcdffb640 0 client.27311 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f62cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 --2- 172.21.15.29:0/3625844175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f62630 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 >> 172.21.15.29:0/3625844175 conn(0x55bdd2ba5300 msgr2=0x55bdd2f52ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 shutdown_connections 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 1 -- 172.21.15.29:0/3625844175 wait complete. 2024-09-21T01:58:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.632+0000 7fabf3481740 20 client.27311 trim_cache size 0 max 16384 2024-09-21T01:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.650+0000 7fabf3481740 1 Processor -- start 2024-09-21T01:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.650+0000 7fabf3481740 1 -- start start 2024-09-21T01:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f61850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f61d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b1580 con 0x55bdd2b6fd80 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b30c0 con 0x55bdd2b882f0 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabf3481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b1c50 con 0x55bdd2b904d0 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:55526/0 (socket says 172.21.15.29:55526) 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 -- 172.21.15.29:0/3845331876 learned_addr learned my addr 172.21.15.29:0/3845331876 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea1cc640 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f61850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabe99cb640 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 -- 172.21.15.29:0/3845331876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 msgr2=0x55bdd2f61d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f61d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 -- 172.21.15.29:0/3845331876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b6fd80 msgr2=0x55bdd2f61850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f61850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.651+0000 7fabea9cd640 1 -- 172.21.15.29:0/3845331876 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd2f4ecf0 con 0x55bdd2b904d0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.652+0000 7fabea9cd640 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fabdc003f10 tx=0x7fabdc016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a44291df8be9944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.652+0000 7fabcffff640 1 -- 172.21.15.29:0/3845331876 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00f5a0 con 0x55bdd2b904d0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.652+0000 7fabcffff640 1 -- 172.21.15.29:0/3845331876 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabdc00fbf0 con 0x55bdd2b904d0 2024-09-21T01:58:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.652+0000 7fabcffff640 1 -- 172.21.15.29:0/3845331876 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc01d9c0 con 0x55bdd2b904d0 2024-09-21T01:58:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 -- 172.21.15.29:0/3845331876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 msgr2=0x55bdd2f52ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fabdc003f10 tx=0x7fabdc016000 comp rx=0 tx=0).stop 2024-09-21T01:58:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 -- 172.21.15.29:0/3845331876 shutdown_connections 2024-09-21T01:58:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b904d0 0x55bdd2f52ff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b882f0 0x55bdd2f61d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 --2- 172.21.15.29:0/3845331876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f61850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 -- 172.21.15.29:0/3845331876 >> 172.21.15.29:0/3845331876 conn(0x55bdd2ba5300 msgr2=0x55bdd2f59cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 -- 172.21.15.29:0/3845331876 shutdown_connections 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.653+0000 7fabf3481740 1 -- 172.21.15.29:0/3845331876 wait complete. 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 -- start start 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 Processor -- start 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f69c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f68ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b29f0 con 0x55bdd2b904d0 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b2320 con 0x55bdd2b6fd80 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41af560 con 0x55bdd2b882f0 2024-09-21T01:58:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabf3481740 10 client.0 _ll_register_callbacks cb 0x7fabe0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabea9cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f68ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabe99cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabea9cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f68ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55170/0 (socket says 172.21.15.29:55170) 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabea9cd640 1 -- 172.21.15.29:0/1670166171 learned_addr learned my addr 172.21.15.29:0/1670166171 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.654+0000 7fabea1cc640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f69c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 -- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b6fd80 msgr2=0x55bdd2f69c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f69c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 -- 172.21.15.29:0/1670166171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 msgr2=0x55bdd2f68ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f68ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 -- 172.21.15.29:0/1670166171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd2f69240 con 0x55bdd2b882f0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabe99cb640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f6a2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55bdd2f70620 tx=0x7fabd40184c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c966c551ceb1e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd401d040 con 0x55bdd2b882f0 2024-09-21T01:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fabd40188f0 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabdc00fbf0 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fabd4005040 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabf3481740 10 client.27277 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.655+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bdd2f68140 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabdc00fbf0 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fabd4010b10 con 0x55bdd2b882f0 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 10 client.27277 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 20 client.27277 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2bb11b0 0x55bdd2fdcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 4 client.27277 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:58:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.656+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdd2ecd640 con 0x55bdd2bb11b0 2024-09-21T01:58:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.657+0000 7fabea1cc640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2bb11b0 0x55bdd2fdcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.657+0000 7fabea1cc640 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2bb11b0 0x55bdd2fdcd50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f433d5071d15315 server_cookie=b6d5f567d66ff0cc in_seq=0 out_seq=0 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.657+0000 7fabcdffb640 10 client.27277 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.717+0000 7fabcdffb640 1 -- 172.21.15.29:0/1670166171 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdd2ecd640 con 0x55bdd2bb11b0 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.717+0000 7fabcdffb640 10 client.27277 handle_command_reply: tid=0 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.717+0000 7fabf3481740 1 client.27277 shutdown 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.718+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 msgr2=0x55bdd2f6a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.718+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f6a2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55bdd2f70620 tx=0x7fabd40184c0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 shutdown_connections 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55bdd2bb11b0 0x55bdd2fdcd50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd2b904d0 0x55bdd2f68ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdd2b882f0 0x55bdd2f6a2e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 --2- 172.21.15.29:0/1670166171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bdd2b6fd80 0x55bdd2f69c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabcdffb640 0 client.27277 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 >> 172.21.15.29:0/1670166171 conn(0x55bdd2ba5300 msgr2=0x55bdd2f60960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 shutdown_connections 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 1 -- 172.21.15.29:0/1670166171 wait complete. 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabf3481740 20 client.27277 trim_cache size 0 max 16384 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:58:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 msgr2=0x7fabd00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 0x7fabd00b8aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fabdc005fd0 tx=0x7fabdc009500 comp rx=0 tx=0).stop 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 msgr2=0x7fabe41a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.719+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fabe0006010 tx=0x7fabe0032f90 comp rx=0 tx=0).stop 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 shutdown_connections 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabd00b6590 0x7fabd00b8aa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabe410b6e0 0x7fabe41a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe410ad40 0x7fabe41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 --2- 172.21.15.29:0/3007722779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe4109cc0 0x7fabe41a0550 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 >> 172.21.15.29:0/3007722779 conn(0x7fabe4079010 msgr2=0x7fabe410e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 shutdown_connections 2024-09-21T01:58:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:09.720+0000 7fabec430640 1 -- 172.21.15.29:0/3007722779 wait complete. 2024-09-21T01:58:09.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:09 smithi066 ceph-mon[35001]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-09-21T01:58:09.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:09 smithi029 ceph-mon[31228]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-09-21T01:58:09.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:09 smithi190 ceph-mon[34368]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-09-21T01:58:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:11 smithi066 ceph-mon[35001]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-09-21T01:58:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:11 smithi029 ceph-mon[31228]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-09-21T01:58:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:11 smithi190 ceph-mon[34368]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-09-21T01:58:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 575.0 s, 1711.4 tps, lat 58.583 ms stddev 65.902 2024-09-21T01:58:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:13 smithi066 ceph-mon[35001]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.1 MiB/s wr, 687 op/s 2024-09-21T01:58:13.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:13 smithi029 ceph-mon[31228]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.1 MiB/s wr, 687 op/s 2024-09-21T01:58:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:13 smithi190 ceph-mon[34368]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.1 MiB/s wr, 687 op/s 2024-09-21T01:58:15.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:15 smithi066 ceph-mon[35001]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.8 MiB/s wr, 647 op/s 2024-09-21T01:58:15.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:15 smithi029 ceph-mon[31228]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.8 MiB/s wr, 647 op/s 2024-09-21T01:58:15.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:15 smithi190 ceph-mon[34368]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.8 MiB/s wr, 647 op/s 2024-09-21T01:58:17.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:17 smithi066 ceph-mon[35001]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-21T01:58:17.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:17 smithi029 ceph-mon[31228]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-21T01:58:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 580.0 s, 2396.6 tps, lat 41.280 ms stddev 53.546 2024-09-21T01:58:17.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:17 smithi190 ceph-mon[34368]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-21T01:58:19.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:19 smithi066 ceph-mon[35001]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-21T01:58:19.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:19 smithi029 ceph-mon[31228]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-21T01:58:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:19 smithi190 ceph-mon[34368]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-21T01:58:21.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:21 smithi066 ceph-mon[35001]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-09-21T01:58:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:21 smithi029 ceph-mon[31228]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-09-21T01:58:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:21 smithi190 ceph-mon[34368]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-09-21T01:58:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.882+0000 7feab7e8b640 1 -- 172.21.15.29:0/228314159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 msgr2=0x7feab0112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.882+0000 7feab7e8b640 1 --2- 172.21.15.29:0/228314159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab0112080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009ca0 tx=0x7feaa802a8f0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.883+0000 7feab7e8b640 1 -- 172.21.15.29:0/228314159 shutdown_connections 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.883+0000 7feab7e8b640 1 --2- 172.21.15.29:0/228314159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab0112080 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.883+0000 7feab7e8b640 1 --2- 172.21.15.29:0/228314159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab010b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.883+0000 7feab7e8b640 1 --2- 172.21.15.29:0/228314159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.883+0000 7feab7e8b640 1 -- 172.21.15.29:0/228314159 >> 172.21.15.29:0/228314159 conn(0x7feab0079010 msgr2=0x7feab007b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.884+0000 7feab7e8b640 1 -- 172.21.15.29:0/228314159 shutdown_connections 2024-09-21T01:58:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.885+0000 7feab7e8b640 1 -- 172.21.15.29:0/228314159 wait complete. 2024-09-21T01:58:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 Processor -- start 2024-09-21T01:58:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 -- start start 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab01a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feab0115a00 con 0x7feab010ad40 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feab0115880 con 0x7feab0109cc0 2024-09-21T01:58:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.886+0000 7feab7e8b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feab0115b80 con 0x7feab010b6e0 2024-09-21T01:58:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41680/0 (socket says 172.21.15.29:41680) 2024-09-21T01:58:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 -- 172.21.15.29:0/2494672486 learned_addr learned my addr 172.21.15.29:0/2494672486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5426640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab6428640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab01a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 -- 172.21.15.29:0/2494672486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 msgr2=0x7feab01a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab01a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 -- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 msgr2=0x7feab01a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab01a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5c27640 1 -- 172.21.15.29:0/2494672486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab01a20c0 con 0x7feab0109cc0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.887+0000 7feab5426640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab01a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feab5c27640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feaa000b450 tx=0x7feaa000b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feaa6ffd640 1 -- 172.21.15.29:0/2494672486 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0015020 con 0x7feab0109cc0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feaa6ffd640 1 -- 172.21.15.29:0/2494672486 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa0011ce0 con 0x7feab0109cc0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab01a23b0 con 0x7feab0109cc0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feaa6ffd640 1 -- 172.21.15.29:0/2494672486 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0010e20 con 0x7feab0109cc0 2024-09-21T01:58:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.888+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab01a8aa0 con 0x7feab0109cc0 2024-09-21T01:58:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.892+0000 7feabedfd740 1 Processor -- start 2024-09-21T01:58:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 -- start start 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e02f0 0x5608c3db14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e84d0 0x5608c3db2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5608c3d236f0 con 0x5608c39e84d0 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5608c3d12b20 con 0x5608c3a091b0 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feabedfd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608c3d16930 con 0x5608c39e02f0 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.893+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41684/0 (socket says 172.21.15.29:41684) 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 -- 172.21.15.29:0/4247745734 learned_addr learned my addr 172.21.15.29:0/4247745734 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab5426640 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e84d0 0x5608c3db2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 -- 172.21.15.29:0/4247745734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e02f0 msgr2=0x5608c3db14c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:58:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e02f0 0x5608c3db14c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 -- 172.21.15.29:0/4247745734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e84d0 msgr2=0x5608c3db2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feaa6ffd640 1 -- 172.21.15.29:0/2494672486 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7feaa0018020 con 0x7feab0109cc0 2024-09-21T01:58:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e84d0 0x5608c3db2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.894+0000 7feab6428640 1 -- 172.21.15.29:0/4247745734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c3cd8530 con 0x5608c3a091b0 2024-09-21T01:58:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.895+0000 7feab6428640 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feaa802f4f0 tx=0x7feaa8009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6366ebe66a5b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.896+0000 7feaa6ffd640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 0x7fea9c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.896+0000 7fea9bfff640 1 -- 172.21.15.29:0/4247745734 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8031680 con 0x5608c3a091b0 2024-09-21T01:58:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.896+0000 7fea9bfff640 1 -- 172.21.15.29:0/4247745734 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa8036040 con 0x5608c3a091b0 2024-09-21T01:58:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.896+0000 7fea9bfff640 1 -- 172.21.15.29:0/4247745734 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa802f640 con 0x5608c3a091b0 2024-09-21T01:58:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.896+0000 7feaa6ffd640 1 -- 172.21.15.29:0/2494672486 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feaa00e69d0 con 0x7feab0109cc0 2024-09-21T01:58:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.897+0000 7feab5c27640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 0x7fea9c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- 172.21.15.29:0/4247745734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 msgr2=0x5608c3db0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feab5c27640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 0x7fea9c0b8b60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feaa00acaa0 tx=0x7feaa00106a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feaa802f4f0 tx=0x7feaa8009a10 comp rx=0 tx=0).stop 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- 172.21.15.29:0/4247745734 shutdown_connections 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e84d0 0x5608c3db2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e02f0 0x5608c3db14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 --2- 172.21.15.29:0/4247745734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3db0e50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- 172.21.15.29:0/4247745734 >> 172.21.15.29:0/4247745734 conn(0x5608c39fd300 msgr2=0x5608c3dac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- 172.21.15.29:0/4247745734 shutdown_connections 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- 172.21.15.29:0/4247745734 wait complete. 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 -- start start 2024-09-21T01:58:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.898+0000 7feabedfd740 1 Processor -- start 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3dba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dbacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5608c3c4aab0 con 0x5608c39e02f0 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5608c3cb8730 con 0x5608c3a091b0 2024-09-21T01:58:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608c3db2ad0 con 0x5608c39e84d0 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feabedfd740 10 client.0 _ll_register_callbacks cb 0x5608c3d0c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41700/0 (socket says 172.21.15.29:41700) 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feab6428640 1 -- 172.21.15.29:0/3596674206 learned_addr learned my addr 172.21.15.29:0/3596674206 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feab6428640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3dba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.899+0000 7feab5426640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dbacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 -- 172.21.15.29:0/3596674206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 msgr2=0x5608c3dbacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dbacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 -- 172.21.15.29:0/3596674206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 msgr2=0x5608c3dba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3dba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 -- 172.21.15.29:0/3596674206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c3a74790 con 0x5608c3a091b0 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feab6428640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005cc0 tx=0x7feaa8005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f4df828528be79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7fea997fa640 10 client.27326 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.900+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa8036040 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.901+0000 7fea997fa640 1 -- 172.21.15.29:0/3596674206 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8004b40 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.901+0000 7fea997fa640 1 -- 172.21.15.29:0/3596674206 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feaa8004ce0 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.901+0000 7fea997fa640 1 -- 172.21.15.29:0/3596674206 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feaa805a050 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.901+0000 7feabedfd740 10 client.27326 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.901+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5608c3aed620 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7fea997fa640 1 -- 172.21.15.29:0/3596674206 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7feaa802f8a0 con 0x5608c3a091b0 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 10 client.27326 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 10 client.27326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 20 client.27326 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c39c7d80 0x5608c3dc77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 4 client.27326 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608c3d25640 con 0x5608c39c7d80 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feab5426640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c39c7d80 0x5608c3dc77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7feab5426640 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c39c7d80 0x5608c3dc77c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c70aa92ffdc3ef8 server_cookie=769c6d64d534101b in_seq=0 out_seq=0 2024-09-21T01:58:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.902+0000 7fea997fa640 10 client.27326 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7fea997fa640 1 -- 172.21.15.29:0/3596674206 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608c3d25640 con 0x5608c39c7d80 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7fea997fa640 10 client.27326 handle_command_reply: tid=0 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 client.27326 shutdown 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 msgr2=0x5608c3dbb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005cc0 tx=0x7feaa8005c10 comp rx=0 tx=0).stop 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 shutdown_connections 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c39c7d80 0x5608c3dc77c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dbb820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dbacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 --2- 172.21.15.29:0/3596674206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3dba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 >> 172.21.15.29:0/3596674206 conn(0x5608c39fd300 msgr2=0x5608c3daaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.903+0000 7fea997fa640 0 client.27326 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.904+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 shutdown_connections 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.904+0000 7feabedfd740 1 -- 172.21.15.29:0/3596674206 wait complete. 2024-09-21T01:58:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.904+0000 7feabedfd740 20 client.27326 trim_cache size 0 max 16384 2024-09-21T01:58:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.924+0000 7feabedfd740 1 Processor -- start 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.924+0000 7feabedfd740 1 -- start start 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3db5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3db9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dabd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feab01b1690 con 0x5608c39e02f0 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feab01b31d0 con 0x5608c3a091b0 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feabedfd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feab01b1d60 con 0x5608c39e84d0 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab6428640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3db5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3db9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab6428640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3db5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48316/0 (socket says 172.21.15.29:48316) 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab6428640 1 -- 172.21.15.29:0/4270012134 learned_addr learned my addr 172.21.15.29:0/4270012134 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab6428640 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dabd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 -- 172.21.15.29:0/4270012134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 msgr2=0x5608c3dabd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dabd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 -- 172.21.15.29:0/4270012134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 msgr2=0x5608c3db5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3db5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 -- 172.21.15.29:0/4270012134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c39c7fb0 con 0x5608c39e84d0 2024-09-21T01:58:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.925+0000 7feab5426640 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3db9b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5608c3d16a80 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8807b747489231 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.926+0000 7fea9b7fe640 1 -- 172.21.15.29:0/4270012134 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac026420 con 0x5608c39e84d0 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.926+0000 7fea9b7fe640 1 -- 172.21.15.29:0/4270012134 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaac021030 con 0x5608c39e84d0 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.926+0000 7fea9b7fe640 1 -- 172.21.15.29:0/4270012134 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac0156b0 con 0x5608c39e84d0 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.926+0000 7feabedfd740 1 -- 172.21.15.29:0/4270012134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 msgr2=0x5608c3db9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.926+0000 7feabedfd740 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3db9b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5608c3d16a80 tx=0x7feaac008040 comp rx=0 tx=0).stop 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 -- 172.21.15.29:0/4270012134 shutdown_connections 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c3a091b0 0x5608c3dabd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3db9b40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 --2- 172.21.15.29:0/4270012134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c39e02f0 0x5608c3db5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 -- 172.21.15.29:0/4270012134 >> 172.21.15.29:0/4270012134 conn(0x5608c39fd300 msgr2=0x5608c3dab610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 -- 172.21.15.29:0/4270012134 shutdown_connections 2024-09-21T01:58:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.927+0000 7feabedfd740 1 -- 172.21.15.29:0/4270012134 wait complete. 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.928+0000 7feabedfd740 1 -- start start 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.928+0000 7feabedfd740 1 Processor -- start 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.928+0000 7feabedfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c3a091b0 0x5608c3dc0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feab01b2b00 con 0x5608c3a091b0 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feab01b2430 con 0x5608c39e02f0 2024-09-21T01:58:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feab01af670 con 0x5608c39e84d0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feabedfd740 10 client.0 _ll_register_callbacks cb 0x7feaa802f440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41726/0 (socket says 172.21.15.29:41726) 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab5426640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 -- 172.21.15.29:0/2096186398 learned_addr learned my addr 172.21.15.29:0/2096186398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c3a091b0 0x5608c3dc0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 -- 172.21.15.29:0/2096186398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 msgr2=0x5608c3dc0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dc0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 -- 172.21.15.29:0/2096186398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c3a091b0 msgr2=0x5608c3dc0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c3a091b0 0x5608c3dc0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 -- 172.21.15.29:0/2096186398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c3dc1410 con 0x5608c39e02f0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7feab6428640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feaa802fec0 tx=0x7feaa804d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ab53d6996f552f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7fea997fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80307f0 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.929+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7feaa8030990 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaac021030 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feaa8030b90 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 10 client.27332 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5608c3dc2700 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaac021030 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7feaa8045a30 con 0x5608c39e02f0 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 10 client.27332 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 10 client.27332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 20 client.27332 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c399d1c0 0x5608c3e34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 4 client.27332 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:58:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608c3d210f0 con 0x5608c399d1c0 2024-09-21T01:58:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.930+0000 7feab5426640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c399d1c0 0x5608c3e34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.931+0000 7fea997fa640 10 client.27332 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.932+0000 7feab5426640 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c399d1c0 0x5608c3e34370 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee363b209bf27f60 server_cookie=42b64c9711e4b3ad in_seq=0 out_seq=0 2024-09-21T01:58:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.945+0000 7fea997fa640 1 -- 172.21.15.29:0/2096186398 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608c3d210f0 con 0x5608c399d1c0 2024-09-21T01:58:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.945+0000 7fea997fa640 10 client.27332 handle_command_reply: tid=0 2024-09-21T01:58:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.945+0000 7feabedfd740 1 client.27332 shutdown 2024-09-21T01:58:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.945+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 msgr2=0x5608c3dbfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.945+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feaa802fec0 tx=0x7feaa804d000 comp rx=0 tx=0).stop 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 shutdown_connections 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5608c399d1c0 0x5608c3e34370 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7fea997fa640 0 client.27332 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c3a091b0 0x5608c3dc0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c39e84d0 0x5608c3dc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 --2- 172.21.15.29:0/2096186398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c39e02f0 0x5608c3dbfad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 >> 172.21.15.29:0/2096186398 conn(0x5608c39fd300 msgr2=0x5608c3db5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 shutdown_connections 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 1 -- 172.21.15.29:0/2096186398 wait complete. 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feabedfd740 20 client.27332 trim_cache size 0 max 16384 2024-09-21T01:58:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:58:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:58:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:58:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:58:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 msgr2=0x7fea9c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 0x7fea9c0b8b60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feaa00acaa0 tx=0x7feaa00106a0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 msgr2=0x7feab01a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feaa000b450 tx=0x7feaa000b850 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 shutdown_connections 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea9c0b6650 0x7fea9c0b8b60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab010b6e0 0x7feab01a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.946+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feab010ad40 0x7feab01a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.947+0000 7feab7e8b640 1 --2- 172.21.15.29:0/2494672486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feab0109cc0 0x7feab01a0680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.947+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 >> 172.21.15.29:0/2494672486 conn(0x7feab0079010 msgr2=0x7feab010e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.947+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 shutdown_connections 2024-09-21T01:58:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:22.947+0000 7feab7e8b640 1 -- 172.21.15.29:0/2494672486 wait complete. 2024-09-21T01:58:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 585.0 s, 2375.6 tps, lat 42.568 ms stddev 48.214 2024-09-21T01:58:23.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:23 smithi190 ceph-mon[34368]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 706 op/s 2024-09-21T01:58:23.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:23 smithi066 ceph-mon[35001]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 706 op/s 2024-09-21T01:58:23.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:23 smithi029 ceph-mon[31228]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 706 op/s 2024-09-21T01:58:25.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:25 smithi066 ceph-mon[35001]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-09-21T01:58:25.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:25 smithi029 ceph-mon[31228]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-09-21T01:58:25.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:25 smithi190 ceph-mon[34368]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-09-21T01:58:27.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:27 smithi066 ceph-mon[35001]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 801 op/s 2024-09-21T01:58:27.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:27 smithi029 ceph-mon[31228]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 801 op/s 2024-09-21T01:58:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 590.0 s, 1921.6 tps, lat 51.838 ms stddev 59.875 2024-09-21T01:58:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:27 smithi190 ceph-mon[34368]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 801 op/s 2024-09-21T01:58:29.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:29 smithi066 ceph-mon[35001]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-21T01:58:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:29 smithi029 ceph-mon[31228]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-21T01:58:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:29 smithi190 ceph-mon[34368]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-21T01:58:31.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:31 smithi066 ceph-mon[35001]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 752 op/s 2024-09-21T01:58:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:31 smithi029 ceph-mon[31228]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 752 op/s 2024-09-21T01:58:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:31 smithi190 ceph-mon[34368]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 752 op/s 2024-09-21T01:58:32.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:58:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 595.0 s, 2079.2 tps, lat 48.342 ms stddev 56.427 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 -- 172.21.15.29:0/2365114440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 msgr2=0x7fd39810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/2365114440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39810f520 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd390009ca0 tx=0x7fd39002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 -- 172.21.15.29:0/2365114440 shutdown_connections 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/2365114440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39810f520 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/2365114440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3981083a0 0x7fd3981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/2365114440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd398102a60 0x7fd398102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 -- 172.21.15.29:0/2365114440 >> 172.21.15.29:0/2365114440 conn(0x7fd398079730 msgr2=0x7fd398079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 -- 172.21.15.29:0/2365114440 shutdown_connections 2024-09-21T01:58:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.396+0000 7fd3a00fd640 1 -- 172.21.15.29:0/2365114440 wait complete. 2024-09-21T01:58:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 Processor -- start 2024-09-21T01:58:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 -- start start 2024-09-21T01:58:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd398102a60 0x7fd39819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981129b0 con 0x7fd398108cf0 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd398112830 con 0x7fd3981083a0 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd3a00fd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd398112b30 con 0x7fd398102a60 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd39d698640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd39e69a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd39d698640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35100/0 (socket says 172.21.15.29:35100) 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd39e69a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47234/0 (socket says 172.21.15.29:47234) 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.397+0000 7fd39d698640 1 -- 172.21.15.29:0/1950500054 learned_addr learned my addr 172.21.15.29:0/1950500054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39de99640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd398102a60 0x7fd39819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 -- 172.21.15.29:0/1950500054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd398102a60 msgr2=0x7fd39819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd398102a60 0x7fd39819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 -- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 msgr2=0x7fd39819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39819bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 -- 172.21.15.29:0/1950500054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3981a64d0 con 0x7fd3981083a0 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd39d698640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd38800d770 tx=0x7fd38800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd38effd640 1 -- 172.21.15.29:0/1950500054 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388017070 con 0x7fd3981083a0 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd38effd640 1 -- 172.21.15.29:0/1950500054 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd388009c30 con 0x7fd3981083a0 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3981a67c0 con 0x7fd3981083a0 2024-09-21T01:58:33.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.398+0000 7fd38effd640 1 -- 172.21.15.29:0/1950500054 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800f620 con 0x7fd3981083a0 2024-09-21T01:58:33.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.399+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3981a6d50 con 0x7fd3981083a0 2024-09-21T01:58:33.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.401+0000 7fd3a6671740 1 Processor -- start 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 -- start start 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd38effd640 1 -- 172.21.15.29:0/1950500054 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd38801a070 con 0x7fd3981083a0 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba789f390 0x564ba7c680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564ba7bdde50 con 0x564ba78c0070 2024-09-21T01:58:33.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564ba7bcd280 con 0x564ba789f390 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd3a6671740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ba7bd1090 con 0x564ba78971b0 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39de99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39e69a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39d698640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba789f390 0x564ba7c680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39de99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49606/0 (socket says 172.21.15.29:49606) 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39de99640 1 -- 172.21.15.29:0/1046394227 learned_addr learned my addr 172.21.15.29:0/1046394227 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.402+0000 7fd39e69a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47244/0 (socket says 172.21.15.29:47244) 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 -- 172.21.15.29:0/1046394227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba789f390 msgr2=0x564ba7c680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba789f390 0x564ba7c680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 -- 172.21.15.29:0/1046394227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 msgr2=0x564ba7c67650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 -- 172.21.15.29:0/1046394227 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba792b650 con 0x564ba78971b0 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39e69a640 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:33.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39de99640 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd394009c90 tx=0x7fd394011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77423aa15d29670 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd383fff640 1 -- 172.21.15.29:0/1046394227 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394009e40 con 0x564ba78971b0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd383fff640 1 -- 172.21.15.29:0/1046394227 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3940054d0 con 0x564ba78971b0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd383fff640 1 -- 172.21.15.29:0/1046394227 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394019670 con 0x564ba78971b0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd38effd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 0x7fd3840b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.403+0000 7fd39e69a640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 0x7fd3840b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.404+0000 7fd38effd640 1 -- 172.21.15.29:0/1950500054 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd3880e6cd0 con 0x7fd3981083a0 2024-09-21T01:58:33.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.404+0000 7fd3a6671740 1 -- 172.21.15.29:0/1046394227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 msgr2=0x564ba7c67b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.404+0000 7fd39e69a640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 0x7fd3840b8ad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd390006010 tx=0x7fd390005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.404+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd394009c90 tx=0x7fd394011810 comp rx=0 tx=0).stop 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 -- 172.21.15.29:0/1046394227 shutdown_connections 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba789f390 0x564ba7c680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba78971b0 0x564ba7c67b90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1046394227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c67650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 -- 172.21.15.29:0/1046394227 >> 172.21.15.29:0/1046394227 conn(0x564ba78b4460 msgr2=0x564ba7c65540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.405+0000 7fd3a6671740 1 -- 172.21.15.29:0/1046394227 shutdown_connections 2024-09-21T01:58:33.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.408+0000 7fd3a6671740 1 -- 172.21.15.29:0/1046394227 wait complete. 2024-09-21T01:58:33.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.409+0000 7fd3a6671740 1 -- start start 2024-09-21T01:58:33.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.409+0000 7fd3a6671740 1 Processor -- start 2024-09-21T01:58:33.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c77b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd39de99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd39de99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35118/0 (socket says 172.21.15.29:35118) 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd39d698640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd39d698640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c775f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49618/0 (socket says 172.21.15.29:49618) 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd39de99640 1 -- 172.21.15.29:0/3224931621 learned_addr learned my addr 172.21.15.29:0/3224931621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.410+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564ba7a2e700 con 0x564ba78c0070 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564ba7b68110 con 0x564ba78971b0 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ba7c7a190 con 0x564ba789f390 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd3a6671740 10 client.0 _ll_register_callbacks cb 0x564ba7b81990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 -- 172.21.15.29:0/3224931621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 msgr2=0x564ba7c775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c775f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 -- 172.21.15.29:0/3224931621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 msgr2=0x564ba7c77b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c77b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba7b95c40 con 0x564ba78971b0 2024-09-21T01:58:33.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd39de99640 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd394009380 tx=0x7fd394005060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6bbdc28f3fa6e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.411+0000 7fd3817fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:33.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.412+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394009e40 con 0x564ba78971b0 2024-09-21T01:58:33.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.412+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3940054d0 con 0x564ba78971b0 2024-09-21T01:58:33.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.412+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd394004030 con 0x564ba78971b0 2024-09-21T01:58:33.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.412+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd394033050 con 0x564ba78971b0 2024-09-21T01:58:33.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.413+0000 7fd3a6671740 10 client.27344 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.413+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564ba7c85e10 con 0x564ba78971b0 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.413+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd394004230 con 0x564ba78971b0 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.413+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fd39402f4f0 con 0x564ba78971b0 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 10 client.27344 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:33.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 20 client.27344 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:33.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba787e9f0 0x564ba7c89230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 4 client.27344 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:33.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.415+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd3940054d0 con 0x564ba787e9f0 2024-09-21T01:58:33.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.418+0000 7fd39d698640 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba787e9f0 0x564ba7c89230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.423+0000 7fd39d698640 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba787e9f0 0x564ba7c89230 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc900eb4395219 server_cookie=e22806ca5b0f0e81 in_seq=0 out_seq=0 2024-09-21T01:58:33.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.423+0000 7fd3817fa640 10 client.27344 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.425+0000 7fd3817fa640 1 -- 172.21.15.29:0/3224931621 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fd3940054d0 con 0x564ba787e9f0 2024-09-21T01:58:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.425+0000 7fd3817fa640 10 client.27344 handle_command_reply: tid=0 2024-09-21T01:58:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.425+0000 7fd3a6671740 1 client.27344 shutdown 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 msgr2=0x564ba7c770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd394009380 tx=0x7fd394005060 comp rx=0 tx=0).stop 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 shutdown_connections 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba787e9f0 0x564ba7c89230 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7c77b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3224931621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c770b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 >> 172.21.15.29:0/3224931621 conn(0x564ba78b4460 msgr2=0x564ba7c69040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.433+0000 7fd3817fa640 0 client.27344 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:33.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.436+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 shutdown_connections 2024-09-21T01:58:33.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.436+0000 7fd3a6671740 1 -- 172.21.15.29:0/3224931621 wait complete. 2024-09-21T01:58:33.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.439+0000 7fd3a6671740 20 client.27344 trim_cache size 0 max 16384 2024-09-21T01:58:33.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.477+0000 7fd3a6671740 1 Processor -- start 2024-09-21T01:58:33.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.478+0000 7fd3a6671740 1 -- start start 2024-09-21T01:58:33.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.478+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 0x564ba7c68d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.479+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.479+0000 7fd39de99640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 0x564ba7c68d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.480+0000 7fd39de99640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 0x564ba7c68d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47258/0 (socket says 172.21.15.29:47258) 2024-09-21T01:58:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.480+0000 7fd39de99640 1 -- 172.21.15.29:0/3375436644 learned_addr learned my addr 172.21.15.29:0/3375436644 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:33.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.480+0000 7fd39d698640 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.480+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78c0070 0x564ba7c747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.481+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981ad220 con 0x564ba78971b0 2024-09-21T01:58:33.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.481+0000 7fd39e69a640 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78c0070 0x564ba7c747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.481+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981aed60 con 0x564ba78c0070 2024-09-21T01:58:33.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.481+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981ad8f0 con 0x564ba789f390 2024-09-21T01:58:33.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.482+0000 7fd39d698640 1 -- 172.21.15.29:0/3375436644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78c0070 msgr2=0x564ba7c747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.482+0000 7fd39d698640 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78c0070 0x564ba7c747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.482+0000 7fd39d698640 1 -- 172.21.15.29:0/3375436644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 msgr2=0x564ba7c68d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.482+0000 7fd39d698640 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 0x564ba7c68d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.482+0000 7fd39d698640 1 -- 172.21.15.29:0/3375436644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba7c797f0 con 0x564ba789f390 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.483+0000 7fd39d698640 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c742a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3880e4830 tx=0x7fd3880d0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8412781020c937a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.483+0000 7fd3837fe640 1 -- 172.21.15.29:0/3375436644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388021040 con 0x564ba789f390 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.483+0000 7fd3837fe640 1 -- 172.21.15.29:0/3375436644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3880e8040 con 0x564ba789f390 2024-09-21T01:58:33.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.483+0000 7fd3837fe640 1 -- 172.21.15.29:0/3375436644 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880981c0 con 0x564ba789f390 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.484+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 msgr2=0x564ba7c742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.485+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c742a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3880e4830 tx=0x7fd3880d0040 comp rx=0 tx=0).stop 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.485+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 shutdown_connections 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.485+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78c0070 0x564ba7c747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.485+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c742a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.485+0000 7fd3a6671740 1 --2- 172.21.15.29:0/3375436644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78971b0 0x564ba7c68d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.486+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 >> 172.21.15.29:0/3375436644 conn(0x564ba78b4460 msgr2=0x564ba7c68680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.489+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 shutdown_connections 2024-09-21T01:58:33.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.490+0000 7fd3a6671740 1 -- 172.21.15.29:0/3375436644 wait complete. 2024-09-21T01:58:33.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.490+0000 7fd3a6671740 1 -- start start 2024-09-21T01:58:33.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.491+0000 7fd3a6671740 1 Processor -- start 2024-09-21T01:58:33.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.491+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd3a6671740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd39de99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd39de99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35138/0 (socket says 172.21.15.29:35138) 2024-09-21T01:58:33.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd39d698640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd39d698640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49632/0 (socket says 172.21.15.29:49632) 2024-09-21T01:58:33.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd39de99640 1 -- 172.21.15.29:0/1346221264 learned_addr learned my addr 172.21.15.29:0/1346221264 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:33.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.492+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7cebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981ae690 con 0x564ba78c0070 2024-09-21T01:58:33.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981adfc0 con 0x564ba78971b0 2024-09-21T01:58:33.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981ab200 con 0x564ba789f390 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd3a6671740 10 client.0 _ll_register_callbacks cb 0x7fd3880ea680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd39de99640 1 -- 172.21.15.29:0/1346221264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 msgr2=0x564ba7c7d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd39de99640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd39de99640 1 -- 172.21.15.29:0/1346221264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 msgr2=0x564ba7cebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd39de99640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7cebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.493+0000 7fd39de99640 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba7cec380 con 0x564ba78971b0 2024-09-21T01:58:33.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.494+0000 7fd39d698640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:33.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.496+0000 7fd39de99640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd394009250 tx=0x7fd394004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9755a555925ef1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:33.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.497+0000 7fd3817fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:33.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.497+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394009e40 con 0x564ba78971b0 2024-09-21T01:58:33.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.497+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3880e8040 con 0x564ba78971b0 2024-09-21T01:58:33.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.497+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd39402fb50 con 0x564ba78971b0 2024-09-21T01:58:33.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.498+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd39402fd50 con 0x564ba78971b0 2024-09-21T01:58:33.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.499+0000 7fd3a6671740 10 client.27350 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:33.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.499+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564ba7cf3740 con 0x564ba78971b0 2024-09-21T01:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.501+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3940044f0 con 0x564ba78971b0 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fd39402f530 con 0x564ba78971b0 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 10 client.27350 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 10 client.27350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 20 client.27350 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba7853da0 0x564ba7cf5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 4 client.27350 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:58:33.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.502+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd3880e8040 con 0x564ba7853da0 2024-09-21T01:58:33.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.506+0000 7fd39d698640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba7853da0 0x564ba7cf5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.508+0000 7fd39d698640 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba7853da0 0x564ba7cf5e90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60ee0a99d70b6 server_cookie=31b83b9e25c3cbf8 in_seq=0 out_seq=0 2024-09-21T01:58:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.508+0000 7fd3817fa640 10 client.27350 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.508+0000 7fd3817fa640 1 -- 172.21.15.29:0/1346221264 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x7fd3880e8040 con 0x564ba7853da0 2024-09-21T01:58:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.508+0000 7fd3817fa640 10 client.27350 handle_command_reply: tid=0 2024-09-21T01:58:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.508+0000 7fd3a6671740 1 client.27350 shutdown 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 msgr2=0x564ba7c7d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd394009250 tx=0x7fd394004cf0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 shutdown_connections 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564ba7853da0 0x564ba7cf5e90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564ba78c0070 0x564ba7cebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.509+0000 7fd3817fa640 0 client.27350 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.510+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ba789f390 0x564ba7c7d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.510+0000 7fd3a6671740 1 --2- 172.21.15.29:0/1346221264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ba78971b0 0x564ba7c7d060 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.510+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 >> 172.21.15.29:0/1346221264 conn(0x564ba78b4460 msgr2=0x564ba7c82410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.510+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 shutdown_connections 2024-09-21T01:58:33.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.510+0000 7fd3a6671740 1 -- 172.21.15.29:0/1346221264 wait complete. 2024-09-21T01:58:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.512+0000 7fd3a6671740 20 client.27350 trim_cache size 0 max 16384 2024-09-21T01:58:33.513 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:58:33.513 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:58:33.513 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.514 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.515 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:58:33.516 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:58:33.517 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:33.520 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.515+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 msgr2=0x7fd3840b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.515+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 0x7fd3840b8ad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd390006010 tx=0x7fd390005d80 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.515+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 msgr2=0x7fd39819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.515+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd38800d770 tx=0x7fd38800dc30 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 shutdown_connections 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd3840b65c0 0x7fd3840b8ad0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd398108cf0 0x7fd39819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3981083a0 0x7fd39819b780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 --2- 172.21.15.29:0/1950500054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd398102a60 0x7fd39819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 >> 172.21.15.29:0/1950500054 conn(0x7fd398079730 msgr2=0x7fd398104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 shutdown_connections 2024-09-21T01:58:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:33.516+0000 7fd3a00fd640 1 -- 172.21.15.29:0/1950500054 wait complete. 2024-09-21T01:58:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:33 smithi066 ceph-mon[35001]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T01:58:33.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:33 smithi029 ceph-mon[31228]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T01:58:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:33 smithi190 ceph-mon[34368]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T01:58:34.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:58:34.488 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:58:34.887 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:35.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 -- 172.21.15.29:0/2091922698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 msgr2=0x7f18c410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:35.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 --2- 172.21.15.29:0/2091922698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c410a3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009ca0 tx=0x7f18bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 -- 172.21.15.29:0/2091922698 shutdown_connections 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 --2- 172.21.15.29:0/2091922698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c411cc30 0x7f18c411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 --2- 172.21.15.29:0/2091922698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c410a9e0 0x7f18c410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 --2- 172.21.15.29:0/2091922698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c410a3c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 -- 172.21.15.29:0/2091922698 >> 172.21.15.29:0/2091922698 conn(0x7f18c4070ec0 msgr2=0x7f18c40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.457+0000 7f18c9817640 1 -- 172.21.15.29:0/2091922698 shutdown_connections 2024-09-21T01:58:35.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.458+0000 7f18c9817640 1 -- 172.21.15.29:0/2091922698 wait complete. 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.458+0000 7f18c9817640 1 Processor -- start 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 -- start start 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c41a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 0x7f18c41aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18c41228b0 con 0x7f18c410a9e0 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4122730 con 0x7f18c411cc30 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c9817640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4122a30 con 0x7f18c4111f40 2024-09-21T01:58:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c41a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47284/0 (socket says 172.21.15.29:47284) 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c2ffd640 1 -- 172.21.15.29:0/2062210485 learned_addr learned my addr 172.21.15.29:0/2062210485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.459+0000 7f18c37fe640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 0x7f18c41aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 -- 172.21.15.29:0/2062210485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 msgr2=0x7f18c41a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c41a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 -- 172.21.15.29:0/2062210485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 msgr2=0x7f18c41aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 0x7f18c41aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 -- 172.21.15.29:0/2062210485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c41af330 con 0x7f18c410a9e0 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c27fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c41a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c37fe640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 0x7f18c41aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:35.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.460+0000 7f18c2ffd640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f18bc033040 tx=0x7f18bc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.461+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc003b70 con 0x7f18c410a9e0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.461+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18bc003d10 con 0x7f18c410a9e0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.461+0000 7f18c9817640 1 -- 172.21.15.29:0/2062210485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c41af5c0 con 0x7f18c410a9e0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.461+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc02f9e0 con 0x7f18c410a9e0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.461+0000 7f18c9817640 1 -- 172.21.15.29:0/2062210485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c41b5d80 con 0x7f18c410a9e0 2024-09-21T01:58:35.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.463+0000 7f18c9817640 1 -- 172.21.15.29:0/2062210485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c410adf0 con 0x7f18c410a9e0 2024-09-21T01:58:35.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.466+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f18bc02fb80 con 0x7f18c410a9e0 2024-09-21T01:58:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.466+0000 7f18c8815640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 0x7f18ac0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.467+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f18bc108630 con 0x7f18c410a9e0 2024-09-21T01:58:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.467+0000 7f18c27fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 0x7f18ac0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:35.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.467+0000 7f18c27fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 0x7f18ac0b8b70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c41112f0 tx=0x7f18b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:35.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.471+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18bc0ceef0 con 0x7f18c410a9e0 2024-09-21T01:58:35.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:35 smithi029 ceph-mon[31228]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-09-21T01:58:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:35 smithi066 ceph-mon[35001]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-09-21T01:58:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.825+0000 7f18c9817640 1 -- 172.21.15.29:0/2062210485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18c41b6020 con 0x7f18c410a9e0 2024-09-21T01:58:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.834+0000 7f18c8815640 1 -- 172.21.15.29:0/2062210485 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11970 (secure 0 0 0) 0x7f18bc0d32d0 con 0x7f18c410a9e0 2024-09-21T01:58:35.835 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:58:35.835 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"btime":"2024-09-21T01:58:01:441472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:58:01.260733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:58:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 45 2024-09-21T01:58:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.839+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 msgr2=0x7f18ac0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.839+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 0x7f18ac0b8b70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c41112f0 tx=0x7f18b8005f90 comp rx=0 tx=0).stop 2024-09-21T01:58:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.839+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 msgr2=0x7f18c41a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.839+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f18bc033040 tx=0x7f18bc0316f0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 shutdown_connections 2024-09-21T01:58:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18ac0b6660 0x7f18ac0b8b70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18c411cc30 0x7f18c41aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c4111f40 0x7f18c41a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 --2- 172.21.15.29:0/2062210485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18c410a9e0 0x7f18c41a93c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.843+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 >> 172.21.15.29:0/2062210485 conn(0x7f18c4070ec0 msgr2=0x7f18c4071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.849+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 shutdown_connections 2024-09-21T01:58:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:35.849+0000 7f18aa7fc640 1 -- 172.21.15.29:0/2062210485 wait complete. 2024-09-21T01:58:35.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:35 smithi190 ceph-mon[34368]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 -- 172.21.15.29:0/3737567954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410b6e0 msgr2=0x7f5574112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3737567954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410b6e0 0x7f5574112080 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5564009860 tx=0x7f556402a340 comp rx=0 tx=0).stop 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 -- 172.21.15.29:0/3737567954 shutdown_connections 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3737567954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410b6e0 0x7f5574112080 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3737567954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f557410ad40 0x7f557410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3737567954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5574109cc0 0x7f557410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 -- 172.21.15.29:0/3737567954 >> 172.21.15.29:0/3737567954 conn(0x7f5574079050 msgr2=0x7f557407b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 -- 172.21.15.29:0/3737567954 shutdown_connections 2024-09-21T01:58:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.107+0000 7f557aa0a640 1 -- 172.21.15.29:0/3737567954 wait complete. 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.108+0000 7f557aa0a640 1 Processor -- start 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.108+0000 7f557aa0a640 1 -- start start 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.108+0000 7f557aa0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5574109cc0 0x7f55741a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f557aa0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f557aa0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557410b6e0 0x7f55741a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f557aa0a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55741158f0 con 0x7f5574109cc0 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f55737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f55737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49662/0 (socket says 172.21.15.29:49662) 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f55737fe640 1 -- 172.21.15.29:0/3953915036 learned_addr learned my addr 172.21.15.29:0/3953915036 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5574115770 con 0x7f557410b6e0 2024-09-21T01:58:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5574115a70 con 0x7f557410ad40 2024-09-21T01:58:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.109+0000 7f5573fff640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5574109cc0 0x7f55741a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 -- 172.21.15.29:0/3953915036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557410b6e0 msgr2=0x7f55741a1700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557410b6e0 0x7f55741a1700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 -- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5574109cc0 msgr2=0x7f55741a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5574109cc0 0x7f55741a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 -- 172.21.15.29:0/3953915036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741a1f20 con 0x7f557410ad40 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55737fe640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9c0 tx=0x7f556800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.110+0000 7f55717fa640 1 -- 172.21.15.29:0/3953915036 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556800cde0 con 0x7f557410ad40 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.111+0000 7f55717fa640 1 -- 172.21.15.29:0/3953915036 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55680035f0 con 0x7f557410ad40 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.111+0000 7f55717fa640 1 -- 172.21.15.29:0/3953915036 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568010690 con 0x7f557410ad40 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.111+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741a2210 con 0x7f557410ad40 2024-09-21T01:58:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.111+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741a8900 con 0x7f557410ad40 2024-09-21T01:58:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f55717fa640 1 -- 172.21.15.29:0/3953915036 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5568004000 con 0x7f557410ad40 2024-09-21T01:58:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 Processor -- start 2024-09-21T01:58:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 -- start start 2024-09-21T01:58:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cc04c8b6f0 con 0x55cc049504d0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cc04c7ab20 con 0x55cc049482f0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.114+0000 7f5581a53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc04c7e930 con 0x55cc049711b0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5573fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49676/0 (socket says 172.21.15.29:49676) 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 -- 172.21.15.29:0/336051003 learned_addr learned my addr 172.21.15.29:0/336051003 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5573fff640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d194c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35172/0 (socket says 172.21.15.29:35172) 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f55737fe640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 -- 172.21.15.29:0/336051003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 msgr2=0x55cc04d194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 -- 172.21.15.29:0/336051003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 msgr2=0x55cc04d1a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d1a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 -- 172.21.15.29:0/336051003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc04c40530 con 0x55cc049711b0 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f55737fe640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d1a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f5578fa7640 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f556402a930 tx=0x7f5564005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9fb2deeab43b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.115+0000 7f55717fa640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 0x7f555c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.116+0000 7f555a7fc640 1 -- 172.21.15.29:0/336051003 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564037040 con 0x55cc049711b0 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.116+0000 7f555a7fc640 1 -- 172.21.15.29:0/336051003 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5564030550 con 0x55cc049711b0 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.116+0000 7f5573fff640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 0x7f555c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.116+0000 7f55717fa640 1 -- 172.21.15.29:0/3953915036 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f55680e7670 con 0x7f557410ad40 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.116+0000 7f555a7fc640 1 -- 172.21.15.29:0/336051003 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564005020 con 0x55cc049711b0 2024-09-21T01:58:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 -- 172.21.15.29:0/336051003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 msgr2=0x55cc04d18e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f556402a930 tx=0x7f5564005c90 comp rx=0 tx=0).stop 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5573fff640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 0x7f555c0b8b20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f556c005fd0 tx=0x7f556c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 -- 172.21.15.29:0/336051003 shutdown_connections 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d1a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 --2- 172.21.15.29:0/336051003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d18e50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 -- 172.21.15.29:0/336051003 >> 172.21.15.29:0/336051003 conn(0x55cc04965300 msgr2=0x55cc04d148e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 -- 172.21.15.29:0/336051003 shutdown_connections 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.117+0000 7f5581a53740 1 -- 172.21.15.29:0/336051003 wait complete. 2024-09-21T01:58:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.118+0000 7f5581a53740 1 -- start start 2024-09-21T01:58:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.118+0000 7f5581a53740 1 Processor -- start 2024-09-21T01:58:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.118+0000 7f5581a53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.118+0000 7f5581a53740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.118+0000 7f5581a53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5581a53740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cc04bb2ab0 con 0x55cc049504d0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5581a53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cc04c20730 con 0x55cc049482f0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5581a53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc04d1ab70 con 0x55cc049711b0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35188/0 (socket says 172.21.15.29:35188) 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 -- 172.21.15.29:0/3657150601 learned_addr learned my addr 172.21.15.29:0/3657150601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f55737fe640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5581a53740 10 client.0 _ll_register_callbacks cb 0x55cc04c746b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5578fa7640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 -- 172.21.15.29:0/3657150601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 msgr2=0x55cc04d237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 -- 172.21.15.29:0/3657150601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 msgr2=0x55cc04d22c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d22c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 -- 172.21.15.29:0/3657150601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc049dc790 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5578fa7640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5573fff640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f556c012010 tx=0x7f556c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1072f809965e38b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f55537fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00f830 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f556c00f9d0 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.119+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5564030550 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f556c00fbd0 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 10 client.27365 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cc04a55620 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5564030550 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f556c026070 con 0x55cc049482f0 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 10 client.27365 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 20 client.27365 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc0492fd80 0x55cc04d2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 4 client.27365 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc04c8d640 con 0x55cc0492fd80 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.120+0000 7f55737fe640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc0492fd80 0x55cc04d2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.121+0000 7f55537fe640 10 client.27365 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.121+0000 7f55737fe640 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc0492fd80 0x55cc04d2fd70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d049a34263d79edd server_cookie=5173e304d8ac8898 in_seq=0 out_seq=0 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f55537fe640 1 -- 172.21.15.29:0/3657150601 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc04c8d640 con 0x55cc0492fd80 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f55537fe640 10 client.27365 handle_command_reply: tid=0 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 client.27365 shutdown 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 msgr2=0x55cc04d225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f556c012010 tx=0x7f556c004ab0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 shutdown_connections 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc0492fd80 0x55cc04d2fd70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049711b0 0x55cc04d237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 --2- 172.21.15.29:0/3657150601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049482f0 0x55cc04d225b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.122+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 >> 172.21.15.29:0/3657150601 conn(0x55cc04965300 msgr2=0x55cc04d12ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.123+0000 7f55537fe640 0 client.27365 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.123+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 shutdown_connections 2024-09-21T01:58:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.123+0000 7f5581a53740 1 -- 172.21.15.29:0/3657150601 wait complete. 2024-09-21T01:58:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.124+0000 7f5581a53740 20 client.27365 trim_cache size 0 max 16384 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 Processor -- start 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 -- start start 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049711b0 0x55cc04d21050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55741b1420 con 0x55cc049504d0 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55741b2f60 con 0x55cc049711b0 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5581a53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55741b1af0 con 0x55cc049482f0 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f55737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5573fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f55737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47330/0 (socket says 172.21.15.29:47330) 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f55737fe640 1 -- 172.21.15.29:0/4069754577 learned_addr learned my addr 172.21.15.29:0/4069754577 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5573fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49682/0 (socket says 172.21.15.29:49682) 2024-09-21T01:58:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.143+0000 7f5578fa7640 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049711b0 0x55cc04d21050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 -- 172.21.15.29:0/4069754577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049711b0 msgr2=0x55cc04d21050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049711b0 0x55cc04d21050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 -- 172.21.15.29:0/4069754577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 msgr2=0x55cc04d14220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 -- 172.21.15.29:0/4069754577 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc04d0f5c0 con 0x55cc049482f0 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f55737fe640 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5573fff640 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f80 tx=0x7f556c003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fa934e21cfd19d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5559ffb640 1 -- 172.21.15.29:0/4069754577 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00fa30 con 0x55cc049482f0 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5559ffb640 1 -- 172.21.15.29:0/4069754577 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c037030 con 0x55cc049482f0 2024-09-21T01:58:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.144+0000 7f5559ffb640 1 -- 172.21.15.29:0/4069754577 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c005280 con 0x55cc049482f0 2024-09-21T01:58:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 -- 172.21.15.29:0/4069754577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 msgr2=0x55cc04d2fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f80 tx=0x7f556c003fa0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 -- 172.21.15.29:0/4069754577 shutdown_connections 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049711b0 0x55cc04d21050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049504d0 0x55cc04d14220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.145+0000 7f5581a53740 1 --2- 172.21.15.29:0/4069754577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d2fd10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.146+0000 7f5581a53740 1 -- 172.21.15.29:0/4069754577 >> 172.21.15.29:0/4069754577 conn(0x55cc04965300 msgr2=0x55cc04d19e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.146+0000 7f5581a53740 1 -- 172.21.15.29:0/4069754577 shutdown_connections 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.146+0000 7f5581a53740 1 -- 172.21.15.29:0/4069754577 wait complete. 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.146+0000 7f5581a53740 1 -- start start 2024-09-21T01:58:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.147+0000 7f5581a53740 1 Processor -- start 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.147+0000 7f5581a53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 0x55cc04d27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049711b0 0x55cc04d28440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5573fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f55737fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 0x55cc04d27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5573fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49686/0 (socket says 172.21.15.29:49686) 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f55737fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 0x55cc04d27920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35218/0 (socket says 172.21.15.29:35218) 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5573fff640 1 -- 172.21.15.29:0/733435024 learned_addr learned my addr 172.21.15.29:0/733435024 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5578fa7640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049711b0 0x55cc04d28440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55741b2890 con 0x55cc049711b0 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55741b21c0 con 0x55cc049504d0 2024-09-21T01:58:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55741af400 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.148+0000 7f5581a53740 10 client.0 _ll_register_callbacks cb 0x7f556c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 -- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 msgr2=0x55cc04d27920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 0x55cc04d27920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 -- 172.21.15.29:0/733435024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049711b0 msgr2=0x55cc04d28440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049711b0 0x55cc04d28440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc04d28ba0 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f556c037030 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5573fff640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f556c018040 tx=0x7f556c01ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c2a74dc7e30482 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f55537fe640 10 client.27331 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f55537fe640 1 -- 172.21.15.29:0/733435024 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c04a050 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f55537fe640 1 -- 172.21.15.29:0/733435024 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f556c004c40 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f55537fe640 1 -- 172.21.15.29:0/733435024 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f556c040070 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5581a53740 10 client.27331 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.149+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55cc04d29df0 con 0x55cc049482f0 2024-09-21T01:58:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f55537fe640 1 -- 172.21.15.29:0/733435024 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f556c026040 con 0x55cc049482f0 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 10 client.27331 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 10 client.27331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 20 client.27331 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc049051c0 0x55cc04d2a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 4 client.27331 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc04c890f0 con 0x55cc049051c0 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f55737fe640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc049051c0 0x55cc04d2a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f55537fe640 10 client.27331 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.150+0000 7f55737fe640 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc049051c0 0x55cc04d2a6c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979b8e9aecf2f0dc server_cookie=cf50f0da58a8d3ce in_seq=0 out_seq=0 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f55537fe640 1 -- 172.21.15.29:0/733435024 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc04c890f0 con 0x55cc049051c0 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f55537fe640 10 client.27331 handle_command_reply: tid=0 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 client.27331 shutdown 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 msgr2=0x55cc04d27260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f556c018040 tx=0x7f556c01ee80 comp rx=0 tx=0).stop 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 shutdown_connections 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55cc049051c0 0x55cc04d2a6c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cc049711b0 0x55cc04d28440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc049504d0 0x55cc04d27920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f55537fe640 0 client.27331 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 --2- 172.21.15.29:0/733435024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc049482f0 0x55cc04d27260 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 >> 172.21.15.29:0/733435024 conn(0x55cc04965300 msgr2=0x55cc04d19db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 shutdown_connections 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 1 -- 172.21.15.29:0/733435024 wait complete. 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.164+0000 7f5581a53740 20 client.27331 trim_cache size 0 max 16384 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:58:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 msgr2=0x7f555c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 0x7f555c0b8b20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f556c005fd0 tx=0x7f556c009500 comp rx=0 tx=0).stop 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 msgr2=0x7f55741a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9c0 tx=0x7f556800ee80 comp rx=0 tx=0).stop 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 shutdown_connections 2024-09-21T01:58:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f555c0b6610 0x7f555c0b8b20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557410b6e0 0x7f55741a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557410ad40 0x7f55741a0b50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 --2- 172.21.15.29:0/3953915036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5574109cc0 0x7f55741a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 >> 172.21.15.29:0/3953915036 conn(0x7f5574079050 msgr2=0x7f557410e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 shutdown_connections 2024-09-21T01:58:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:36.165+0000 7f557aa0a640 1 -- 172.21.15.29:0/3953915036 wait complete. 2024-09-21T01:58:36.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:36 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:36.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:36 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2062210485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:58:36.677 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:58:36.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:36 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:36.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:36 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2062210485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:58:36.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:36 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:36.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:36 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2062210485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:58:37.086 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.842 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:37 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 -- 172.21.15.29:0/4097191777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c10de30 msgr2=0x7fc30c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 --2- 172.21.15.29:0/4097191777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c10de30 0x7fc30c110300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc304009ca0 tx=0x7fc30402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 -- 172.21.15.29:0/4097191777 shutdown_connections 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 --2- 172.21.15.29:0/4097191777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c10de30 0x7fc30c110300 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 --2- 172.21.15.29:0/4097191777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c1052b0 0x7fc30c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 --2- 172.21.15.29:0/4097191777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c104880 0x7fc30c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 -- 172.21.15.29:0/4097191777 >> 172.21.15.29:0/4097191777 conn(0x7fc30c079730 msgr2=0x7fc30c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 -- 172.21.15.29:0/4097191777 shutdown_connections 2024-09-21T01:58:37.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.847+0000 7fc310940640 1 -- 172.21.15.29:0/4097191777 wait complete. 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.848+0000 7fc310940640 1 Processor -- start 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.848+0000 7fc310940640 1 -- start start 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 0x7fc30c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c10de30 0x7fc30c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c113a70 con 0x7fc30c104880 2024-09-21T01:58:37.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1138f0 con 0x7fc30c1052b0 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc310940640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c113bf0 con 0x7fc30c10de30 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 0x7fc30c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 0x7fc30c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47362/0 (socket says 172.21.15.29:47362) 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35222/0 (socket says 172.21.15.29:35222) 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc30a59c640 1 -- 172.21.15.29:0/3720524964 learned_addr learned my addr 172.21.15.29:0/3720524964 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc30ad9d640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c10de30 0x7fc30c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 -- 172.21.15.29:0/3720524964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c10de30 msgr2=0x7fc30c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c10de30 0x7fc30c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 -- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 msgr2=0x7fc30c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 0x7fc30c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.849+0000 7fc309d9b640 1 -- 172.21.15.29:0/3720524964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c1a0390 con 0x7fc30c1052b0 2024-09-21T01:58:37.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc309d9b640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc30000d6a0 tx=0x7fc30000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc2fb7fe640 1 -- 172.21.15.29:0/3720524964 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300017070 con 0x7fc30c1052b0 2024-09-21T01:58:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc2fb7fe640 1 -- 172.21.15.29:0/3720524964 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc300009820 con 0x7fc30c1052b0 2024-09-21T01:58:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c1a0680 con 0x7fc30c1052b0 2024-09-21T01:58:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc2fb7fe640 1 -- 172.21.15.29:0/3720524964 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300004c40 con 0x7fc30c1052b0 2024-09-21T01:58:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.850+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c102670 con 0x7fc30c1052b0 2024-09-21T01:58:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.855+0000 7fc2fb7fe640 1 -- 172.21.15.29:0/3720524964 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc300005220 con 0x7fc30c1052b0 2024-09-21T01:58:37.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.856+0000 7fc2fb7fe640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 0x7fc2f40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.856+0000 7fc316eb4740 1 Processor -- start 2024-09-21T01:58:37.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.856+0000 7fc2fb7fe640 1 -- 172.21.15.29:0/3720524964 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc3000e6ab0 con 0x7fc30c1052b0 2024-09-21T01:58:37.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.856+0000 7fc30a59c640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 0x7fc2f40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 -- start start 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a92f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a92fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a930120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56468a8a5ea0 con 0x56468a588070 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56468a8952d0 con 0x56468a567390 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc316eb4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56468a8990e0 con 0x56468a55f1b0 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a92f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc30ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a930120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc30ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a930120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35232/0 (socket says 172.21.15.29:35232) 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc30ad9d640 1 -- 172.21.15.29:0/3221221989 learned_addr learned my addr 172.21.15.29:0/3221221989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.857+0000 7fc30a59c640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 0x7fc2f40b8a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc309d9b640 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a92fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 -- 172.21.15.29:0/3221221989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 msgr2=0x56468a92fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a92fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 -- 172.21.15.29:0/3221221989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 msgr2=0x56468a930120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a930120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 -- 172.21.15.29:0/3221221989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a5f3650 con 0x56468a588070 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.858+0000 7fc30a59c640 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a92f6a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc010c30 tx=0x7fc2fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547cc1e3c349152b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.859+0000 7fc2f8ff9640 1 -- 172.21.15.29:0/3221221989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc026670 con 0x56468a588070 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.859+0000 7fc2f8ff9640 1 -- 172.21.15.29:0/3221221989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc021070 con 0x56468a588070 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.859+0000 7fc2f8ff9640 1 -- 172.21.15.29:0/3221221989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc01c780 con 0x56468a588070 2024-09-21T01:58:37.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 -- 172.21.15.29:0/3221221989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 msgr2=0x56468a92f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a92f6a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc010c30 tx=0x7fc2fc01b000 comp rx=0 tx=0).stop 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 -- 172.21.15.29:0/3221221989 shutdown_connections 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a930120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a92fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 --2- 172.21.15.29:0/3221221989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a92f6a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 -- 172.21.15.29:0/3221221989 >> 172.21.15.29:0/3221221989 conn(0x56468a57c460 msgr2=0x56468a92d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 -- 172.21.15.29:0/3221221989 shutdown_connections 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.860+0000 7fc316eb4740 1 -- 172.21.15.29:0/3221221989 wait complete. 2024-09-21T01:58:37.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.861+0000 7fc316eb4740 1 -- start start 2024-09-21T01:58:37.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.861+0000 7fc316eb4740 1 Processor -- start 2024-09-21T01:58:37.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.862+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a93ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.862+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 0x56468a93b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.862+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.862+0000 7fc316eb4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56468a6f6700 con 0x56468a567390 2024-09-21T01:58:37.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35234/0 (socket says 172.21.15.29:35234) 2024-09-21T01:58:37.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 -- 172.21.15.29:0/385160617 learned_addr learned my addr 172.21.15.29:0/385160617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:37.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56468a830160 con 0x56468a588070 2024-09-21T01:58:37.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56468a93de80 con 0x56468a55f1b0 2024-09-21T01:58:37.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc316eb4740 10 client.0 _ll_register_callbacks cb 0x56468a8499e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30a59c640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a93ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 -- 172.21.15.29:0/385160617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 msgr2=0x56468a93ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a93ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc30ad9d640 1 -- 172.21.15.29:0/385160617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 msgr2=0x56468a93b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.863+0000 7fc309d9b640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 0x56468a93b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.864+0000 7fc30ad9d640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 0x56468a93b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.864+0000 7fc30ad9d640 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a85dc90 con 0x56468a588070 2024-09-21T01:58:37.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.864+0000 7fc309d9b640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 0x56468a93b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:58:37.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.869+0000 7fc30ad9d640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc304031450 tx=0x7fc304031480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9697d13f397a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.869+0000 7fc2f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.872+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30404d060 con 0x56468a588070 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.872+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc304003970 con 0x56468a588070 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.873+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2fc021070 con 0x56468a588070 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.874+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc304003b70 con 0x56468a588070 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.874+0000 7fc316eb4740 10 client.27383 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:37.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.874+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56468a949c10 con 0x56468a588070 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.875+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc304036040 con 0x56468a588070 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.875+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fc3040042f0 con 0x56468a588070 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 10 client.27383 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 10 client.27383 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 20 client.27383 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a5469f0 0x56468a94d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 4 client.27383 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:37.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.876+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2fc021070 con 0x56468a5469f0 2024-09-21T01:58:37.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.879+0000 7fc30a59c640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a5469f0 0x56468a94d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.894+0000 7fc30a59c640 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a5469f0 0x56468a94d0e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b47b1d472ba78be server_cookie=c215911923801a7 in_seq=0 out_seq=0 2024-09-21T01:58:37.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.895+0000 7fc2f1ffb640 10 client.27383 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:58:37.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.897+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/385160617 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc2fc021070 con 0x56468a5469f0 2024-09-21T01:58:37.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.897+0000 7fc2f1ffb640 10 client.27383 handle_command_reply: tid=0 2024-09-21T01:58:37.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.897+0000 7fc316eb4740 1 client.27383 shutdown 2024-09-21T01:58:37.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.903+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 msgr2=0x56468a947170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.903+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc304031450 tx=0x7fc304031480 comp rx=0 tx=0).stop 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 shutdown_connections 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a5469f0 0x56468a94d0e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc2f1ffb640 0 client.27383 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a588070 0x56468a947170 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a567390 0x56468a93b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 --2- 172.21.15.29:0/385160617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a93ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.904+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 >> 172.21.15.29:0/385160617 conn(0x56468a57c460 msgr2=0x56468a932880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.906+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 shutdown_connections 2024-09-21T01:58:37.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.908+0000 7fc316eb4740 1 -- 172.21.15.29:0/385160617 wait complete. 2024-09-21T01:58:37.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.908+0000 7fc316eb4740 20 client.27383 trim_cache size 0 max 16384 2024-09-21T01:58:37.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.937+0000 7fc316eb4740 1 Processor -- start 2024-09-21T01:58:37.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 -- start start 2024-09-21T01:58:37.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a55f1b0 0x56468a932eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a947070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1ad230 con 0x56468a55f1b0 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1aed70 con 0x56468a567390 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.938+0000 7fc316eb4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1ad900 con 0x56468a5469f0 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49720/0 (socket says 172.21.15.29:49720) 2024-09-21T01:58:37.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 -- 172.21.15.29:0/2898066701 learned_addr learned my addr 172.21.15.29:0/2898066701 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 -- 172.21.15.29:0/2898066701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 msgr2=0x56468a947070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a947070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 -- 172.21.15.29:0/2898066701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a55f1b0 msgr2=0x56468a932eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a55f1b0 0x56468a932eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.939+0000 7fc30a59c640 1 -- 172.21.15.29:0/2898066701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a9392e0 con 0x56468a5469f0 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.940+0000 7fc30a59c640 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x56468a931b40 tx=0x7fc2fc005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5e5cf1a6d7c844 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.940+0000 7fc2f3fff640 1 -- 172.21.15.29:0/2898066701 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc005540 con 0x56468a5469f0 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.940+0000 7fc2f3fff640 1 -- 172.21.15.29:0/2898066701 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc021040 con 0x56468a5469f0 2024-09-21T01:58:37.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.940+0000 7fc2f3fff640 1 -- 172.21.15.29:0/2898066701 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc004970 con 0x56468a5469f0 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 -- 172.21.15.29:0/2898066701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 msgr2=0x56468a939cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x56468a931b40 tx=0x7fc2fc005250 comp rx=0 tx=0).stop 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 -- 172.21.15.29:0/2898066701 shutdown_connections 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a947070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a55f1b0 0x56468a932eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 --2- 172.21.15.29:0/2898066701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a5469f0 0x56468a939cd0 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x56468a931b40 tx=0x7fc2fc005250 comp rx=0 tx=0).stop 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 -- 172.21.15.29:0/2898066701 >> 172.21.15.29:0/2898066701 conn(0x56468a57c460 msgr2=0x56468a9327a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.942+0000 7fc316eb4740 1 -- 172.21.15.29:0/2898066701 shutdown_connections 2024-09-21T01:58:37.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.943+0000 7fc316eb4740 1 -- 172.21.15.29:0/2898066701 wait complete. 2024-09-21T01:58:37.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.943+0000 7fc316eb4740 1 -- start start 2024-09-21T01:58:37.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.943+0000 7fc316eb4740 1 Processor -- start 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.944+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.944+0000 7fc316eb4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a943610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc30a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49732/0 (socket says 172.21.15.29:49732) 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc30a59c640 1 -- 172.21.15.29:0/1137451914 learned_addr learned my addr 172.21.15.29:0/1137451914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc309d9b640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a943610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a9b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1ae6a0 con 0x56468a588070 2024-09-21T01:58:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1adfd0 con 0x56468a567390 2024-09-21T01:58:37.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1ab210 con 0x56468a55f1b0 2024-09-21T01:58:37.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.945+0000 7fc316eb4740 10 client.0 _ll_register_callbacks cb 0x7fc30402f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:37.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 -- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 msgr2=0x56468a943610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a943610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 -- 172.21.15.29:0/1137451914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 msgr2=0x56468a9b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a9b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a943d00 con 0x56468a55f1b0 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc309d9b640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a943610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc30a59c640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011d50 tx=0x7fc2fc00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad85c2361b0e84ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56468a9289e0 con 0x56468a55f1b0 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc2f1ffb640 10 client.27343 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:37.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.946+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/1137451914 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc005540 con 0x56468a55f1b0 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.947+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/1137451914 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2fc01d430 con 0x56468a55f1b0 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.947+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/1137451914 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc01d690 con 0x56468a55f1b0 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.947+0000 7fc316eb4740 10 client.27343 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.947+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=45}) -- 0x56468a9d15b0 con 0x56468a55f1b0 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/1137451914 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fc2fc0046f0 con 0x56468a55f1b0 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 10 client.27343 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 10 client.27343 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 20 client.27343 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a51bda0 0x56468a9d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 4 client.27343 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:58:37.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.948+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56468a8a7df0 con 0x56468a51bda0 2024-09-21T01:58:37.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.951+0000 7fc309d9b640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a51bda0 0x56468a9d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:37.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc309d9b640 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a51bda0 0x56468a9d38d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e34895d257a97ff9 server_cookie=6525031b2d4b66cc in_seq=0 out_seq=0 2024-09-21T01:58:37.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc2f1ffb640 10 client.27343 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc2f1ffb640 1 -- 172.21.15.29:0/1137451914 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56468a8a7df0 con 0x56468a51bda0 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc2f1ffb640 10 client.27343 handle_command_reply: tid=0 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc316eb4740 1 client.27343 shutdown 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 msgr2=0x56468a9430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.958+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011d50 tx=0x7fc2fc00bfd0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 shutdown_connections 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x56468a51bda0 0x56468a9d38d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56468a588070 0x56468a9b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc2f1ffb640 0 client.27343 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468a567390 0x56468a943610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 --2- 172.21.15.29:0/1137451914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56468a55f1b0 0x56468a9430d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 >> 172.21.15.29:0/1137451914 conn(0x56468a57c460 msgr2=0x56468a92ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 shutdown_connections 2024-09-21T01:58:37.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 1 -- 172.21.15.29:0/1137451914 wait complete. 2024-09-21T01:58:37.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.959+0000 7fc316eb4740 20 client.27343 trim_cache size 0 max 16384 2024-09-21T01:58:37.961 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 msgr2=0x7fc2f40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 0x7fc2f40b8a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc00b4a0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 msgr2=0x7fc30c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc30000d6a0 tx=0x7fc30000db60 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 shutdown_connections 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc2f40b6510 0x7fc2f40b8a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.960+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc30c10de30 0x7fc30c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.961+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc30c1052b0 0x7fc30c19a9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.961+0000 7fc310940640 1 --2- 172.21.15.29:0/3720524964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c104880 0x7fc30c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.961+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 >> 172.21.15.29:0/3720524964 conn(0x7fc30c079730 msgr2=0x7fc30c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.961+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 shutdown_connections 2024-09-21T01:58:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:37.963+0000 7fc310940640 1 -- 172.21.15.29:0/3720524964 wait complete. 2024-09-21T01:58:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 600.0 s, 1290.0 tps, lat 75.809 ms stddev 87.090 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:37 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-09-21T01:58:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:38.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:37 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:39.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:58:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:39 smithi066 ceph-mon[35001]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T01:58:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:39 smithi029 ceph-mon[31228]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T01:58:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:39 smithi190 ceph-mon[34368]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T01:58:41.140 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:41.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.741+0000 7fde9aec0640 1 -- 172.21.15.29:0/1708888367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 msgr2=0x7fde9410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.741+0000 7fde9aec0640 1 --2- 172.21.15.29:0/1708888367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde9410a3c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde84009860 tx=0x7fde8402a380 comp rx=0 tx=0).stop 2024-09-21T01:58:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.744+0000 7fde9aec0640 1 -- 172.21.15.29:0/1708888367 shutdown_connections 2024-09-21T01:58:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.744+0000 7fde9aec0640 1 --2- 172.21.15.29:0/1708888367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 0x7fde9411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.744+0000 7fde9aec0640 1 --2- 172.21.15.29:0/1708888367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.744+0000 7fde9aec0640 1 --2- 172.21.15.29:0/1708888367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde9410a3c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.744+0000 7fde9aec0640 1 -- 172.21.15.29:0/1708888367 >> 172.21.15.29:0/1708888367 conn(0x7fde94070ec0 msgr2=0x7fde940712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.745+0000 7fde9aec0640 1 -- 172.21.15.29:0/1708888367 shutdown_connections 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.745+0000 7fde9aec0640 1 -- 172.21.15.29:0/1708888367 wait complete. 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.746+0000 7fde9aec0640 1 Processor -- start 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.746+0000 7fde9aec0640 1 -- start start 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.746+0000 7fde9aec0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9411a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9aec0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde94113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9aec0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 0x7fde94113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9aec0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde94122960 con 0x7fde9411cc30 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9aec0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde941227e0 con 0x7fde9410a9e0 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9aec0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde94122ae0 con 0x7fde94111f40 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde98c5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9411a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde98c5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9411a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35280/0 (socket says 172.21.15.29:35280) 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde98c5c640 1 -- 172.21.15.29:0/3979592010 learned_addr learned my addr 172.21.15.29:0/3979592010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde94113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde9945d640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 0x7fde94113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 -- 172.21.15.29:0/3979592010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 msgr2=0x7fde9411a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9411a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 -- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 msgr2=0x7fde94113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 0x7fde94113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.747+0000 7fde93fff640 1 -- 172.21.15.29:0/3979592010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde94114540 con 0x7fde94111f40 2024-09-21T01:58:41.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.749+0000 7fde93fff640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde94113730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde9406c6e0 tx=0x7fde8800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.751+0000 7fde91ffb640 1 -- 172.21.15.29:0/3979592010 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88015020 con 0x7fde94111f40 2024-09-21T01:58:41.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.751+0000 7fde91ffb640 1 -- 172.21.15.29:0/3979592010 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde88011ce0 con 0x7fde94111f40 2024-09-21T01:58:41.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.751+0000 7fde91ffb640 1 -- 172.21.15.29:0/3979592010 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88010da0 con 0x7fde94111f40 2024-09-21T01:58:41.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.751+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde941191f0 con 0x7fde94111f40 2024-09-21T01:58:41.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.751+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde94119650 con 0x7fde94111f40 2024-09-21T01:58:41.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 Processor -- start 2024-09-21T01:58:41.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 -- start start 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c11ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd56390 0x558e6c11f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558e6c094ea0 con 0x558e6bd77070 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558e6c0842d0 con 0x558e6bd4e1b0 2024-09-21T01:58:41.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.755+0000 7fdea1434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558e6c0880e0 con 0x558e6bd56390 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47406/0 (socket says 172.21.15.29:47406) 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 -- 172.21.15.29:0/1810140658 learned_addr learned my addr 172.21.15.29:0/1810140658 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 -- 172.21.15.29:0/1810140658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd56390 msgr2=0x558e6c11f120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd56390 0x558e6c11f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 -- 172.21.15.29:0/1810140658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 msgr2=0x558e6c11ebe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c11ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.756+0000 7fde9945d640 1 -- 172.21.15.29:0/1810140658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6bde2650 con 0x558e6bd77070 2024-09-21T01:58:41.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.757+0000 7fde9945d640 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00f480 tx=0x7fde8c00f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22944e5ee2f9a995 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.758+0000 7fde7affd640 1 -- 172.21.15.29:0/1810140658 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c01d070 con 0x558e6bd77070 2024-09-21T01:58:41.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.759+0000 7fde7affd640 1 -- 172.21.15.29:0/1810140658 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c00cb50 con 0x558e6bd77070 2024-09-21T01:58:41.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.759+0000 7fde7affd640 1 -- 172.21.15.29:0/1810140658 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c018c80 con 0x558e6bd77070 2024-09-21T01:58:41.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.759+0000 7fde91ffb640 1 -- 172.21.15.29:0/3979592010 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fde88018020 con 0x7fde94111f40 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.760+0000 7fde91ffb640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 0x7fde7c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.760+0000 7fdea1434740 1 -- 172.21.15.29:0/1810140658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 msgr2=0x558e6c11e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.760+0000 7fdea1434740 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00f480 tx=0x7fde8c00f940 comp rx=0 tx=0).stop 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 -- 172.21.15.29:0/1810140658 shutdown_connections 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fde91ffb640 1 -- 172.21.15.29:0/3979592010 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde880e6a50 con 0x7fde94111f40 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd56390 0x558e6c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c11ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 --2- 172.21.15.29:0/1810140658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c11e6a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 -- 172.21.15.29:0/1810140658 >> 172.21.15.29:0/1810140658 conn(0x558e6bd6b460 msgr2=0x558e6c11c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fde98c5c640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 0x7fde7c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 -- 172.21.15.29:0/1810140658 shutdown_connections 2024-09-21T01:58:41.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.761+0000 7fdea1434740 1 -- 172.21.15.29:0/1810140658 wait complete. 2024-09-21T01:58:41.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.762+0000 7fdea1434740 1 -- start start 2024-09-21T01:58:41.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.762+0000 7fde98c5c640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 0x7fde7c0b8c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fde8402a970 tx=0x7fde8402a880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.762+0000 7fdea1434740 1 Processor -- start 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd4e1b0 0x558e6c129d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd56390 0x558e6c12a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558e6bee5700 con 0x558e6bd77070 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558e6c01f160 con 0x558e6bd56390 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558e6c12ce80 con 0x558e6bd4e1b0 2024-09-21T01:58:41.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.763+0000 7fdea1434740 10 client.0 _ll_register_callbacks cb 0x558e6c0389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:41.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde98c5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd4e1b0 0x558e6c129d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47420/0 (socket says 172.21.15.29:47420) 2024-09-21T01:58:41.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 -- 172.21.15.29:0/3902369058 learned_addr learned my addr 172.21.15.29:0/3902369058 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:41.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde93fff640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd56390 0x558e6c12a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 -- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd4e1b0 msgr2=0x558e6c129d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd4e1b0 0x558e6c129d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 -- 172.21.15.29:0/3902369058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd56390 msgr2=0x558e6c12a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.764+0000 7fde9945d640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd56390 0x558e6c12a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.765+0000 7fde9945d640 1 -- 172.21.15.29:0/3902369058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6c04cc90 con 0x558e6bd77070 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.765+0000 7fde9945d640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000f80 tx=0x7fde8c018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e69d8ce748ab80a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.765+0000 7fde73fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c01d070 con 0x558e6bd77070 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde8c00c930 con 0x558e6bd77070 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c00cb50 con 0x558e6bd77070 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fde8c018a20 con 0x558e6bd77070 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fdea1434740 10 client.17598 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:41.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.766+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e6c138b50 con 0x558e6bd77070 2024-09-21T01:58:41.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c00cb50 con 0x558e6bd77070 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fde8c00b040 con 0x558e6bd77070 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 10 client.17598 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 10 client.17598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 20 client.17598 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd359f0 0x558e6c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 4 client.17598 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.767+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6c096df0 con 0x558e6bd359f0 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.768+0000 7fde98c5c640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd359f0 0x558e6c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.768+0000 7fde73fff640 10 client.17598 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:58:41.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.768+0000 7fde98c5c640 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd359f0 0x558e6c13bf10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10c2f5c71c20335e server_cookie=da81908bf66edd17 in_seq=0 out_seq=0 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fde73fff640 1 -- 172.21.15.29:0/3902369058 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e6c096df0 con 0x558e6bd359f0 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fde73fff640 10 client.17598 handle_command_reply: tid=0 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 client.17598 shutdown 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 msgr2=0x558e6c136170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000f80 tx=0x7fde8c018640 comp rx=0 tx=0).stop 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 shutdown_connections 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd77070 0x558e6c136170 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd56390 0x558e6c12a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd359f0 0x558e6c13bf10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 --2- 172.21.15.29:0/3902369058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd4e1b0 0x558e6c129d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 >> 172.21.15.29:0/3902369058 conn(0x558e6bd6b460 msgr2=0x558e6c11f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.769+0000 7fde73fff640 0 client.17598 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:58:41.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.770+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 shutdown_connections 2024-09-21T01:58:41.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.770+0000 7fdea1434740 1 -- 172.21.15.29:0/3902369058 wait complete. 2024-09-21T01:58:41.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.770+0000 7fdea1434740 20 client.17598 trim_cache size 0 max 16384 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.791+0000 7fdea1434740 1 Processor -- start 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.791+0000 7fdea1434740 1 -- start start 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c1360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c12b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c6b40 con 0x558e6bd56390 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c8680 con 0x558e6bd4e1b0 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fdea1434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c7210 con 0x558e6bd77070 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fde98c5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fde98c5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:35320/0 (socket says 172.21.15.29:35320) 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fde98c5c640 1 -- 172.21.15.29:0/2324349906 learned_addr learned my addr 172.21.15.29:0/2324349906 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.792+0000 7fde9945d640 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c12b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 -- 172.21.15.29:0/2324349906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 msgr2=0x558e6c12b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c12b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 -- 172.21.15.29:0/2324349906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 msgr2=0x558e6c1360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c1360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 -- 172.21.15.29:0/2324349906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6c12afe0 con 0x558e6bd4e1b0 2024-09-21T01:58:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde98c5c640 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde84009980 tx=0x7fde8403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f47261af3504c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.793+0000 7fde7a7fc640 1 -- 172.21.15.29:0/2324349906 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84037040 con 0x558e6bd4e1b0 2024-09-21T01:58:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.794+0000 7fde7a7fc640 1 -- 172.21.15.29:0/2324349906 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde84038040 con 0x558e6bd4e1b0 2024-09-21T01:58:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.794+0000 7fde7a7fc640 1 -- 172.21.15.29:0/2324349906 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840326a0 con 0x558e6bd4e1b0 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- 172.21.15.29:0/2324349906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 msgr2=0x558e6c12a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde84009980 tx=0x7fde8403d000 comp rx=0 tx=0).stop 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- 172.21.15.29:0/2324349906 shutdown_connections 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c12b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c1360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 --2- 172.21.15.29:0/2324349906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c12a1e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- 172.21.15.29:0/2324349906 >> 172.21.15.29:0/2324349906 conn(0x558e6bd6b460 msgr2=0x558e6c129ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- 172.21.15.29:0/2324349906 shutdown_connections 2024-09-21T01:58:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- 172.21.15.29:0/2324349906 wait complete. 2024-09-21T01:58:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 -- start start 2024-09-21T01:58:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.795+0000 7fdea1434740 1 Processor -- start 2024-09-21T01:58:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c126a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c127010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c7fb0 con 0x558e6bd56390 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c78e0 con 0x558e6bd4e1b0 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde941c4b20 con 0x558e6bd77070 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fdea1434740 10 client.0 _ll_register_callbacks cb 0x7fde8c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fde9945d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fde9945d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:49802/0 (socket says 172.21.15.29:49802) 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fde9945d640 1 -- 172.21.15.29:0/607835292 learned_addr learned my addr 172.21.15.29:0/607835292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fde93fff640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c127010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.796+0000 7fde98c5c640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c126a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 -- 172.21.15.29:0/607835292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 msgr2=0x558e6c126a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c126a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 -- 172.21.15.29:0/607835292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 msgr2=0x558e6c127010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c127010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 -- 172.21.15.29:0/607835292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6c19dcc0 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde98c5c640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c126a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde9945d640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fde8c002230 tx=0x7fde8c009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34078658e3ed1965 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde73fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c01d040 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fde8c018bc0 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.797+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde84038040 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fde8c00cb40 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 10 client.27358 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e6c1a6150 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c03d030 con 0x558e6bd77070 2024-09-21T01:58:41.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fde8c00b040 con 0x558e6bd77070 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 10 client.27358 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 10 client.27358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 20 client.27358 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd0ada0 0x558e6c1a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 4 client.27358 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.798+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fde84038040 con 0x558e6bd0ada0 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.799+0000 7fde98c5c640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd0ada0 0x558e6c1a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.799+0000 7fde73fff640 10 client.27358 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.799+0000 7fde98c5c640 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd0ada0 0x558e6c1a8850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cae74921014a6728 server_cookie=61b6afa13aafd403 in_seq=0 out_seq=0 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.799+0000 7fde73fff640 1 -- 172.21.15.29:0/607835292 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fde84038040 con 0x558e6bd0ada0 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.799+0000 7fde73fff640 10 client.27358 handle_command_reply: tid=0 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 client.27358 shutdown 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 msgr2=0x558e6c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fde8c002230 tx=0x7fde8c009e10 comp rx=0 tx=0).stop 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 shutdown_connections 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x558e6bd0ada0 0x558e6c1a8850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558e6bd77070 0x558e6c19d650 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fde73fff640 0 client.27358 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558e6bd56390 0x558e6c127010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.800+0000 7fdea1434740 1 --2- 172.21.15.29:0/607835292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e6bd4e1b0 0x558e6c126a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.801+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 >> 172.21.15.29:0/607835292 conn(0x558e6bd6b460 msgr2=0x558e6c12b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.801+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 shutdown_connections 2024-09-21T01:58:41.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.801+0000 7fdea1434740 1 -- 172.21.15.29:0/607835292 wait complete. 2024-09-21T01:58:41.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.801+0000 7fdea1434740 20 client.27358 trim_cache size 0 max 16384 2024-09-21T01:58:41.804 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 msgr2=0x7fde7c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 0x7fde7c0b8c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fde8402a970 tx=0x7fde8402a880 comp rx=0 tx=0).stop 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 msgr2=0x7fde94113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde94113730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde9406c6e0 tx=0x7fde8800b850 comp rx=0 tx=0).stop 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 shutdown_connections 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fde7c0b6720 0x7fde7c0b8c30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.802+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde9411cc30 0x7fde94113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.803+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde94111f40 0x7fde94113730 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.803+0000 7fde9aec0640 1 --2- 172.21.15.29:0/3979592010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde9410a9e0 0x7fde9411a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:41.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.803+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 >> 172.21.15.29:0/3979592010 conn(0x7fde94070ec0 msgr2=0x7fde94071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:41.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.803+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 shutdown_connections 2024-09-21T01:58:41.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:41.803+0000 7fde9aec0640 1 -- 172.21.15.29:0/3979592010 wait complete. 2024-09-21T01:58:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:41 smithi066 ceph-mon[35001]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 717 op/s 2024-09-21T01:58:42.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:41 smithi029 ceph-mon[31228]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 717 op/s 2024-09-21T01:58:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:41 smithi190 ceph-mon[34368]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 717 op/s 2024-09-21T01:58:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 605.0 s, 1192.2 tps, lat 84.127 ms stddev 100.705 2024-09-21T01:58:43.487 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:58:43.906 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:44.037 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:43 smithi029 ceph-mon[31228]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T01:58:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:43 smithi066 ceph-mon[35001]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T01:58:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:43 smithi190 ceph-mon[34368]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 -- 172.21.15.29:0/421170868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 msgr2=0x7f467c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 --2- 172.21.15.29:0/421170868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c103f90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f466c009860 tx=0x7f466c02a340 comp rx=0 tx=0).stop 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 -- 172.21.15.29:0/421170868 shutdown_connections 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 --2- 172.21.15.29:0/421170868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c109e10 0x7f467c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 --2- 172.21.15.29:0/421170868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c1094c0 0x7f467c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 --2- 172.21.15.29:0/421170868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c103f90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 -- 172.21.15.29:0/421170868 >> 172.21.15.29:0/421170868 conn(0x7f467c0ff610 msgr2=0x7f467c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 -- 172.21.15.29:0/421170868 shutdown_connections 2024-09-21T01:58:44.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.546+0000 7f4680bcd640 1 -- 172.21.15.29:0/421170868 wait complete. 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.547+0000 7f4680bcd640 1 Processor -- start 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.547+0000 7f4680bcd640 1 -- start start 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c10dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c1094c0 0x7f467c10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f467c077160 con 0x7f467c103b80 2024-09-21T01:58:44.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f467c076fe0 con 0x7f467c109e10 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4680bcd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f467c0772e0 con 0x7f467c1094c0 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f4679d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c1094c0 0x7f467c10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46160/0 (socket says 172.21.15.29:46160) 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 -- 172.21.15.29:0/3465724895 learned_addr learned my addr 172.21.15.29:0/3465724895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467a59c640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c10dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 -- 172.21.15.29:0/3465724895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c1094c0 msgr2=0x7f467c10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c1094c0 0x7f467c10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 -- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 msgr2=0x7f467c10dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.548+0000 7f467ad9d640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c10dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.549+0000 7f467ad9d640 1 -- 172.21.15.29:0/3465724895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c10eab0 con 0x7f467c109e10 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.549+0000 7f467ad9d640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f466400e940 tx=0x7f466400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.549+0000 7f46737fe640 1 -- 172.21.15.29:0/3465724895 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400cd80 con 0x7f467c109e10 2024-09-21T01:58:44.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.549+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c1acf50 con 0x7f467c109e10 2024-09-21T01:58:44.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.549+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1ad400 con 0x7f467c109e10 2024-09-21T01:58:44.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.550+0000 7f46737fe640 1 -- 172.21.15.29:0/3465724895 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46640035f0 con 0x7f467c109e10 2024-09-21T01:58:44.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.550+0000 7f46737fe640 1 -- 172.21.15.29:0/3465724895 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664010690 con 0x7f467c109e10 2024-09-21T01:58:44.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.553+0000 7f4687141740 1 Processor -- start 2024-09-21T01:58:44.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.553+0000 7f46737fe640 1 -- 172.21.15.29:0/3465724895 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4664010830 con 0x7f467c109e10 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 -- start start 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d82070 0x55f9031296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 0x55f90312a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f90309fea0 con 0x55f902d61390 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f90308f2d0 con 0x55f902d591b0 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4687141740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f9030930e0 con 0x55f902d82070 2024-09-21T01:58:44.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f467a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f467a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46162/0 (socket says 172.21.15.29:46162) 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f467a59c640 1 -- 172.21.15.29:0/3571965195 learned_addr learned my addr 172.21.15.29:0/3571965195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.554+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 0x55f90312a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 -- 172.21.15.29:0/3571965195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d82070 msgr2=0x55f9031296a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d82070 0x55f9031296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 -- 172.21.15.29:0/3571965195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 msgr2=0x55f90312a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 0x55f90312a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 -- 172.21.15.29:0/3571965195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f902ded650 con 0x55f902d591b0 2024-09-21T01:58:44.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 0x55f90312a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f467a59c640 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f466c02a860 tx=0x7f466c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ea8580fe1bafbe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f4670ff9640 1 -- 172.21.15.29:0/3571965195 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c037040 con 0x55f902d591b0 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f4670ff9640 1 -- 172.21.15.29:0/3571965195 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f466c030450 con 0x55f902d591b0 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f46737fe640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 0x7f46680b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f4670ff9640 1 -- 172.21.15.29:0/3571965195 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c003950 con 0x55f902d591b0 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.555+0000 7f46737fe640 1 -- 172.21.15.29:0/3465724895 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f46640e7320 con 0x7f467c109e10 2024-09-21T01:58:44.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.556+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 0x7f46680b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.557+0000 7f4687141740 1 -- 172.21.15.29:0/3571965195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 msgr2=0x55f903129be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.557+0000 7f4687141740 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f466c02a860 tx=0x7f466c005d30 comp rx=0 tx=0).stop 2024-09-21T01:58:44.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.558+0000 7f4687141740 1 -- 172.21.15.29:0/3571965195 shutdown_connections 2024-09-21T01:58:44.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.558+0000 7f4687141740 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d61390 0x55f90312a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.558+0000 7f4687141740 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d591b0 0x55f903129be0 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f466c02a860 tx=0x7f466c005d30 comp rx=0 tx=0).stop 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.558+0000 7f4687141740 1 --2- 172.21.15.29:0/3571965195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d82070 0x55f9031296a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.558+0000 7f4687141740 1 -- 172.21.15.29:0/3571965195 >> 172.21.15.29:0/3571965195 conn(0x55f902d76460 msgr2=0x55f903127590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.559+0000 7f4687141740 1 -- 172.21.15.29:0/3571965195 shutdown_connections 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.559+0000 7f4687141740 1 -- 172.21.15.29:0/3571965195 wait complete. 2024-09-21T01:58:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.559+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 0x7f46680b8b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f467c109000 tx=0x7f4674009d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.559+0000 7f4687141740 1 -- start start 2024-09-21T01:58:44.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 Processor -- start 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903135290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d409f0 0x55f903152200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f902ef0700 con 0x55f902d82070 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f90302a160 con 0x55f902d61390 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f903148e80 con 0x55f902d409f0 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f467a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4687141740 10 client.0 _ll_register_callbacks cb 0x55f9030439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f467a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46174/0 (socket says 172.21.15.29:46174) 2024-09-21T01:58:44.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f467a59c640 1 -- 172.21.15.29:0/3754193204 learned_addr learned my addr 172.21.15.29:0/3754193204 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:44.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.560+0000 7f467ad9d640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d409f0 0x55f903152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 -- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d409f0 msgr2=0x55f903152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d409f0 0x55f903152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 -- 172.21.15.29:0/3754193204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 msgr2=0x55f903135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 -- 172.21.15.29:0/3754193204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f903057c90 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f467a59c640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f466c005940 tx=0x7f466c009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9fde5232fe53b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f4661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c030890 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f466c03d680 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.561+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466c030450 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f466c03d880 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 10 client.27410 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f903154b30 con 0x55f902d61390 2024-09-21T01:58:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f466c030450 con 0x55f902d61390 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f466c047c80 con 0x55f902d61390 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 10 client.27410 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 10 client.27410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 20 client.27410 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f902d15da0 0x55f903157e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 4 client.27410 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.562+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9030a1df0 con 0x55f902d15da0 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.563+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f902d15da0 0x55f903157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.563+0000 7f4679d9b640 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f902d15da0 0x55f903157e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35999ee29ab68f40 server_cookie=addf06df62ff1aa5 in_seq=0 out_seq=0 2024-09-21T01:58:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.563+0000 7f4661ffb640 10 client.27410 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4661ffb640 1 -- 172.21.15.29:0/3754193204 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9030a1df0 con 0x55f902d15da0 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4661ffb640 10 client.27410 handle_command_reply: tid=0 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 client.27410 shutdown 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 msgr2=0x55f903134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f466c005940 tx=0x7f466c009450 comp rx=0 tx=0).stop 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 shutdown_connections 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f902d15da0 0x55f903157e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d409f0 0x55f903152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 --2- 172.21.15.29:0/3754193204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d61390 0x55f903134d50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 >> 172.21.15.29:0/3754193204 conn(0x55f902d76460 msgr2=0x55f90312c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.564+0000 7f4661ffb640 0 client.27410 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:58:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.565+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 shutdown_connections 2024-09-21T01:58:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.565+0000 7f4687141740 1 -- 172.21.15.29:0/3754193204 wait complete. 2024-09-21T01:58:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.565+0000 7f4687141740 20 client.27410 trim_cache size 0 max 16384 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 Processor -- start 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 -- start start 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d409f0 0x55f903152170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f903131f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 0x55f903132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00e420 con 0x55f902d409f0 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00ff60 con 0x55f902d82070 2024-09-21T01:58:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f4687141740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00eaf0 con 0x55f902d61390 2024-09-21T01:58:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 0x55f903132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 0x55f903132480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46182/0 (socket says 172.21.15.29:46182) 2024-09-21T01:58:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.590+0000 7f467ad9d640 1 -- 172.21.15.29:0/336605773 learned_addr learned my addr 172.21.15.29:0/336605773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f903131f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 -- 172.21.15.29:0/336605773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 msgr2=0x55f903132480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 0x55f903132480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 -- 172.21.15.29:0/336605773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d409f0 msgr2=0x55f903152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d409f0 0x55f903152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.593+0000 7f4679d9b640 1 -- 172.21.15.29:0/336605773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f903133250 con 0x55f902d61390 2024-09-21T01:58:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.594+0000 7f4679d9b640 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f903131f40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55f903148e80 tx=0x7f4674004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e644fb1ddeb87f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.597+0000 7f4663fff640 1 -- 172.21.15.29:0/336605773 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400fc20 con 0x55f902d61390 2024-09-21T01:58:44.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.597+0000 7f4663fff640 1 -- 172.21.15.29:0/336605773 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f467400fdc0 con 0x55f902d61390 2024-09-21T01:58:44.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 -- 172.21.15.29:0/336605773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 msgr2=0x55f903131f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f903131f40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55f903148e80 tx=0x7f4674004ab0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 -- 172.21.15.29:0/336605773 shutdown_connections 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d82070 0x55f903132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f903131f40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 --2- 172.21.15.29:0/336605773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d409f0 0x55f903152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.598+0000 7f4687141740 1 -- 172.21.15.29:0/336605773 >> 172.21.15.29:0/336605773 conn(0x55f902d76460 msgr2=0x55f9031335d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.611+0000 7f4687141740 1 -- 172.21.15.29:0/336605773 shutdown_connections 2024-09-21T01:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.615+0000 7f4687141740 1 -- 172.21.15.29:0/336605773 wait complete. 2024-09-21T01:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 -- start start 2024-09-21T01:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 Processor -- start 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 0x55f903158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f9031593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00f890 con 0x55f902d82070 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00f1c0 con 0x55f902d409f0 2024-09-21T01:58:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f465c00c400 con 0x55f902d61390 2024-09-21T01:58:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.616+0000 7f4687141740 10 client.0 _ll_register_callbacks cb 0x7f4664005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:44.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.621+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.621+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34504/0 (socket says 172.21.15.29:34504) 2024-09-21T01:58:44.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.621+0000 7f467ad9d640 1 -- 172.21.15.29:0/2108611415 learned_addr learned my addr 172.21.15.29:0/2108611415 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:44.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.621+0000 7f4679d9b640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f9031593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.623+0000 7f467a59c640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 0x55f903158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 -- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 msgr2=0x55f9031593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f9031593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 -- 172.21.15.29:0/2108611415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 msgr2=0x55f903159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 -- 172.21.15.29:0/2108611415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9031c1610 con 0x55f902d409f0 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.625+0000 7f467a59c640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 0x55f903158e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f466c0325b0 tx=0x7f466c005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66454dca86d8cd2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:44.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.626+0000 7f4661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:44.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.626+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c05e030 con 0x55f902d409f0 2024-09-21T01:58:44.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.626+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467400fdc0 con 0x55f902d409f0 2024-09-21T01:58:44.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.626+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f466c030450 con 0x55f902d409f0 2024-09-21T01:58:44.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.626+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f466c037040 con 0x55f902d409f0 2024-09-21T01:58:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.629+0000 7f4687141740 10 client.27416 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.629+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f9031c2890 con 0x55f902d409f0 2024-09-21T01:58:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.630+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f467400fdc0 con 0x55f902d409f0 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.630+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f466c005030 con 0x55f902d409f0 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 10 client.27416 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 10 client.27416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 20 client.27416 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f9031c3040 0x55f9031c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 4 client.27416 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:58:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f90309d8a0 con 0x55f9031c3040 2024-09-21T01:58:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.631+0000 7f4679d9b640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f9031c3040 0x55f9031c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.632+0000 7f4679d9b640 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f9031c3040 0x55f9031c5460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1128b033c7edde49 server_cookie=3aaef5a7fe48ba0e in_seq=0 out_seq=0 2024-09-21T01:58:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.632+0000 7f4661ffb640 10 client.27416 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:58:44.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.637+0000 7f4661ffb640 1 -- 172.21.15.29:0/2108611415 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f90309d8a0 con 0x55f9031c3040 2024-09-21T01:58:44.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.637+0000 7f4661ffb640 10 client.27416 handle_command_reply: tid=0 2024-09-21T01:58:44.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.637+0000 7f4687141740 1 client.27416 shutdown 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 msgr2=0x55f903158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 0x55f903158e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f466c0325b0 tx=0x7f466c005da0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 shutdown_connections 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55f9031c3040 0x55f9031c5460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f902d82070 0x55f903159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f902d61390 0x55f9031593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 --2- 172.21.15.29:0/2108611415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f902d409f0 0x55f903158e80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 >> 172.21.15.29:0/2108611415 conn(0x55f902d76460 msgr2=0x55f90312a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4661ffb640 0 client.27416 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:58:44.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 shutdown_connections 2024-09-21T01:58:44.643 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:58:44.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.640+0000 7f4687141740 1 -- 172.21.15.29:0/2108611415 wait complete. 2024-09-21T01:58:44.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.641+0000 7f4687141740 20 client.27416 trim_cache size 0 max 16384 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 msgr2=0x7f46680b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 0x7f46680b8b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f467c109000 tx=0x7f4674009d10 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 msgr2=0x7f467c1ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f466400e940 tx=0x7f466400ee00 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 shutdown_connections 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f46680b6610 0x7f46680b8b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f467c109e10 0x7f467c1ac8f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c1094c0 0x7f467c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 --2- 172.21.15.29:0/3465724895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f467c103b80 0x7f467c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.643+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 >> 172.21.15.29:0/3465724895 conn(0x7f467c0ff610 msgr2=0x7f467c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.646+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 shutdown_connections 2024-09-21T01:58:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:44.647+0000 7f4680bcd640 1 -- 172.21.15.29:0/3465724895 wait complete. 2024-09-21T01:58:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:45.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:45.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:58:46.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:58:46.042 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:45 smithi029 ceph-mon[31228]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2024-09-21T01:58:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:45 smithi066 ceph-mon[35001]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2024-09-21T01:58:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:45 smithi190 ceph-mon[34368]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2024-09-21T01:58:46.396 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/126998146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 msgr2=0x7fcdbc102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/126998146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc102380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009860 tx=0x7fcdb802a340 comp rx=0 tx=0).stop 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/126998146 shutdown_connections 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/126998146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc109e00 0x7fcdbc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/126998146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/126998146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc102380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.982+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/126998146 >> 172.21.15.29:0/126998146 conn(0x7fcdbc077c60 msgr2=0x7fcdbc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.983+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/126998146 shutdown_connections 2024-09-21T01:58:46.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.983+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/126998146 wait complete. 2024-09-21T01:58:46.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.984+0000 7fcdc3cd5640 1 Processor -- start 2024-09-21T01:58:46.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.984+0000 7fcdc3cd5640 1 -- start start 2024-09-21T01:58:46.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.985+0000 7fcdc3cd5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.985+0000 7fcdc3cd5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.985+0000 7fcdc3cd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc109e00 0x7fcdbc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc3cd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113ac0 con 0x7fcdbc109e00 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc3cd5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113940 con 0x7fcdbc1094b0 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc3cd5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113c40 con 0x7fcdbc101f70 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46196/0 (socket says 172.21.15.29:46196) 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc1270640 1 -- 172.21.15.29:0/2729229556 learned_addr learned my addr 172.21.15.29:0/2729229556 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:46.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.986+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.987+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2729229556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 msgr2=0x7fcdbc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.987+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.987+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc109e00 msgr2=0x7fcdbc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.987+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc109e00 0x7fcdbc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.987+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2729229556 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc1a0420 con 0x7fcdbc101f70 2024-09-21T01:58:46.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.988+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc19a4b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8035040 tx=0x7fcdb8032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:46.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.988+0000 7fcdb2ffd640 1 -- 172.21.15.29:0/2729229556 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8037040 con 0x7fcdbc101f70 2024-09-21T01:58:46.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.988+0000 7fcdb2ffd640 1 -- 172.21.15.29:0/2729229556 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb8030c10 con 0x7fcdbc101f70 2024-09-21T01:58:46.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.988+0000 7fcdb2ffd640 1 -- 172.21.15.29:0/2729229556 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80035f0 con 0x7fcdbc101f70 2024-09-21T01:58:46.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.991+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc1a06b0 con 0x7fcdbc101f70 2024-09-21T01:58:46.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.991+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a0ae0 con 0x7fcdbc101f70 2024-09-21T01:58:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.994+0000 7fcdb2ffd640 1 -- 172.21.15.29:0/2729229556 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcdb8043020 con 0x7fcdbc101f70 2024-09-21T01:58:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.995+0000 7fcdb2ffd640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 0x7fcda80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.995+0000 7fcdca249740 1 Processor -- start 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.995+0000 7fcdb2ffd640 1 -- 172.21.15.29:0/2729229556 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcdb8105fd0 con 0x7fcdbc101f70 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.995+0000 7fcdc1270640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 0x7fcda80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.995+0000 7fcdca249740 1 -- start start 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb913476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91347c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91348150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb912b9ea0 con 0x55cb90f9c070 2024-09-21T01:58:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb912a92d0 con 0x55cb90f731b0 2024-09-21T01:58:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdca249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb912ad0e0 con 0x55cb90f7b390 2024-09-21T01:58:46.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb913476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91347c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb913476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34524/0 (socket says 172.21.15.29:34524) 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc1a71640 1 -- 172.21.15.29:0/3316102155 learned_addr learned my addr 172.21.15.29:0/3316102155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91348150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc1270640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 0x7fcda80b8bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc109ab0 tx=0x7fcdac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 -- 172.21.15.29:0/3316102155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 msgr2=0x55cb91347c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91347c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 -- 172.21.15.29:0/3316102155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 msgr2=0x55cb913476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb913476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.996+0000 7fcdc2272640 1 -- 172.21.15.29:0/3316102155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb91007650 con 0x55cb90f7b390 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.997+0000 7fcdc2272640 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91348150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005fd0 tx=0x7fcdb400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2721651266605a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.997+0000 7fcda7fff640 1 -- 172.21.15.29:0/3316102155 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4004d20 con 0x55cb90f7b390 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.997+0000 7fcda7fff640 1 -- 172.21.15.29:0/3316102155 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb400ee70 con 0x55cb90f7b390 2024-09-21T01:58:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.997+0000 7fcda7fff640 1 -- 172.21.15.29:0/3316102155 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005020 con 0x55cb90f7b390 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 -- 172.21.15.29:0/3316102155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 msgr2=0x55cb91348150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91348150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005fd0 tx=0x7fcdb400e720 comp rx=0 tx=0).stop 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 -- 172.21.15.29:0/3316102155 shutdown_connections 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91348150 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91347c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 --2- 172.21.15.29:0/3316102155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb913476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 -- 172.21.15.29:0/3316102155 >> 172.21.15.29:0/3316102155 conn(0x55cb90f90460 msgr2=0x55cb913455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 -- 172.21.15.29:0/3316102155 shutdown_connections 2024-09-21T01:58:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.998+0000 7fcdca249740 1 -- 172.21.15.29:0/3316102155 wait complete. 2024-09-21T01:58:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdca249740 1 -- start start 2024-09-21T01:58:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdca249740 1 Processor -- start 2024-09-21T01:58:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdca249740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdca249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91353310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdca249740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb9135f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:46.999+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46206/0 (socket says 172.21.15.29:46206) 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91353310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91353310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35856/0 (socket says 172.21.15.29:35856) 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdca249740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb9110a700 con 0x55cb90f9c070 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdc2272640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb9135f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2610430532 learned_addr learned my addr 172.21.15.29:0/2610430532 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb91244160 con 0x55cb90f731b0 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb91355eb0 con 0x55cb90f7b390 2024-09-21T01:58:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.000+0000 7fcdca249740 10 client.0 _ll_register_callbacks cb 0x55cb9125d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2610430532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 msgr2=0x55cb91353310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91353310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2610430532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 msgr2=0x55cb9135f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb9135f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc1a71640 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb91271c90 con 0x55cb90f731b0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.001+0000 7fcdc2272640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb9135f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.002+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80c8950 tx=0x7fcdb802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9212ef59e9f7e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.002+0000 7fcda57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.002+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80cbca0 con 0x55cb90f731b0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.002+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb400ee70 con 0x55cb90f731b0 2024-09-21T01:58:47.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.002+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcdb80d5930 con 0x55cb90f731b0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcdb80cc890 con 0x55cb90f731b0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 10 client.27428 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cb91361ba0 con 0x55cb90f731b0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb400ee70 con 0x55cb90f731b0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fcdb80cae60 con 0x55cb90f731b0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 10 client.27428 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 10 client.27428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 20 client.27428 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f5a9f0 0x55cb91365350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 4 client.27428 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.003+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb912bbdf0 con 0x55cb90f5a9f0 2024-09-21T01:58:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.004+0000 7fcdc1270640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f5a9f0 0x55cb91365350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.005+0000 7fcda57fa640 10 client.27428 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.005+0000 7fcdc1270640 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f5a9f0 0x55cb91365350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35539950b19f7e6a server_cookie=562f323b105a150e in_seq=0 out_seq=0 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.005+0000 7fcda57fa640 1 -- 172.21.15.29:0/2610430532 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb912bbdf0 con 0x55cb90f5a9f0 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.005+0000 7fcda57fa640 10 client.27428 handle_command_reply: tid=0 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.005+0000 7fcdca249740 1 client.27428 shutdown 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 msgr2=0x55cb91352dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80c8950 tx=0x7fcdb802a340 comp rx=0 tx=0).stop 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 shutdown_connections 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f5a9f0 0x55cb91365350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f9c070 0x55cb9135f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f7b390 0x55cb91353310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcda57fa640 0 client.27428 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:58:47.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 --2- 172.21.15.29:0/2610430532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb91352dd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 >> 172.21.15.29:0/2610430532 conn(0x55cb90f90460 msgr2=0x55cb9134a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:47.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 shutdown_connections 2024-09-21T01:58:47.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 1 -- 172.21.15.29:0/2610430532 wait complete. 2024-09-21T01:58:47.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.006+0000 7fcdca249740 20 client.27428 trim_cache size 0 max 16384 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 Processor -- start 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 -- start start 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 0x55cb9135f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9134ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ad290 con 0x55cb90f7b390 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1aedd0 con 0x55cb90f5a9f0 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdca249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ad960 con 0x55cb90f731b0 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.027+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1a71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46212/0 (socket says 172.21.15.29:46212) 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1a71640 1 -- 172.21.15.29:0/1906865521 learned_addr learned my addr 172.21.15.29:0/1906865521 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 0x55cb9135f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc2272640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9134ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 -- 172.21.15.29:0/1906865521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 msgr2=0x55cb91353290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 -- 172.21.15.29:0/1906865521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 msgr2=0x55cb9134ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9134ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 -- 172.21.15.29:0/1906865521 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb91354090 con 0x55cb90f731b0 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.028+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 0x55cb9135f0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcdac013010 tx=0x7fcdac018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101bfd66fac346f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.029+0000 7fcda77fe640 1 -- 172.21.15.29:0/1906865521 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac035070 con 0x55cb90f731b0 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.029+0000 7fcda77fe640 1 -- 172.21.15.29:0/1906865521 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac030920 con 0x55cb90f731b0 2024-09-21T01:58:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.029+0000 7fcda77fe640 1 -- 172.21.15.29:0/1906865521 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac01d920 con 0x55cb90f731b0 2024-09-21T01:58:47.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- 172.21.15.29:0/1906865521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 msgr2=0x55cb9135f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 0x55cb9135f0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcdac013010 tx=0x7fcdac018000 comp rx=0 tx=0).stop 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- 172.21.15.29:0/1906865521 shutdown_connections 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9134ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f731b0 0x55cb9135f0a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 --2- 172.21.15.29:0/1906865521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f5a9f0 0x55cb91353290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- 172.21.15.29:0/1906865521 >> 172.21.15.29:0/1906865521 conn(0x55cb90f90460 msgr2=0x55cb91352b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- 172.21.15.29:0/1906865521 shutdown_connections 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- 172.21.15.29:0/1906865521 wait complete. 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.030+0000 7fcdca249740 1 -- start start 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 Processor -- start 2024-09-21T01:58:47.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f5a9f0 0x55cb913446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9135b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ae700 con 0x55cb90f7b390 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ae030 con 0x55cb90f731b0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ab270 con 0x55cb90f5a9f0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdca249740 10 client.0 _ll_register_callbacks cb 0x7fcdb80cbf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdc1270640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46216/0 (socket says 172.21.15.29:46216) 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdc1270640 1 -- 172.21.15.29:0/1812847648 learned_addr learned my addr 172.21.15.29:0/1812847648 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f5a9f0 0x55cb913446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.031+0000 7fcdc2272640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9135b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 -- 172.21.15.29:0/1812847648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f5a9f0 msgr2=0x55cb913446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f5a9f0 0x55cb913446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 -- 172.21.15.29:0/1812847648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 msgr2=0x55cb9135b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9135b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 -- 172.21.15.29:0/1812847648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb9135bdd0 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc2272640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9135b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdc1270640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdac030780 tx=0x7fcdac0307b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2f76bb837aa550 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcda57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac035070 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fcdac005030 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac030920 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcdac030e20 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdca249740 10 client.27434 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cb913d4540 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.032+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac0046c0 con 0x55cb90f731b0 2024-09-21T01:58:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fcdac03a5f0 con 0x55cb90f731b0 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 10 client.27434 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 10 client.27434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 20 client.27434 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f9c070 0x55cb913d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 4 client.27434 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcdac030920 con 0x55cb90f9c070 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.033+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f9c070 0x55cb913d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdc1a71640 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f9c070 0x55cb913d6c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=922e81198d83bf5c server_cookie=ce7ca337f7903a7b in_seq=0 out_seq=0 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcda57fa640 10 client.27434 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcda57fa640 1 -- 172.21.15.29:0/1812847648 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcdac030920 con 0x55cb90f9c070 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcda57fa640 10 client.27434 handle_command_reply: tid=0 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 client.27434 shutdown 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 msgr2=0x55cb9135b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdac030780 tx=0x7fcdac0307b0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 shutdown_connections 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55cb90f9c070 0x55cb913d6c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcda57fa640 0 client.27434 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb90f7b390 0x55cb9135b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb90f731b0 0x55cb9135b100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 --2- 172.21.15.29:0/1812847648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb90f5a9f0 0x55cb913446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 >> 172.21.15.29:0/1812847648 conn(0x55cb90f90460 msgr2=0x55cb91354260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 shutdown_connections 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 1 -- 172.21.15.29:0/1812847648 wait complete. 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.034+0000 7fcdca249740 20 client.27434 trim_cache size 0 max 16384 2024-09-21T01:58:47.040 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:58:47.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 msgr2=0x7fcda80b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 0x7fcda80b8bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc109ab0 tx=0x7fcdac009260 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 msgr2=0x7fcdbc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc19a4b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8035040 tx=0x7fcdb8032d50 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 shutdown_connections 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fcda80b66e0 0x7fcda80b8bf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc109e00 0x7fcdbc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc1094b0 0x7fcdbc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 --2- 172.21.15.29:0/2729229556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc101f70 0x7fcdbc19a4b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 >> 172.21.15.29:0/2729229556 conn(0x7fcdbc077c60 msgr2=0x7fcdbc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 shutdown_connections 2024-09-21T01:58:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:47.035+0000 7fcdc3cd5640 1 -- 172.21.15.29:0/2729229556 wait complete. 2024-09-21T01:58:47.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:47 smithi029 ceph-mon[31228]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 703 op/s 2024-09-21T01:58:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 610.0 s, 1235.2 tps, lat 81.325 ms stddev 87.075 2024-09-21T01:58:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:47 smithi066 ceph-mon[35001]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 703 op/s 2024-09-21T01:58:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:47 smithi190 ceph-mon[34368]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 703 op/s 2024-09-21T01:58:48.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:58:48.938 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 -- 172.21.15.29:0/3524264349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 msgr2=0x7f5464103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 --2- 172.21.15.29:0/3524264349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464103a30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5454009860 tx=0x7f545402a360 comp rx=0 tx=0).stop 2024-09-21T01:58:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 -- 172.21.15.29:0/3524264349 shutdown_connections 2024-09-21T01:58:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 --2- 172.21.15.29:0/3524264349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5464103fc0 0x7f546410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 --2- 172.21.15.29:0/3524264349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464103a30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 --2- 172.21.15.29:0/3524264349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54641025a0 0x7f5464102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.321+0000 7f546bb07640 1 -- 172.21.15.29:0/3524264349 >> 172.21.15.29:0/3524264349 conn(0x7f54640fdee0 msgr2=0x7f5464100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.322+0000 7f546bb07640 1 -- 172.21.15.29:0/3524264349 shutdown_connections 2024-09-21T01:58:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.322+0000 7f546bb07640 1 -- 172.21.15.29:0/3524264349 wait complete. 2024-09-21T01:58:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.322+0000 7f546bb07640 1 Processor -- start 2024-09-21T01:58:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 -- start start 2024-09-21T01:58:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464104a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 0x7f5464104f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f546410e1e0 con 0x7f5464103fc0 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f546410e060 con 0x7f5464103620 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546bb07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f546410e360 con 0x7f54641025a0 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54690a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35902/0 (socket says 172.21.15.29:35902) 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 -- 172.21.15.29:0/3419941143 learned_addr learned my addr 172.21.15.29:0/3419941143 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546a0a4640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 0x7f5464104f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 -- 172.21.15.29:0/3419941143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 msgr2=0x7f5464104a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464104a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 -- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 msgr2=0x7f5464104f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 0x7f5464104f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54698a3640 1 -- 172.21.15.29:0/3419941143 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54641056a0 con 0x7f54641025a0 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f546a0a4640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 0x7f5464104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.323+0000 7f54690a2640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f54698a3640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f546000ea60 tx=0x7f546000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f545affd640 1 -- 172.21.15.29:0/3419941143 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546000cde0 con 0x7f54641025a0 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f545affd640 1 -- 172.21.15.29:0/3419941143 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54600035f0 con 0x7f54641025a0 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f545affd640 1 -- 172.21.15.29:0/3419941143 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460010690 con 0x7f54641025a0 2024-09-21T01:58:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54641a3fe0 con 0x7f54641025a0 2024-09-21T01:58:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.324+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5464199d80 con 0x7f54641025a0 2024-09-21T01:58:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 Processor -- start 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 -- start start 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29208ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29208b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b291ffd6f0 con 0x55b291cba2f0 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b291fecb20 con 0x55b291cc24d0 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.327+0000 7f5472a71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b291ff0930 con 0x55b291ce31b0 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46252/0 (socket says 172.21.15.29:46252) 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 -- 172.21.15.29:0/3279697723 learned_addr learned my addr 172.21.15.29:0/3279697723 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f546a0a4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29208ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 -- 172.21.15.29:0/3279697723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 msgr2=0x55b29208ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29208ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 -- 172.21.15.29:0/3279697723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 msgr2=0x55b29208b4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29208b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 -- 172.21.15.29:0/3279697723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b291fb2530 con 0x55b291cc24d0 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f545affd640 1 -- 172.21.15.29:0/3419941143 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f54600037a0 con 0x7f54641025a0 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.328+0000 7f54690a2640 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f545402a880 tx=0x7f5454005cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d575393eb03a700 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f544ffff640 1 -- 172.21.15.29:0/3279697723 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454041020 con 0x55b291cc24d0 2024-09-21T01:58:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f544ffff640 1 -- 172.21.15.29:0/3279697723 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5454030750 con 0x55b291cc24d0 2024-09-21T01:58:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f544ffff640 1 -- 172.21.15.29:0/3279697723 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454037040 con 0x55b291cc24d0 2024-09-21T01:58:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f545affd640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 0x7f54500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f546a0a4640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 0x7f54500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.329+0000 7f545affd640 1 -- 172.21.15.29:0/3419941143 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54600e7650 con 0x7f54641025a0 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 -- 172.21.15.29:0/3279697723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 msgr2=0x55b29208c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f545402a880 tx=0x7f5454005cb0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f546a0a4640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 0x7f54500b8a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f545c011570 tx=0x7f545c0095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 -- 172.21.15.29:0/3279697723 shutdown_connections 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29208c010 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29208b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 --2- 172.21.15.29:0/3279697723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29208ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 -- 172.21.15.29:0/3279697723 >> 172.21.15.29:0/3279697723 conn(0x55b291cd7300 msgr2=0x55b2920868e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 -- 172.21.15.29:0/3279697723 shutdown_connections 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.330+0000 7f5472a71740 1 -- 172.21.15.29:0/3279697723 wait complete. 2024-09-21T01:58:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.331+0000 7f5472a71740 1 -- start start 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.331+0000 7f5472a71740 1 Processor -- start 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292094c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b291f24ab0 con 0x55b291ce31b0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b291f92730 con 0x55b291cba2f0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b29208cad0 con 0x55b291cc24d0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46266/0 (socket says 172.21.15.29:46266) 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 -- 172.21.15.29:0/3651411858 learned_addr learned my addr 172.21.15.29:0/3651411858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54690a2640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f546a0a4640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f5472a71740 10 client.0 _ll_register_callbacks cb 0x55b291fe66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 -- 172.21.15.29:0/3651411858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 msgr2=0x55b292094c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292094c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 -- 172.21.15.29:0/3651411858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 msgr2=0x55b2920957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.332+0000 7f54698a3640 1 -- 172.21.15.29:0/3651411858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b291d4e790 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f54698a3640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5460009230 tx=0x7f54600ad620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9cf4158845c9cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f544d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460004200 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54600e70d0 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5454030750 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f54600ad2a0 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f5472a71740 10 client.27446 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b291dc7620 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.333+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5454030750 con 0x55b291cba2f0 2024-09-21T01:58:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f54600aeac0 con 0x55b291cba2f0 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 10 client.27446 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 10 client.27446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 20 client.27446 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291ca1d80 0x55b2920a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 4 client.27446 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b291fff640 con 0x55b291ca1d80 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.334+0000 7f54690a2640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291ca1d80 0x55b2920a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.335+0000 7f54690a2640 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291ca1d80 0x55b2920a1cc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a829a009e907b24 server_cookie=f9dd09c155e6a8de in_seq=0 out_seq=0 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.335+0000 7f544d7fa640 10 client.27446 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.335+0000 7f544d7fa640 1 -- 172.21.15.29:0/3651411858 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b291fff640 con 0x55b291ca1d80 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.335+0000 7f544d7fa640 10 client.27446 handle_command_reply: tid=0 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 client.27446 shutdown 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 msgr2=0x55b292094580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5460009230 tx=0x7f54600ad620 comp rx=0 tx=0).stop 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 shutdown_connections 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291ca1d80 0x55b2920a1cc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292094c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 --2- 172.21.15.29:0/3651411858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b292094580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 >> 172.21.15.29:0/3651411858 conn(0x55b291cd7300 msgr2=0x55b292084ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f544d7fa640 0 client.27446 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 shutdown_connections 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 1 -- 172.21.15.29:0/3651411858 wait complete. 2024-09-21T01:58:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.336+0000 7f5472a71740 20 client.27446 trim_cache size 0 max 16384 2024-09-21T01:58:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.361+0000 7f5472a71740 1 Processor -- start 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 -- start start 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b29208bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54641b1570 con 0x55b291ce31b0 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54641b30b0 con 0x55b291cba2f0 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f5472a71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54641b1c40 con 0x55b291cc24d0 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f54690a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f54690a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35940/0 (socket says 172.21.15.29:35940) 2024-09-21T01:58:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f54690a2640 1 -- 172.21.15.29:0/828955190 learned_addr learned my addr 172.21.15.29:0/828955190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f546a0a4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.362+0000 7f54698a3640 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b29208bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 -- 172.21.15.29:0/828955190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 msgr2=0x55b29208bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b29208bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 -- 172.21.15.29:0/828955190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 msgr2=0x55b2920872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 -- 172.21.15.29:0/828955190 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29209ca50 con 0x55b291cc24d0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54698a3640 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b29208bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f54690a2640 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5454000c00 tx=0x7f54540303b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f493262d7f161ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f544f7fe640 1 -- 172.21.15.29:0/828955190 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454041040 con 0x55b291cc24d0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f544f7fe640 1 -- 172.21.15.29:0/828955190 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5454005d20 con 0x55b291cc24d0 2024-09-21T01:58:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.363+0000 7f544f7fe640 1 -- 172.21.15.29:0/828955190 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454037040 con 0x55b291cc24d0 2024-09-21T01:58:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 -- 172.21.15.29:0/828955190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 msgr2=0x55b292086d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5454000c00 tx=0x7f54540303b0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 -- 172.21.15.29:0/828955190 shutdown_connections 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291ce31b0 0x55b2920872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291cc24d0 0x55b292086d80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 --2- 172.21.15.29:0/828955190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cba2f0 0x55b29208bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 -- 172.21.15.29:0/828955190 >> 172.21.15.29:0/828955190 conn(0x55b291cd7300 msgr2=0x55b292092b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 -- 172.21.15.29:0/828955190 shutdown_connections 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.364+0000 7f5472a71740 1 -- 172.21.15.29:0/828955190 wait complete. 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.365+0000 7f5472a71740 1 -- start start 2024-09-21T01:58:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.366+0000 7f5472a71740 1 Processor -- start 2024-09-21T01:58:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29209b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29209b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54641b29e0 con 0x55b291cba2f0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54641b2310 con 0x55b291cc24d0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54641af570 con 0x55b291ce31b0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f5472a71740 10 client.0 _ll_register_callbacks cb 0x7f546000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f54698a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f54690a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29209b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f54698a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34596/0 (socket says 172.21.15.29:34596) 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f54698a3640 1 -- 172.21.15.29:0/1341634153 learned_addr learned my addr 172.21.15.29:0/1341634153 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29209b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 -- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 msgr2=0x55b29209b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29209b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 -- 172.21.15.29:0/1341634153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 msgr2=0x55b29209c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f546a0a4640 1 -- 172.21.15.29:0/1341634153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b292095b20 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.367+0000 7f54698a3640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f546a0a4640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29209b7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f545c012010 tx=0x7f545c004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9143c947ad9a25b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f544d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c02e060 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b291ffb0f0 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f545c01caf0 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f545c00fe20 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f5472a71740 10 client.27409 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b29210c790 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.368+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b291ffb0f0 con 0x55b291ce31b0 2024-09-21T01:58:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f545c0343d0 con 0x55b291ce31b0 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 10 client.27409 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 10 client.27409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 20 client.27409 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291c771c0 0x55b29210f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 4 client.27409 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b291ff8520 con 0x55b291c771c0 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f54698a3640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291c771c0 0x55b29210f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.369+0000 7f544d7fa640 10 client.27409 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.370+0000 7f54698a3640 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291c771c0 0x55b29210f190 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f3d5c1e8e07676 server_cookie=28fc5070f619c101 in_seq=0 out_seq=0 2024-09-21T01:58:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.381+0000 7f544d7fa640 1 -- 172.21.15.29:0/1341634153 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b291ff8520 con 0x55b291c771c0 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.381+0000 7f544d7fa640 10 client.27409 handle_command_reply: tid=0 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 client.27409 shutdown 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 msgr2=0x55b29209b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29209b7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f545c012010 tx=0x7f545c004a90 comp rx=0 tx=0).stop 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 shutdown_connections 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b291c771c0 0x55b29210f190 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b291ce31b0 0x55b29209b7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b291cc24d0 0x55b29209b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 --2- 172.21.15.29:0/1341634153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b291cba2f0 0x55b29209c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 >> 172.21.15.29:0/1341634153 conn(0x55b291cd7300 msgr2=0x55b29208eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f544d7fa640 0 client.27409 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 shutdown_connections 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 1 -- 172.21.15.29:0/1341634153 wait complete. 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.382+0000 7f5472a71740 20 client.27409 trim_cache size 0 max 16384 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:58:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 msgr2=0x7f54500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 0x7f54500b8a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f545c011570 tx=0x7f545c0095a0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 msgr2=0x7f5464106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f546000ea60 tx=0x7f546000ef20 comp rx=0 tx=0).stop 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 shutdown_connections 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54500b6510 0x7f54500b8a20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5464103fc0 0x7f5464104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5464103620 0x7f5464104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 --2- 172.21.15.29:0/3419941143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54641025a0 0x7f5464106170 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 >> 172.21.15.29:0/3419941143 conn(0x7f54640fdee0 msgr2=0x7f54640ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 shutdown_connections 2024-09-21T01:58:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:58:49.383+0000 7f546bb07640 1 -- 172.21.15.29:0/3419941143 wait complete. 2024-09-21T01:58:49.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.680+0000 7f94d62b2640 1 -- 172.21.15.29:0/1498775735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d0113eb0 msgr2=0x7f94d01142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.680+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1498775735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d0113eb0 0x7f94d01142c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009860 tx=0x7f94b802a340 comp rx=0 tx=0).stop 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.680+0000 7f94d62b2640 1 -- 172.21.15.29:0/1498775735 shutdown_connections 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.680+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1498775735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d011b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.680+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1498775735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d0113eb0 0x7f94d01142c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1498775735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d010e5c0 0x7f94d010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 -- 172.21.15.29:0/1498775735 >> 172.21.15.29:0/1498775735 conn(0x7f94d010a3c0 msgr2=0x7f94d010c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 -- 172.21.15.29:0/1498775735 shutdown_connections 2024-09-21T01:58:49.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 -- 172.21.15.29:0/1498775735 wait complete. 2024-09-21T01:58:49.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 Processor -- start 2024-09-21T01:58:49.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.681+0000 7f94d62b2640 1 -- start start 2024-09-21T01:58:49.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 0x7f94d0118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94d011e470 con 0x7f94d0114800 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94d011e2f0 con 0x7f94d0113eb0 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d62b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94d011e5f0 con 0x7f94d010e5c0 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94cffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94cffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35966/0 (socket says 172.21.15.29:35966) 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94cffff640 1 -- 172.21.15.29:0/1356269275 learned_addr learned my addr 172.21.15.29:0/1356269275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34618/0 (socket says 172.21.15.29:34618) 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 0x7f94d0118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 -- 172.21.15.29:0/1356269275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 msgr2=0x7f94d01184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 -- 172.21.15.29:0/1356269275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 msgr2=0x7f94d0118a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 0x7f94d0118a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94d484f640 1 -- 172.21.15.29:0/1356269275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d01b5340 con 0x7f94d0114800 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94cffff640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.682+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 0x7f94d0118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.683+0000 7f94d484f640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94c800ef00 tx=0x7f94c800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.683+0000 7f94cdffb640 1 -- 172.21.15.29:0/1356269275 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8004b20 con 0x7f94d0114800 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.683+0000 7f94cdffb640 1 -- 172.21.15.29:0/1356269275 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94c8003e20 con 0x7f94d0114800 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.683+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d01b5630 con 0x7f94d0114800 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.683+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d01b5a80 con 0x7f94d0114800 2024-09-21T01:58:49.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.684+0000 7f94cdffb640 1 -- 172.21.15.29:0/1356269275 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c800b570 con 0x7f94d0114800 2024-09-21T01:58:49.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.687+0000 7f94cdffb640 1 -- 172.21.15.29:0/1356269275 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f94c801d020 con 0x7f94d0114800 2024-09-21T01:58:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 Processor -- start 2024-09-21T01:58:49.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 -- start start 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94cdffb640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 0x7f94bc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5df390 0x5583ff9a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5583ff91dea0 con 0x5583ff5df390 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5583ff90d2d0 con 0x5583ff5d71b0 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94dc826740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583ff9110e0 con 0x5583ff600070 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.688+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 0x7f94bc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46326/0 (socket says 172.21.15.29:46326) 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94d484f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5df390 0x5583ff9a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 -- 172.21.15.29:0/1482403847 learned_addr learned my addr 172.21.15.29:0/1482403847 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94cffff640 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 -- 172.21.15.29:0/1482403847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 msgr2=0x5583ff9a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 -- 172.21.15.29:0/1482403847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5df390 msgr2=0x5583ff9a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5df390 0x5583ff9a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94c7fff640 1 -- 172.21.15.29:0/1482403847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ff66b650 con 0x5583ff5d71b0 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.689+0000 7f94cdffb640 1 -- 172.21.15.29:0/1356269275 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f94c80e6f70 con 0x7f94d0114800 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.690+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f94b80303f0 tx=0x7f94b80043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744e8b3c4ff5f489 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.690+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 0x7f94bc0b8b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b802c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.690+0000 7f94c67fc640 1 -- 172.21.15.29:0/1482403847 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b804f070 con 0x5583ff5d71b0 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.690+0000 7f94c67fc640 1 -- 172.21.15.29:0/1482403847 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b80308e0 con 0x5583ff5d71b0 2024-09-21T01:58:49.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.690+0000 7f94c67fc640 1 -- 172.21.15.29:0/1482403847 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8037040 con 0x5583ff5d71b0 2024-09-21T01:58:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- 172.21.15.29:0/1482403847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 msgr2=0x5583ff9a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f94b80303f0 tx=0x7f94b80043f0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- 172.21.15.29:0/1482403847 shutdown_connections 2024-09-21T01:58:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5df390 0x5583ff9a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9a7800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 --2- 172.21.15.29:0/1482403847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- 172.21.15.29:0/1482403847 >> 172.21.15.29:0/1482403847 conn(0x5583ff5f4460 msgr2=0x5583ff9a51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- 172.21.15.29:0/1482403847 shutdown_connections 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- 172.21.15.29:0/1482403847 wait complete. 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.691+0000 7f94dc826740 1 -- start start 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 Processor -- start 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5583ff76e700 con 0x5583ff5d71b0 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5583ff8a8160 con 0x5583ff5df390 2024-09-21T01:58:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583ff9b5a40 con 0x5583ff600070 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.692+0000 7f94dc826740 10 client.0 _ll_register_callbacks cb 0x5583ff8c19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:35984/0 (socket says 172.21.15.29:35984) 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 -- 172.21.15.29:0/3081472325 learned_addr learned my addr 172.21.15.29:0/3081472325 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 -- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 msgr2=0x5583ff9b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 -- 172.21.15.29:0/3081472325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 msgr2=0x5583ff9b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.693+0000 7f94d484f640 1 -- 172.21.15.29:0/3081472325 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ff8d5c90 con 0x5583ff600070 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94d484f640 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f94c80a9680 tx=0x7f94c8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b643fb77f957e0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c801a030 con 0x5583ff600070 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b80308e0 con 0x5583ff600070 2024-09-21T01:58:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f94c80041d0 con 0x5583ff600070 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f94c8070730 con 0x5583ff600070 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94dc826740 10 client.27421 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5583ff9d26f0 con 0x5583ff600070 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b80308e0 con 0x5583ff600070 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.694+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f94c800b040 con 0x5583ff600070 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 10 client.27421 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 20 client.27421 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff5be9f0 0x5583ff9d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 4 client.27421 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ff91fdf0 con 0x5583ff5be9f0 2024-09-21T01:58:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94cffff640 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff5be9f0 0x5583ff9d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94af7fe640 10 client.27421 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.695+0000 7f94cffff640 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff5be9f0 0x5583ff9d60f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c7d7e11b4045d2 server_cookie=a462d50617965b8b in_seq=0 out_seq=0 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94af7fe640 1 -- 172.21.15.29:0/3081472325 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583ff91fdf0 con 0x5583ff5be9f0 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94af7fe640 10 client.27421 handle_command_reply: tid=0 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 client.27421 shutdown 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 msgr2=0x5583ff9b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f94c80a9680 tx=0x7f94c8004ab0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 shutdown_connections 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff5be9f0 0x5583ff9d60f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9b3360 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 --2- 172.21.15.29:0/3081472325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.696+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 >> 172.21.15.29:0/3081472325 conn(0x5583ff5f4460 msgr2=0x5583ff9aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.697+0000 7f94af7fe640 0 client.27421 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.697+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 shutdown_connections 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.697+0000 7f94dc826740 1 -- 172.21.15.29:0/3081472325 wait complete. 2024-09-21T01:58:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.697+0000 7f94dc826740 20 client.27421 trim_cache size 0 max 16384 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 Processor -- start 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 -- start start 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000e410 con 0x5583ff5d71b0 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000ff50 con 0x5583ff5df390 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94dc826740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000eae0 con 0x5583ff600070 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34642/0 (socket says 172.21.15.29:34642) 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94cffff640 1 -- 172.21.15.29:0/881864001 learned_addr learned my addr 172.21.15.29:0/881864001 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.714+0000 7f94d484f640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 -- 172.21.15.29:0/881864001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 msgr2=0x5583ff9a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 -- 172.21.15.29:0/881864001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 msgr2=0x5583ff9b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 -- 172.21.15.29:0/881864001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ff9b3ba0 con 0x5583ff5df390 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94cffff640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94d484f640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c7fff640 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9a4760 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005eb0 tx=0x7f94b8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e1d825d2e4f47d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c5ffb640 1 -- 172.21.15.29:0/881864001 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8058070 con 0x5583ff5df390 2024-09-21T01:58:49.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c5ffb640 1 -- 172.21.15.29:0/881864001 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b80308e0 con 0x5583ff5df390 2024-09-21T01:58:49.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.715+0000 7f94c5ffb640 1 -- 172.21.15.29:0/881864001 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b804f040 con 0x5583ff5df390 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.716+0000 7f94dc826740 1 -- 172.21.15.29:0/881864001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 msgr2=0x5583ff9a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.716+0000 7f94dc826740 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9a4760 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005eb0 tx=0x7f94b8005f90 comp rx=0 tx=0).stop 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 -- 172.21.15.29:0/881864001 shutdown_connections 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff600070 0x5583ff9a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5df390 0x5583ff9a4760 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 --2- 172.21.15.29:0/881864001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff5d71b0 0x5583ff9b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 -- 172.21.15.29:0/881864001 >> 172.21.15.29:0/881864001 conn(0x5583ff5f4460 msgr2=0x5583ff9b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.717+0000 7f94dc826740 1 -- 172.21.15.29:0/881864001 shutdown_connections 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.718+0000 7f94dc826740 1 -- 172.21.15.29:0/881864001 wait complete. 2024-09-21T01:58:49.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.718+0000 7f94dc826740 1 -- start start 2024-09-21T01:58:49.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 Processor -- start 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 0x5583ff9b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff600070 0x5583ff9b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000f880 con 0x5583ff600070 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000f1b0 con 0x5583ff5d71b0 2024-09-21T01:58:49.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000c3f0 con 0x5583ff5df390 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94dc826740 10 client.0 _ll_register_callbacks cb 0x7f94c80a9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94c7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 0x5583ff9b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46360/0 (socket says 172.21.15.29:46360) 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.719+0000 7f94cffff640 1 -- 172.21.15.29:0/1685639192 learned_addr learned my addr 172.21.15.29:0/1685639192 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94d484f640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff600070 0x5583ff9b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 -- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 msgr2=0x5583ff9bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 -- 172.21.15.29:0/1685639192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff600070 msgr2=0x5583ff9b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff600070 0x5583ff9b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 -- 172.21.15.29:0/1685639192 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ff9b77e0 con 0x5583ff5df390 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94cffff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:49.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94c7fff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 0x5583ff9b6ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009780 tx=0x7f94b8003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a4629b8bfcd8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8058070 con 0x5583ff5df390 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.720+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f94b8032c00 con 0x5583ff5df390 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b80308e0 con 0x5583ff5df390 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f94b804f040 con 0x5583ff5df390 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 10 client.27427 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:49.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5583ffa40aa0 con 0x5583ff5df390 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b8004040 con 0x5583ff5df390 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f94b8049070 con 0x5583ff5df390 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 10 client.27427 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 20 client.27427 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.721+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff593da0 0x5583ffa42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94dc826740 4 client.27427 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f94b80308e0 con 0x5583ff593da0 2024-09-21T01:58:49.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94cffff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff593da0 0x5583ffa42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94cffff640 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff593da0 0x5583ffa42c80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7be6069889e71f server_cookie=c5455083fe402cff in_seq=0 out_seq=0 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94af7fe640 10 client.27427 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94af7fe640 1 -- 172.21.15.29:0/1685639192 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f94b80308e0 con 0x5583ff593da0 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94af7fe640 10 client.27427 handle_command_reply: tid=0 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.722+0000 7f94dc826740 1 client.27427 shutdown 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 msgr2=0x5583ff9b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 0x5583ff9b6ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009780 tx=0x7f94b8003a00 comp rx=0 tx=0).stop 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 shutdown_connections 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5583ff593da0 0x5583ffa42c80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583ff600070 0x5583ff9b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583ff5df390 0x5583ff9b6ab0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 --2- 172.21.15.29:0/1685639192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583ff5d71b0 0x5583ff9bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 >> 172.21.15.29:0/1685639192 conn(0x5583ff5f4460 msgr2=0x5583ff9b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94af7fe640 0 client.27427 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:49.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 shutdown_connections 2024-09-21T01:58:49.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.723+0000 7f94dc826740 1 -- 172.21.15.29:0/1685639192 wait complete. 2024-09-21T01:58:49.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.724+0000 7f94dc826740 20 client.27427 trim_cache size 0 max 16384 2024-09-21T01:58:49.727 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 msgr2=0x7f94bc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 0x7f94bc0b8b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b802c000 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 msgr2=0x7f94d0118f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94c800ef00 tx=0x7f94c800c740 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 shutdown_connections 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f94bc0b6610 0x7f94bc0b8b20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94d0114800 0x7f94d0118f70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94d0113eb0 0x7f94d0118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 --2- 172.21.15.29:0/1356269275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d010e5c0 0x7f94d01184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 >> 172.21.15.29:0/1356269275 conn(0x7f94d010a3c0 msgr2=0x7f94d01124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 shutdown_connections 2024-09-21T01:58:49.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:49.725+0000 7f94d62b2640 1 -- 172.21.15.29:0/1356269275 wait complete. 2024-09-21T01:58:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:49 smithi029 ceph-mon[31228]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-09-21T01:58:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:49 smithi190 ceph-mon[34368]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-09-21T01:58:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:49 smithi066 ceph-mon[35001]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-09-21T01:58:51.547 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:58:51.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:58:51.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:51 smithi029 ceph-mon[31228]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T01:58:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:51 smithi190 ceph-mon[34368]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T01:58:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:51 smithi066 ceph-mon[35001]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T01:58:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 -- 172.21.15.29:0/2373987526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 msgr2=0x7f16b4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 --2- 172.21.15.29:0/2373987526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4110620 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009ca0 tx=0x7f16ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 -- 172.21.15.29:0/2373987526 shutdown_connections 2024-09-21T01:58:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 --2- 172.21.15.29:0/2373987526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4110620 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 --2- 172.21.15.29:0/2373987526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 --2- 172.21.15.29:0/2373987526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 0x7f16b4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.486+0000 7f16bad5e640 1 -- 172.21.15.29:0/2373987526 >> 172.21.15.29:0/2373987526 conn(0x7f16b407b850 msgr2=0x7f16b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.487+0000 7f16bad5e640 1 -- 172.21.15.29:0/2373987526 shutdown_connections 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.487+0000 7f16bad5e640 1 -- 172.21.15.29:0/2373987526 wait complete. 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 Processor -- start 2024-09-21T01:58:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 -- start start 2024-09-21T01:58:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 0x7f16b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4113ab0 con 0x7f16b4109df0 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4113930 con 0x7f16b41094a0 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16bad5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4113c30 con 0x7f16b4103b60 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36410/0 (socket says 172.21.15.29:36410) 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45052/0 (socket says 172.21.15.29:45052) 2024-09-21T01:58:52.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.488+0000 7f16b3fff640 1 -- 172.21.15.29:0/1041021479 learned_addr learned my addr 172.21.15.29:0/1041021479 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 0x7f16b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b3fff640 1 -- 172.21.15.29:0/1041021479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 msgr2=0x7f16b4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 0x7f16b4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b3fff640 1 -- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 msgr2=0x7f16b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.489+0000 7f16b3fff640 1 -- 172.21.15.29:0/1041021479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a8760 con 0x7f16b41094a0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.490+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b2c0 tx=0x7f16a000b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.490+0000 7f16b1ffb640 1 -- 172.21.15.29:0/1041021479 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0022070 con 0x7f16b41094a0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.490+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b419a210 con 0x7f16b41094a0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.490+0000 7f16b1ffb640 1 -- 172.21.15.29:0/1041021479 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a001b9d0 con 0x7f16b41094a0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.490+0000 7f16b1ffb640 1 -- 172.21.15.29:0/1041021479 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a001daa0 con 0x7f16b41094a0 2024-09-21T01:58:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.491+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b419a740 con 0x7f16b41094a0 2024-09-21T01:58:52.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.493+0000 7f16b1ffb640 1 -- 172.21.15.29:0/1041021479 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f16a0025070 con 0x7f16b41094a0 2024-09-21T01:58:52.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.494+0000 7f16c12d2740 1 Processor -- start 2024-09-21T01:58:52.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.494+0000 7f16c12d2740 1 -- start start 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8aec350 0x561cc8e93430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e93eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561cc8e09e60 con 0x561cc8acb670 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561cc8df9290 con 0x561cc8aec350 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16c12d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561cc8dfd0a0 con 0x561cc8ac3490 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e93eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39254/0 (socket says 172.21.15.29:39254) 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16b8afa640 1 -- 172.21.15.29:0/810357031 learned_addr learned my addr 172.21.15.29:0/810357031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.495+0000 7f16b92fb640 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8aec350 0x561cc8e93430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 -- 172.21.15.29:0/810357031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8aec350 msgr2=0x561cc8e93430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8aec350 0x561cc8e93430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 -- 172.21.15.29:0/810357031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 msgr2=0x561cc8e93eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e93eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 -- 172.21.15.29:0/810357031 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc8b57930 con 0x561cc8ac3490 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b1ffb640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 0x7f16a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b8afa640 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c20 tx=0x7f16a80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5909ad99b0a35e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b92fb640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 0x7f16a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f169effd640 1 -- 172.21.15.29:0/810357031 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8004030 con 0x561cc8ac3490 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f16b1ffb640 1 -- 172.21.15.29:0/1041021479 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f16a00eeeb0 con 0x7f16b41094a0 2024-09-21T01:58:52.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f169effd640 1 -- 172.21.15.29:0/810357031 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80035f0 con 0x561cc8ac3490 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.496+0000 7f169effd640 1 -- 172.21.15.29:0/810357031 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80126b0 con 0x561cc8ac3490 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 -- 172.21.15.29:0/810357031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 msgr2=0x561cc8e93970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16b92fb640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 0x7f16a40b8aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f16ac005e30 tx=0x7f16ac005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c20 tx=0x7f16a80117b0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 -- 172.21.15.29:0/810357031 shutdown_connections 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e93eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8ac3490 0x561cc8e93970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 --2- 172.21.15.29:0/810357031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8aec350 0x561cc8e93430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 -- 172.21.15.29:0/810357031 >> 172.21.15.29:0/810357031 conn(0x561cc8ae0740 msgr2=0x561cc8e91320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 -- 172.21.15.29:0/810357031 shutdown_connections 2024-09-21T01:58:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.497+0000 7f16c12d2740 1 -- 172.21.15.29:0/810357031 wait complete. 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 -- start start 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 Processor -- start 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e9f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8aec350 0x561cc8eaaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561cc8c5a890 con 0x561cc8acb670 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561cc8d93d40 con 0x561cc8ac3490 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561cc8ea1c10 con 0x561cc8aec350 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16c12d2740 10 client.0 _ll_register_callbacks cb 0x561cc8df7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e9f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36426/0 (socket says 172.21.15.29:36426) 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.498+0000 7f16b8afa640 1 -- 172.21.15.29:0/1457918792 learned_addr learned my addr 172.21.15.29:0/1457918792 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:52.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b92fb640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8aec350 0x561cc8eaaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 -- 172.21.15.29:0/1457918792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8aec350 msgr2=0x561cc8eaaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8aec350 0x561cc8eaaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 -- 172.21.15.29:0/1457918792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 msgr2=0x561cc8e9f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e9f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 -- 172.21.15.29:0/1457918792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc8eab570 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003e40 tx=0x7f16a8003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb28fae09253cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f1693fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a801f040 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.499+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f16a8004000 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.500+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a80035f0 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.500+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16a8004200 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.500+0000 7f16c12d2740 10 client.27479 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.500+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cc8eade80 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a8029070 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f16a8012450 con 0x561cc8ac3490 2024-09-21T01:58:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 10 client.27479 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 20 client.27479 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8aaacd0 0x561cc8eb1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 4 client.27479 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f16a80035f0 con 0x561cc8aaacd0 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.501+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8aaacd0 0x561cc8eb1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.502+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8aaacd0 0x561cc8eb1290 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7668fac972af5c73 server_cookie=d0db5fff218ca36f in_seq=0 out_seq=0 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.502+0000 7f1693fff640 10 client.27479 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.503+0000 7f1693fff640 1 -- 172.21.15.29:0/1457918792 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f16a80035f0 con 0x561cc8aaacd0 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.503+0000 7f1693fff640 10 client.27479 handle_command_reply: tid=0 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.503+0000 7f16c12d2740 1 client.27479 shutdown 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 msgr2=0x561cc8e9f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003e40 tx=0x7f16a8003e70 comp rx=0 tx=0).stop 2024-09-21T01:58:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 shutdown_connections 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8aaacd0 0x561cc8eb1290 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8aec350 0x561cc8eaaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8acb670 0x561cc8e9f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f1693fff640 0 client.27479 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1457918792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e9f070 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 >> 172.21.15.29:0/1457918792 conn(0x561cc8ae0740 msgr2=0x561cc8e96610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 shutdown_connections 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 1 -- 172.21.15.29:0/1457918792 wait complete. 2024-09-21T01:58:52.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.504+0000 7f16c12d2740 20 client.27479 trim_cache size 0 max 16384 2024-09-21T01:58:52.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.527+0000 7f16c12d2740 1 Processor -- start 2024-09-21T01:58:52.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.527+0000 7f16c12d2740 1 -- start start 2024-09-21T01:58:52.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e98540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8e98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8e9c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41ad2c0 con 0x561cc8aec350 2024-09-21T01:58:52.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41aee00 con 0x561cc8ac3490 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16c12d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41ad990 con 0x561cc8acb670 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8e9c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8e9c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45084/0 (socket says 172.21.15.29:45084) 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b92fb640 1 -- 172.21.15.29:0/3407681269 learned_addr learned my addr 172.21.15.29:0/3407681269 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8e98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b8afa640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e98540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 -- 172.21.15.29:0/3407681269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 msgr2=0x561cc8e98540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e98540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 -- 172.21.15.29:0/3407681269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 msgr2=0x561cc8e9c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8e9c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.528+0000 7f16b3fff640 1 -- 172.21.15.29:0/3407681269 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc8ea1af0 con 0x561cc8acb670 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.529+0000 7f16b8afa640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e98540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:58:52.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.529+0000 7f16b3fff640 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8e98a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f16a00b4cf0 tx=0x7f16a009c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd86bf5dfcf7df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.529+0000 7f169e7fc640 1 -- 172.21.15.29:0/3407681269 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00b4dc0 con 0x561cc8acb670 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.529+0000 7f169e7fc640 1 -- 172.21.15.29:0/3407681269 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a00bce50 con 0x561cc8acb670 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.529+0000 7f169e7fc640 1 -- 172.21.15.29:0/3407681269 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00b4760 con 0x561cc8acb670 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 -- 172.21.15.29:0/3407681269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 msgr2=0x561cc8e98a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8e98a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f16a00b4cf0 tx=0x7f16a009c040 comp rx=0 tx=0).stop 2024-09-21T01:58:52.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 -- 172.21.15.29:0/3407681269 shutdown_connections 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8e9c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8e98a80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 --2- 172.21.15.29:0/3407681269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8e98540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.530+0000 7f16c12d2740 1 -- 172.21.15.29:0/3407681269 >> 172.21.15.29:0/3407681269 conn(0x561cc8ae0740 msgr2=0x561cc8e9d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.531+0000 7f16c12d2740 1 -- 172.21.15.29:0/3407681269 shutdown_connections 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.531+0000 7f16c12d2740 1 -- 172.21.15.29:0/3407681269 wait complete. 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.531+0000 7f16c12d2740 1 -- start start 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.531+0000 7f16c12d2740 1 Processor -- start 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.531+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8ea3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41ae730 con 0x561cc8aec350 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41ae060 con 0x561cc8ac3490 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41ab2a0 con 0x561cc8acb670 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16c12d2740 10 client.0 _ll_register_callbacks cb 0x7f16a00f27c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36442/0 (socket says 172.21.15.29:36442) 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b92fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45100/0 (socket says 172.21.15.29:45100) 2024-09-21T01:58:52.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 -- 172.21.15.29:0/1957404376 learned_addr learned my addr 172.21.15.29:0/1957404376 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 -- 172.21.15.29:0/1957404376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 msgr2=0x561cc8ea3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8ea3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8ea3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 -- 172.21.15.29:0/1957404376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 msgr2=0x561cc8f1aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 -- 172.21.15.29:0/1957404376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc8f1b050 con 0x561cc8ac3490 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b92fb640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f16b8afa640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003e40 tx=0x7f16a8003b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f117b16cd0aaac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:52.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.532+0000 7f1693fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009870 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f16a8012450 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a00bce50 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16a80035f0 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 10 client.27485 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cc8f1c310 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a00bce50 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f16a8036030 con 0x561cc8ac3490 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 10 client.27485 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 10 client.27485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 20 client.27485 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8a80080 0x561cc8f1ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 4 client.27485 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:58:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.533+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561cc8e0bdb0 con 0x561cc8a80080 2024-09-21T01:58:52.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.534+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8a80080 0x561cc8f1ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:52.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.534+0000 7f1693fff640 10 client.27485 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:52.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.534+0000 7f16b3fff640 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8a80080 0x561cc8f1ea40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2f204fcf2bc7d4 server_cookie=ef234f0d31ac4830 in_seq=0 out_seq=0 2024-09-21T01:58:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.536+0000 7f1693fff640 1 -- 172.21.15.29:0/1957404376 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561cc8e0bdb0 con 0x561cc8a80080 2024-09-21T01:58:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.536+0000 7f1693fff640 10 client.27485 handle_command_reply: tid=0 2024-09-21T01:58:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.537+0000 7f16c12d2740 1 client.27485 shutdown 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.537+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 msgr2=0x561cc8ea35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.537+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003e40 tx=0x7f16a8003b30 comp rx=0 tx=0).stop 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.537+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 shutdown_connections 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561cc8a80080 0x561cc8f1ea40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cc8aec350 0x561cc8f1aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f1693fff640 0 client.27485 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561cc8acb670 0x561cc8ea3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 --2- 172.21.15.29:0/1957404376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561cc8ac3490 0x561cc8ea35f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 >> 172.21.15.29:0/1957404376 conn(0x561cc8ae0740 msgr2=0x561cc8e9d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 shutdown_connections 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 1 -- 172.21.15.29:0/1957404376 wait complete. 2024-09-21T01:58:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.538+0000 7f16c12d2740 20 client.27485 trim_cache size 0 max 16384 2024-09-21T01:58:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:58:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:58:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "40467f92-f1af-4b93-ac87-06d10971b4d5", 2024-09-21T01:58:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:58:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:58:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 msgr2=0x7f16a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 0x7f16a40b8aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f16ac005e30 tx=0x7f16ac005c30 comp rx=0 tx=0).stop 2024-09-21T01:58:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 msgr2=0x7f16b4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b2c0 tx=0x7f16a000b780 comp rx=0 tx=0).stop 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 shutdown_connections 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f16a40b6590 0x7f16a40b8aa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16b4109df0 0x7f16b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b41094a0 0x7f16b4075c50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 --2- 172.21.15.29:0/1041021479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b4103b60 0x7f16b4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.539+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 >> 172.21.15.29:0/1041021479 conn(0x7f16b407b850 msgr2=0x7f16b41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.540+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 shutdown_connections 2024-09-21T01:58:52.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:52.540+0000 7f16bad5e640 1 -- 172.21.15.29:0/1041021479 wait complete. 2024-09-21T01:58:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 615.0 s, 1418.6 tps, lat 69.489 ms stddev 81.931 2024-09-21T01:58:53.335 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:58:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:54.062 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:54.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:58:54.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.063 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:53 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-21T01:58:54.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:53 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:53 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 -- 172.21.15.29:0/394132708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010aee0 msgr2=0x7fef30115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 --2- 172.21.15.29:0/394132708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010aee0 0x7fef30115fd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009860 tx=0x7fef2c02a340 comp rx=0 tx=0).stop 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 -- 172.21.15.29:0/394132708 shutdown_connections 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 --2- 172.21.15.29:0/394132708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef30118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 --2- 172.21.15.29:0/394132708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010aee0 0x7fef30115fd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 --2- 172.21.15.29:0/394132708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010a590 0x7fef3010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 -- 172.21.15.29:0/394132708 >> 172.21.15.29:0/394132708 conn(0x7fef30070ec0 msgr2=0x7fef300712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 -- 172.21.15.29:0/394132708 shutdown_connections 2024-09-21T01:58:54.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.313+0000 7fef3735d640 1 -- 172.21.15.29:0/394132708 wait complete. 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 Processor -- start 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 -- start start 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010a590 0x7fef301a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 0x7fef301a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef301a8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef3011c280 con 0x7fef3010a590 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef3011c100 con 0x7fef30116640 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.314+0000 7fef3735d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef3011c400 con 0x7fef3010aee0 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef348f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 0x7fef301a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef301a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef348f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 0x7fef301a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:39302/0 (socket says 172.21.15.29:39302) 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef348f8640 1 -- 172.21.15.29:0/129569784 learned_addr learned my addr 172.21.15.29:0/129569784 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 -- 172.21.15.29:0/129569784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 msgr2=0x7fef301a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 0x7fef301a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 -- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010a590 msgr2=0x7fef301a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010a590 0x7fef301a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 -- 172.21.15.29:0/129569784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef301a8c60 con 0x7fef30116640 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.315+0000 7fef358fa640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef301a8530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fef2800da10 tx=0x7fef2800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.316+0000 7fef267fc640 1 -- 172.21.15.29:0/129569784 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2800bd70 con 0x7fef30116640 2024-09-21T01:58:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.316+0000 7fef267fc640 1 -- 172.21.15.29:0/129569784 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef28003e20 con 0x7fef30116640 2024-09-21T01:58:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.316+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef301a8f50 con 0x7fef30116640 2024-09-21T01:58:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.316+0000 7fef267fc640 1 -- 172.21.15.29:0/129569784 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280106c0 con 0x7fef30116640 2024-09-21T01:58:54.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.316+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3010bd60 con 0x7fef30116640 2024-09-21T01:58:54.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.319+0000 7fef267fc640 1 -- 172.21.15.29:0/129569784 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fef28004270 con 0x7fef30116640 2024-09-21T01:58:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.320+0000 7fef3d8d1740 1 Processor -- start 2024-09-21T01:58:54.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 -- start start 2024-09-21T01:58:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 0x55d1349a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef267fc640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 0x7fef1c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef350f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 0x55d1349a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d13491ce50 con 0x55d1345d61b0 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d13490c280 con 0x55d1345ff070 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d134910090 con 0x55d1345de390 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef350f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 0x55d1349a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36484/0 (socket says 172.21.15.29:36484) 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.321+0000 7fef350f9640 1 -- 172.21.15.29:0/1980687003 learned_addr learned my addr 172.21.15.29:0/1980687003 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef348f8640 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef267fc640 1 -- 172.21.15.29:0/129569784 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fef280e7720 con 0x7fef30116640 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef348f8640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 0x7fef1c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 -- 172.21.15.29:0/1980687003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 msgr2=0x55d1349a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 0x55d1349a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 -- 172.21.15.29:0/1980687003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 msgr2=0x55d1349a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.322+0000 7fef358fa640 1 -- 172.21.15.29:0/1980687003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13466a650 con 0x55d1345de390 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.323+0000 7fef358fa640 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349a70d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef280ad7f0 tx=0x7fef28004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291a480cd477e692 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.323+0000 7fef1b7fe640 1 -- 172.21.15.29:0/1980687003 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28080070 con 0x55d1345de390 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.323+0000 7fef348f8640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 0x7fef1c0b8aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef2c000c00 tx=0x7fef2c030340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.323+0000 7fef1b7fe640 1 -- 172.21.15.29:0/1980687003 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef280ad370 con 0x55d1345de390 2024-09-21T01:58:54.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.323+0000 7fef1b7fe640 1 -- 172.21.15.29:0/1980687003 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28082a00 con 0x55d1345de390 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 -- 172.21.15.29:0/1980687003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 msgr2=0x55d1349a70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349a70d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef280ad7f0 tx=0x7fef28004610 comp rx=0 tx=0).stop 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 -- 172.21.15.29:0/1980687003 shutdown_connections 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349a70d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/1980687003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345ff070 0x55d1349a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 -- 172.21.15.29:0/1980687003 >> 172.21.15.29:0/1980687003 conn(0x55d1345f3460 msgr2=0x55d1349a4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 -- 172.21.15.29:0/1980687003 shutdown_connections 2024-09-21T01:58:54.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.326+0000 7fef3d8d1740 1 -- 172.21.15.29:0/1980687003 wait complete. 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.327+0000 7fef3d8d1740 1 -- start start 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.327+0000 7fef3d8d1740 1 Processor -- start 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d1349be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d13476d700 con 0x55d1345ff070 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d1348a7110 con 0x55d1345d61b0 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d1349b4e30 con 0x55d1345de390 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36492/0 (socket says 172.21.15.29:36492) 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 -- 172.21.15.29:0/3346035295 learned_addr learned my addr 172.21.15.29:0/3346035295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef3d8d1740 10 client.0 _ll_register_callbacks cb 0x55d1348c0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef358fa640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d1349be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef348f8640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 -- 172.21.15.29:0/3346035295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 msgr2=0x55d1349b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 -- 172.21.15.29:0/3346035295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 msgr2=0x55d1349be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d1349be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.328+0000 7fef350f9640 1 -- 172.21.15.29:0/3346035295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1348d4c40 con 0x55d1345d61b0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef350f9640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef2000f670 tx=0x7fef2000fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0991d956682b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:54.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20026060 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fef20004030 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef280ad370 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef20004270 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef3d8d1740 10 client.27497 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d1349c0b80 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef280ad370 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.329+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fef20003a80 con 0x55d1345d61b0 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 10 client.27497 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 10 client.27497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 20 client.27497 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d1345bd9f0 0x55d1349c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 4 client.27497 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:58:54.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d13491eda0 con 0x55d1345bd9f0 2024-09-21T01:58:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef348f8640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d1345bd9f0 0x55d1349c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef348f8640 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d1345bd9f0 0x55d1349c40b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d6d987816bf231 server_cookie=18f99764f09ecd77 in_seq=0 out_seq=0 2024-09-21T01:58:54.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.330+0000 7fef18ff9640 10 client.27497 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef18ff9640 1 -- 172.21.15.29:0/3346035295 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d13491eda0 con 0x55d1345bd9f0 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef18ff9640 10 client.27497 handle_command_reply: tid=0 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 client.27497 shutdown 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 msgr2=0x55d1349b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef2000f670 tx=0x7fef2000fb30 comp rx=0 tx=0).stop 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 shutdown_connections 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d1345bd9f0 0x55d1349c40b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d1349be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345de390 0x55d1349b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef18ff9640 0 client.27497 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/3346035295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345d61b0 0x55d1349b1d70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.331+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 >> 172.21.15.29:0/3346035295 conn(0x55d1345f3460 msgr2=0x55d1349a8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.332+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 shutdown_connections 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.332+0000 7fef3d8d1740 1 -- 172.21.15.29:0/3346035295 wait complete. 2024-09-21T01:58:54.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.332+0000 7fef3d8d1740 20 client.27497 trim_cache size 0 max 16384 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.360+0000 7fef3d8d1740 1 Processor -- start 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 -- start start 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345ff070 0x55d1349aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b59c0 con 0x55d1345d61b0 2024-09-21T01:58:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b7500 con 0x55d1345de390 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b6090 con 0x55d1345ff070 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45162/0 (socket says 172.21.15.29:45162) 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 -- 172.21.15.29:0/235333024 learned_addr learned my addr 172.21.15.29:0/235333024 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef358fa640 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345ff070 0x55d1349aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef348f8640 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 -- 172.21.15.29:0/235333024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345ff070 msgr2=0x55d1349aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345ff070 0x55d1349aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 -- 172.21.15.29:0/235333024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 msgr2=0x55d1349ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.361+0000 7fef350f9640 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.362+0000 7fef350f9640 1 -- 172.21.15.29:0/235333024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1349b0cd0 con 0x55d1345d61b0 2024-09-21T01:58:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.362+0000 7fef350f9640 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef20003ef0 tx=0x7fef20009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18471f2de983e7da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.362+0000 7fef1affd640 1 -- 172.21.15.29:0/235333024 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20039020 con 0x55d1345d61b0 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.362+0000 7fef1affd640 1 -- 172.21.15.29:0/235333024 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2000ed40 con 0x55d1345d61b0 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.362+0000 7fef1affd640 1 -- 172.21.15.29:0/235333024 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20031070 con 0x55d1345d61b0 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 -- 172.21.15.29:0/235333024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 msgr2=0x55d1349b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef20003ef0 tx=0x7fef20009e20 comp rx=0 tx=0).stop 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 -- 172.21.15.29:0/235333024 shutdown_connections 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345ff070 0x55d1349aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/235333024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345d61b0 0x55d1349b2230 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 -- 172.21.15.29:0/235333024 >> 172.21.15.29:0/235333024 conn(0x55d1345f3460 msgr2=0x55d1349b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 -- 172.21.15.29:0/235333024 shutdown_connections 2024-09-21T01:58:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.363+0000 7fef3d8d1740 1 -- 172.21.15.29:0/235333024 wait complete. 2024-09-21T01:58:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.364+0000 7fef3d8d1740 1 -- start start 2024-09-21T01:58:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.364+0000 7fef3d8d1740 1 Processor -- start 2024-09-21T01:58:54.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345d61b0 0x55d1349b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d134a27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b6e30 con 0x55d1345ff070 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b6760 con 0x55d1345de390 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef301b39a0 con 0x55d1345d61b0 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef3d8d1740 10 client.0 _ll_register_callbacks cb 0x7fef20009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef348f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef348f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36510/0 (socket says 172.21.15.29:36510) 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.365+0000 7fef348f8640 1 -- 172.21.15.29:0/4008489258 learned_addr learned my addr 172.21.15.29:0/4008489258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 -- 172.21.15.29:0/4008489258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345d61b0 msgr2=0x55d1349b6910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345d61b0 0x55d1349b6910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 -- 172.21.15.29:0/4008489258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 msgr2=0x55d134a27a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:58:54.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d134a27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 -- 172.21.15.29:0/4008489258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d134a28100 con 0x55d1345de390 2024-09-21T01:58:54.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.366+0000 7fef348f8640 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef2c035040 tx=0x7fef2c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454f222107580e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.367+0000 7fef18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.367+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c06a070 con 0x55d1345de390 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.367+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fef2c030510 con 0x55d1345de390 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.367+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2000ed40 con 0x55d1345de390 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.368+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef2c074050 con 0x55d1345de390 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.368+0000 7fef3d8d1740 10 client.27503 fetch_fsmap learned FSMap version 45 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.368+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d134a2f460 con 0x55d1345de390 2024-09-21T01:58:54.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2000ed40 con 0x55d1345de390 2024-09-21T01:58:54.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fef2c040480 con 0x55d1345de390 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 10 client.27503 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 10 client.27503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 20 client.27503 populate_metadata read hostname 'smithi029' 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d134592da0 0x55d134a31c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 4 client.27503 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.369+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d13491a850 con 0x55d134592da0 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef350f9640 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d134592da0 0x55d134a31c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef350f9640 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d134592da0 0x55d134a31c40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3dc9af94147b2b server_cookie=97b43bce94b65c58 in_seq=0 out_seq=0 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef18ff9640 10 client.27503 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef18ff9640 1 -- 172.21.15.29:0/4008489258 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55d13491a850 con 0x55d134592da0 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef18ff9640 10 client.27503 handle_command_reply: tid=0 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.370+0000 7fef3d8d1740 1 client.27503 shutdown 2024-09-21T01:58:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 msgr2=0x55d1349b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef2c035040 tx=0x7fef2c005f90 comp rx=0 tx=0).stop 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 shutdown_connections 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d134592da0 0x55d134a31c40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1345ff070 0x55d134a27a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1345de390 0x55d1349b6ef0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef2c035040 tx=0x7fef2c005f90 comp rx=0 tx=0).stop 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 --2- 172.21.15.29:0/4008489258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1345d61b0 0x55d1349b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 >> 172.21.15.29:0/4008489258 conn(0x55d1345f3460 msgr2=0x55d1349a7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef18ff9640 0 client.27503 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 shutdown_connections 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 1 -- 172.21.15.29:0/4008489258 wait complete. 2024-09-21T01:58:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.371+0000 7fef3d8d1740 20 client.27503 trim_cache size 0 max 16384 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "40467f92-f1af-4b93-ac87-06d10971b4d5", 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:58:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:58:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:58:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:58:54.382 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 msgr2=0x7fef1c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 0x7fef1c0b8aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef2c000c00 tx=0x7fef2c030340 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 msgr2=0x7fef301a8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef301a8530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fef2800da10 tx=0x7fef2800ded0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 shutdown_connections 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fef1c0b6590 0x7fef1c0b8aa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef30116640 0x7fef301a8530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef3010aee0 0x7fef301a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 --2- 172.21.15.29:0/129569784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef3010a590 0x7fef301a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:54.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 >> 172.21.15.29:0/129569784 conn(0x7fef30070ec0 msgr2=0x7fef30073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:54.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 shutdown_connections 2024-09-21T01:58:54.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:58:54.372+0000 7fef3735d640 1 -- 172.21.15.29:0/129569784 wait complete. 2024-09-21T01:58:55.244 INFO:tasks.cephfs.filesystem:scrub status for tag:40467f92-f1af-4b93-ac87-06d10971b4d5 - {'path': '/', 'tag': '40467f92-f1af-4b93-ac87-06d10971b4d5', 'options': 'recursive,force'} 2024-09-21T01:58:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:55 smithi029 ceph-mon[31228]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T01:58:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:55 smithi190 ceph-mon[34368]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T01:58:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:55 smithi066 ceph-mon[35001]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T01:58:57.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:56 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:57.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:56 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:57.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:56 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:58:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 620.0 s, 1399.6 tps, lat 73.164 ms stddev 85.866 2024-09-21T01:58:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:57 smithi029 ceph-mon[31228]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-09-21T01:58:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:57 smithi190 ceph-mon[34368]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-09-21T01:58:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:57 smithi066 ceph-mon[35001]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-09-21T01:59:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:58:59 smithi029 ceph-mon[31228]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-09-21T01:59:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:58:59 smithi190 ceph-mon[34368]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-09-21T01:59:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:58:59 smithi066 ceph-mon[35001]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-09-21T01:59:01.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:01.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:01.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:01.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:00 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:01.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:00 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:00 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:01 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:01 smithi190 ceph-mon[34368]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-09-21T01:59:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:01 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:59:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:01 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:01 smithi066 ceph-mon[35001]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-09-21T01:59:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:01 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:59:02.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:01 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:02.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:01 smithi029 ceph-mon[31228]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-09-21T01:59:02.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:01 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:59:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.527+0000 7f4320f2b640 1 -- 172.21.15.29:0/1645888231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 msgr2=0x7f431c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.527+0000 7f4320f2b640 1 --2- 172.21.15.29:0/1645888231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c10a990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4314009ca0 tx=0x7f431402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 -- 172.21.15.29:0/1645888231 shutdown_connections 2024-09-21T01:59:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 --2- 172.21.15.29:0/1645888231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c10a990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 --2- 172.21.15.29:0/1645888231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c103650 0x7f431c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 --2- 172.21.15.29:0/1645888231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c1025d0 0x7f431c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 -- 172.21.15.29:0/1645888231 >> 172.21.15.29:0/1645888231 conn(0x7f431c0fdf30 msgr2=0x7f431c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 -- 172.21.15.29:0/1645888231 shutdown_connections 2024-09-21T01:59:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.528+0000 7f4320f2b640 1 -- 172.21.15.29:0/1645888231 wait complete. 2024-09-21T01:59:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 Processor -- start 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 -- start start 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c1025d0 0x7f431c197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 0x7f431c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10d190 con 0x7f431c103ff0 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10d010 con 0x7f431c103650 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4320f2b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10d310 con 0x7f431c1025d0 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59330/0 (socket says 172.21.15.29:59330) 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f431ad9d640 1 -- 172.21.15.29:0/3849431066 learned_addr learned my addr 172.21.15.29:0/3849431066 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 0x7f431c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f431a59c640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c1025d0 0x7f431c197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4319d9b640 1 -- 172.21.15.29:0/3849431066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c1025d0 msgr2=0x7f431c197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.529+0000 7f4319d9b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c1025d0 0x7f431c197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4319d9b640 1 -- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 msgr2=0x7f431c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4319d9b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4319d9b640 1 -- 172.21.15.29:0/3849431066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c199750 con 0x7f431c103650 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4319d9b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 0x7f431c198380 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f431000e9f0 tx=0x7f431000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f430b7fe640 1 -- 172.21.15.29:0/3849431066 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431000ccc0 con 0x7f431c103650 2024-09-21T01:59:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c199a40 con 0x7f431c103650 2024-09-21T01:59:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.530+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c19a100 con 0x7f431c103650 2024-09-21T01:59:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.531+0000 7f430b7fe640 1 -- 172.21.15.29:0/3849431066 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310005020 con 0x7f431c103650 2024-09-21T01:59:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.531+0000 7f430b7fe640 1 -- 172.21.15.29:0/3849431066 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310004a00 con 0x7f431c103650 2024-09-21T01:59:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.534+0000 7f430b7fe640 1 -- 172.21.15.29:0/3849431066 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4310004ba0 con 0x7f431c103650 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 Processor -- start 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 -- start start 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f430b7fe640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 0x7f43040b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a5498e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a549a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f430b7fe640 1 -- 172.21.15.29:0/3849431066 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f43100e7150 con 0x7f431c103650 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5637a540b6f0 con 0x5637a50c82f0 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5637a53fab20 con 0x5637a50f11b0 2024-09-21T01:59:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4327e9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5637a53fe930 con 0x5637a50d04d0 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f431ad9d640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 0x7f43040b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a549a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.535+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59332/0 (socket says 172.21.15.29:59332) 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a549a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48422/0 (socket says 172.21.15.29:48422) 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 -- 172.21.15.29:0/1873158462 learned_addr learned my addr 172.21.15.29:0/1873158462 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f431a59c640 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a5498e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 -- 172.21.15.29:0/1873158462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 msgr2=0x5637a549a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a549a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 -- 172.21.15.29:0/1873158462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 msgr2=0x5637a5498e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a5498e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 -- 172.21.15.29:0/1873158462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a53c0530 con 0x5637a50c82f0 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f431a59c640 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a5498e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f43100aa880 tx=0x7f43100ada20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb8d75797a2c634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f431ad9d640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 0x7f43040b8a90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4314005e30 tx=0x7f431402f3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4308ff9640 1 -- 172.21.15.29:0/1873158462 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431001c040 con 0x5637a50c82f0 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4308ff9640 1 -- 172.21.15.29:0/1873158462 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100b2d90 con 0x5637a50c82f0 2024-09-21T01:59:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.536+0000 7f4308ff9640 1 -- 172.21.15.29:0/1873158462 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100ad1e0 con 0x5637a50c82f0 2024-09-21T01:59:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.537+0000 7f4327e9a740 1 -- 172.21.15.29:0/1873158462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 msgr2=0x5637a54994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f43100aa880 tx=0x7f43100ada20 comp rx=0 tx=0).stop 2024-09-21T01:59:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 -- 172.21.15.29:0/1873158462 shutdown_connections 2024-09-21T01:59:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a549a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54994c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1873158462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a5498e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 -- 172.21.15.29:0/1873158462 >> 172.21.15.29:0/1873158462 conn(0x5637a50e5300 msgr2=0x5637a54948e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 -- 172.21.15.29:0/1873158462 shutdown_connections 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 -- 172.21.15.29:0/1873158462 wait complete. 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 -- start start 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.538+0000 7f4327e9a740 1 Processor -- start 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50c82f0 0x5637a54a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a54a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5637a5332ab0 con 0x5637a50d04d0 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5637a53a0730 con 0x5637a50f11b0 2024-09-21T01:59:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5637a549ab30 con 0x5637a50c82f0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4327e9a740 10 client.0 _ll_register_callbacks cb 0x5637a53f46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59346/0 (socket says 172.21.15.29:59346) 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f431a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50c82f0 0x5637a54a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 -- 172.21.15.29:0/2007998010 learned_addr learned my addr 172.21.15.29:0/2007998010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f431ad9d640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a54a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 -- 172.21.15.29:0/2007998010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50c82f0 msgr2=0x5637a54a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50c82f0 0x5637a54a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 -- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 msgr2=0x5637a54a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a54a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 -- 172.21.15.29:0/2007998010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a515c790 con 0x5637a50d04d0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4319d9b640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f43100adf60 tx=0x7f43100aa030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736e394cc530b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.539+0000 7f4301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100c7020 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f43100ad180 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43100b2d90 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f431001c070 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 10 client.17700 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5637a51d5620 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100b2d90 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f43100b7210 con 0x5637a50d04d0 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 10 client.17700 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 10 client.17700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 20 client.17700 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50afd80 0x5637a54af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 4 client.17700 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.540+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a540d640 con 0x5637a50afd80 2024-09-21T01:59:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.541+0000 7f431a59c640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50afd80 0x5637a54af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.541+0000 7f431a59c640 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50afd80 0x5637a54af8c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc5fb3560c7bf9a server_cookie=a5b6aa6460012413 in_seq=0 out_seq=0 2024-09-21T01:59:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.541+0000 7f4301ffb640 10 client.17700 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4301ffb640 1 -- 172.21.15.29:0/2007998010 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637a540d640 con 0x5637a50afd80 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4301ffb640 10 client.17700 handle_command_reply: tid=0 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4327e9a740 1 client.17700 shutdown 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 msgr2=0x5637a54a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f43100adf60 tx=0x7f43100aa030 comp rx=0 tx=0).stop 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 shutdown_connections 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.543+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50afd80 0x5637a54af8c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4301ffb640 0 client.17700 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50f11b0 0x5637a54a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50d04d0 0x5637a54a2c70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 --2- 172.21.15.29:0/2007998010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50c82f0 0x5637a54a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 >> 172.21.15.29:0/2007998010 conn(0x5637a50e5300 msgr2=0x5637a5492ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 shutdown_connections 2024-09-21T01:59:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 1 -- 172.21.15.29:0/2007998010 wait complete. 2024-09-21T01:59:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.544+0000 7f4327e9a740 20 client.17700 trim_cache size 0 max 16384 2024-09-21T01:59:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 Processor -- start 2024-09-21T01:59:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 -- start start 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00e3e0 con 0x5637a50c82f0 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00ffe0 con 0x5637a50afd80 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.564+0000 7f4327e9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00eab0 con 0x5637a50d04d0 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f431a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f431a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54a3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:33694/0 (socket says 172.21.15.29:33694) 2024-09-21T01:59:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f431a59c640 1 -- 172.21.15.29:0/99255818 learned_addr learned my addr 172.21.15.29:0/99255818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59356/0 (socket says 172.21.15.29:59356) 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f431ad9d640 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 -- 172.21.15.29:0/99255818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 msgr2=0x5637a54a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 -- 172.21.15.29:0/99255818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 msgr2=0x5637a54a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 -- 172.21.15.29:0/99255818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a54a3b50 con 0x5637a50c82f0 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4319d9b640 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f43100e4fd0 tx=0x7f43100ada20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edac316fd11dd02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4303fff640 1 -- 172.21.15.29:0/99255818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100c7070 con 0x5637a50c82f0 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4303fff640 1 -- 172.21.15.29:0/99255818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100ce030 con 0x5637a50c82f0 2024-09-21T01:59:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.565+0000 7f4303fff640 1 -- 172.21.15.29:0/99255818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100a89b0 con 0x5637a50c82f0 2024-09-21T01:59:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.566+0000 7f4327e9a740 1 -- 172.21.15.29:0/99255818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 msgr2=0x5637a54aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.566+0000 7f4327e9a740 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f43100e4fd0 tx=0x7f43100ada20 comp rx=0 tx=0).stop 2024-09-21T01:59:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 -- 172.21.15.29:0/99255818 shutdown_connections 2024-09-21T01:59:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54aa6c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 --2- 172.21.15.29:0/99255818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 -- 172.21.15.29:0/99255818 >> 172.21.15.29:0/99255818 conn(0x5637a50e5300 msgr2=0x5637a548f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 -- 172.21.15.29:0/99255818 shutdown_connections 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 -- 172.21.15.29:0/99255818 wait complete. 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 -- start start 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.567+0000 7f4327e9a740 1 Processor -- start 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00f850 con 0x5637a50c82f0 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00f180 con 0x5637a50afd80 2024-09-21T01:59:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc00c3c0 con 0x5637a50d04d0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4327e9a740 10 client.0 _ll_register_callbacks cb 0x7f4310005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59366/0 (socket says 172.21.15.29:59366) 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 -- 172.21.15.29:0/1634550895 learned_addr learned my addr 172.21.15.29:0/1634550895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f431ad9d640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f431a59c640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 -- 172.21.15.29:0/1634550895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 msgr2=0x5637a54a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 -- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 msgr2=0x5637a54aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 -- 172.21.15.29:0/1634550895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a54a9c20 con 0x5637a50c82f0 2024-09-21T01:59:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f431a59c640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4319d9b640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f43100b7580 tx=0x7f43100ad420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22303bec909bfe3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.568+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100c7070 con 0x5637a50c82f0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f43100b3f20 con 0x5637a50c82f0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43100ce030 con 0x5637a50c82f0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f43100c30d0 con 0x5637a50c82f0 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4327e9a740 10 client.17706 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5637a54a9c20 con 0x5637a50c82f0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.569+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100991e0 con 0x5637a50c82f0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f431008a020 con 0x5637a50c82f0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 10 client.17706 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 10 client.17706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 20 client.17706 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50f11b0 0x5637a551c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 4 client.17706 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.570+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f43100ce030 con 0x5637a50f11b0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.571+0000 7f431a59c640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50f11b0 0x5637a551c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.572+0000 7f431a59c640 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50f11b0 0x5637a551c940 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf3bbd08193c1fc server_cookie=ffe31cdd2f6f8924 in_seq=0 out_seq=0 2024-09-21T01:59:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.572+0000 7f4301ffb640 10 client.17706 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4301ffb640 1 -- 172.21.15.29:0/1634550895 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f43100ce030 con 0x5637a50f11b0 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4301ffb640 10 client.17706 handle_command_reply: tid=0 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 client.17706 shutdown 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 msgr2=0x5637a54a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f43100b7580 tx=0x7f43100ad420 comp rx=0 tx=0).stop 2024-09-21T01:59:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 shutdown_connections 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5637a50f11b0 0x5637a551c940 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637a50d04d0 0x5637a54a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5637a50c82f0 0x5637a54a8fa0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 --2- 172.21.15.29:0/1634550895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637a50afd80 0x5637a54aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 >> 172.21.15.29:0/1634550895 conn(0x5637a50e5300 msgr2=0x5637a5495420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4301ffb640 0 client.17706 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 shutdown_connections 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.585+0000 7f4327e9a740 1 -- 172.21.15.29:0/1634550895 wait complete. 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.586+0000 7f4327e9a740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T01:59:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:59:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:59:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:59:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:59:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.586+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 msgr2=0x7f43040b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.586+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 0x7f43040b8a90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4314005e30 tx=0x7f431402f3b0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.586+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 msgr2=0x7f431c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.586+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 0x7f431c198380 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f431000e9f0 tx=0x7f431000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 shutdown_connections 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f43040b6580 0x7f43040b8a90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f431c103ff0 0x7f431c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f431c103650 0x7f431c198380 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 --2- 172.21.15.29:0/3849431066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431c1025d0 0x7f431c197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 >> 172.21.15.29:0/3849431066 conn(0x7f431c0fdf30 msgr2=0x7f431c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 shutdown_connections 2024-09-21T01:59:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:02.587+0000 7f4320f2b640 1 -- 172.21.15.29:0/3849431066 wait complete. 2024-09-21T01:59:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 625.0 s, 2040.4 tps, lat 48.680 ms stddev 62.893 2024-09-21T01:59:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:03 smithi190 ceph-mon[34368]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 708 op/s 2024-09-21T01:59:04.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:03 smithi066 ceph-mon[35001]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 708 op/s 2024-09-21T01:59:04.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:03 smithi029 ceph-mon[31228]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 708 op/s 2024-09-21T01:59:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:05 smithi190 ceph-mon[34368]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.7 MiB/s wr, 693 op/s 2024-09-21T01:59:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:05 smithi066 ceph-mon[35001]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.7 MiB/s wr, 693 op/s 2024-09-21T01:59:06.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:05 smithi029 ceph-mon[31228]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.7 MiB/s wr, 693 op/s 2024-09-21T01:59:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 630.0 s, 2606.2 tps, lat 38.853 ms stddev 47.352 2024-09-21T01:59:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:07 smithi190 ceph-mon[34368]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.9 MiB/s wr, 806 op/s 2024-09-21T01:59:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:07 smithi066 ceph-mon[35001]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.9 MiB/s wr, 806 op/s 2024-09-21T01:59:08.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:07 smithi029 ceph-mon[31228]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.9 MiB/s wr, 806 op/s 2024-09-21T01:59:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:09 smithi190 ceph-mon[34368]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-09-21T01:59:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:09 smithi066 ceph-mon[35001]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-09-21T01:59:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:09 smithi029 ceph-mon[31228]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-09-21T01:59:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:11 smithi190 ceph-mon[34368]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T01:59:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:11 smithi066 ceph-mon[35001]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T01:59:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:11 smithi029 ceph-mon[31228]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T01:59:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 635.0 s, 1830.2 tps, lat 54.207 ms stddev 58.574 2024-09-21T01:59:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:13 smithi190 ceph-mon[34368]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-09-21T01:59:14.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:13 smithi066 ceph-mon[35001]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-09-21T01:59:14.346 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:13 smithi029 ceph-mon[31228]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 -- 172.21.15.29:0/2283206809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300006b710 msgr2=0x7f30001040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 --2- 172.21.15.29:0/2283206809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300006b710 0x7f30001040d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009900 tx=0x7f2ffc02a410 comp rx=0 tx=0).stop 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 -- 172.21.15.29:0/2283206809 shutdown_connections 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 --2- 172.21.15.29:0/2283206809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30001071d0 0x7f30001076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 --2- 172.21.15.29:0/2283206809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3000104610 0x7f3000106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 --2- 172.21.15.29:0/2283206809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300006b710 0x7f30001040d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 -- 172.21.15.29:0/2283206809 >> 172.21.15.29:0/2283206809 conn(0x7f30000fdee0 msgr2=0x7f3000100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 -- 172.21.15.29:0/2283206809 shutdown_connections 2024-09-21T01:59:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.723+0000 7f300824d640 1 -- 172.21.15.29:0/2283206809 wait complete. 2024-09-21T01:59:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.724+0000 7f300824d640 1 Processor -- start 2024-09-21T01:59:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.724+0000 7f300824d640 1 -- start start 2024-09-21T01:59:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.725+0000 7f300824d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f300006b710 0x7f30001a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.725+0000 7f300824d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f300824d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30001071d0 0x7f30001a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51852/0 (socket says 172.21.15.29:51852) 2024-09-21T01:59:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 -- 172.21.15.29:0/4286990414 learned_addr learned my addr 172.21.15.29:0/4286990414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3000115900 con 0x7f300006b710 2024-09-21T01:59:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3000115780 con 0x7f3000104610 2024-09-21T01:59:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3000115a80 con 0x7f30001071d0 2024-09-21T01:59:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30067ea640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30001071d0 0x7f30001a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 -- 172.21.15.29:0/4286990414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30001071d0 msgr2=0x7f30001a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30001071d0 0x7f30001a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 -- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f300006b710 msgr2=0x7f30001a05f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.726+0000 7f30057e8640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f300006b710 0x7f30001a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f30057e8640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001a1f30 con 0x7f3000104610 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f30057e8640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c970 tx=0x7f2ff000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f2ff6ffd640 1 -- 172.21.15.29:0/4286990414 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0014070 con 0x7f3000104610 2024-09-21T01:59:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f2ff6ffd640 1 -- 172.21.15.29:0/4286990414 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff0004440 con 0x7f3000104610 2024-09-21T01:59:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f2ff6ffd640 1 -- 172.21.15.29:0/4286990414 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0003e50 con 0x7f3000104610 2024-09-21T01:59:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001a2220 con 0x7f3000104610 2024-09-21T01:59:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.727+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001a87c0 con 0x7f3000104610 2024-09-21T01:59:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.731+0000 7f300f1bd740 1 Processor -- start 2024-09-21T01:59:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.731+0000 7f300f1bd740 1 -- start start 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4439e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df443b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559df43ac6f0 con 0x559df40921b0 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559df439bb20 con 0x559df40714d0 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f300f1bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559df439f930 con 0x559df40692f0 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f30067ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4439e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:43972/0 (socket says 172.21.15.29:43972) 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f30067ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4439e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56400/0 (socket says 172.21.15.29:56400) 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 -- 172.21.15.29:0/587061557 learned_addr learned my addr 172.21.15.29:0/587061557 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f2ff6ffd640 1 -- 172.21.15.29:0/4286990414 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2ff0017070 con 0x7f3000104610 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 -- 172.21.15.29:0/587061557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 msgr2=0x559df443b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df443b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 -- 172.21.15.29:0/587061557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 msgr2=0x559df4439e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4439e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 -- 172.21.15.29:0/587061557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df4361530 con 0x559df40692f0 2024-09-21T01:59:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.732+0000 7f3005fe9640 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc031500 tx=0x7f2ffc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742d55e748120ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f2febfff640 1 -- 172.21.15.29:0/587061557 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc03e030 con 0x559df40692f0 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f2febfff640 1 -- 172.21.15.29:0/587061557 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc032d20 con 0x559df40692f0 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f2febfff640 1 -- 172.21.15.29:0/587061557 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc035040 con 0x559df40692f0 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f2ff6ffd640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 0x7f2fec0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f2ff6ffd640 1 -- 172.21.15.29:0/4286990414 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff00e7160 con 0x7f3000104610 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.733+0000 7f30067ea640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 0x7f2fec0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 -- 172.21.15.29:0/587061557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 msgr2=0x559df443a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc031500 tx=0x7f2ffc000bc0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 -- 172.21.15.29:0/587061557 shutdown_connections 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df443b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df443a4c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 --2- 172.21.15.29:0/587061557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4439e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.734+0000 7f300f1bd740 1 -- 172.21.15.29:0/587061557 >> 172.21.15.29:0/587061557 conn(0x559df4086300 msgr2=0x559df44358e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.735+0000 7f30067ea640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 0x7f2fec0b8bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012640 tx=0x7f2ff80094e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.735+0000 7f300f1bd740 1 -- 172.21.15.29:0/587061557 shutdown_connections 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.735+0000 7f300f1bd740 1 -- 172.21.15.29:0/587061557 wait complete. 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.735+0000 7f300f1bd740 1 -- start start 2024-09-21T01:59:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 Processor -- start 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 0x559df4443590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40714d0 0x559df4443c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df44447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559df42d3ab0 con 0x559df40921b0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559df4341730 con 0x559df40692f0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559df443bad0 con 0x559df40714d0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.736+0000 7f300f1bd740 10 client.0 _ll_register_callbacks cb 0x559df43956b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 0x559df4443590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 0x559df4443590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51860/0 (socket says 172.21.15.29:51860) 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f3005fe9640 1 -- 172.21.15.29:0/2114668046 learned_addr learned my addr 172.21.15.29:0/2114668046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df44447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30057e8640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40714d0 0x559df4443c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 -- 172.21.15.29:0/2114668046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40714d0 msgr2=0x559df4443c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40714d0 0x559df4443c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 -- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 msgr2=0x559df4443590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 0x559df4443590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.737+0000 7f30067ea640 1 -- 172.21.15.29:0/2114668046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df40fd790 con 0x559df40921b0 2024-09-21T01:59:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f30067ea640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df44447c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012ed0 tx=0x7f2ff8004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a0a92e250729d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f2fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802c060 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc032d20 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff800f830 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ff800fa30 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f300f1bd740 10 client.17712 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.738+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559df4176620 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc032d20 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2ff8026030 con 0x559df40921b0 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 10 client.17712 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 10 client.17712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 20 client.17712 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df4050d80 0x559df4451290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 4 client.17712 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559df43ae640 con 0x559df4050d80 2024-09-21T01:59:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.739+0000 7f3005fe9640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df4050d80 0x559df4451290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.740+0000 7f2fe97fa640 10 client.17712 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.740+0000 7f3005fe9640 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df4050d80 0x559df4451290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a230d00e5c96019a server_cookie=468a8652c6c85a4d in_seq=0 out_seq=0 2024-09-21T01:59:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f2fe97fa640 1 -- 172.21.15.29:0/2114668046 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559df43ae640 con 0x559df4050d80 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f2fe97fa640 10 client.17712 handle_command_reply: tid=0 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 client.17712 shutdown 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 msgr2=0x559df44447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df44447c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012ed0 tx=0x7f2ff8004b00 comp rx=0 tx=0).stop 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 shutdown_connections 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df4050d80 0x559df4451290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df44447c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40714d0 0x559df4443c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 --2- 172.21.15.29:0/2114668046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40692f0 0x559df4443590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 >> 172.21.15.29:0/2114668046 conn(0x559df4086300 msgr2=0x559df4433ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f2fe97fa640 0 client.17712 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 shutdown_connections 2024-09-21T01:59:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 1 -- 172.21.15.29:0/2114668046 wait complete. 2024-09-21T01:59:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.741+0000 7f300f1bd740 20 client.17712 trim_cache size 0 max 16384 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 Processor -- start 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 -- start start 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4442060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40921b0 0x559df4439df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a8cd0 con 0x559df40692f0 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30001aa8d0 con 0x559df40714d0 2024-09-21T01:59:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f300f1bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a93a0 con 0x559df40921b0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f30057e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4442060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56422/0 (socket says 172.21.15.29:56422) 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.763+0000 7f3005fe9640 1 -- 172.21.15.29:0/1550828310 learned_addr learned my addr 172.21.15.29:0/1550828310 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f30067ea640 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40921b0 0x559df4439df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 -- 172.21.15.29:0/1550828310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40921b0 msgr2=0x559df4439df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40921b0 0x559df4439df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 -- 172.21.15.29:0/1550828310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 msgr2=0x559df4442060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4442060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 -- 172.21.15.29:0/1550828310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df4443fc0 con 0x559df40692f0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f30057e8640 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4442060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f3005fe9640 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009a20 tx=0x7f2ffc02ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf4179ed0a90797 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f2feb7fe640 1 -- 172.21.15.29:0/1550828310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc050020 con 0x559df40692f0 2024-09-21T01:59:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f2feb7fe640 1 -- 172.21.15.29:0/1550828310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc005040 con 0x559df40692f0 2024-09-21T01:59:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.764+0000 7f2feb7fe640 1 -- 172.21.15.29:0/1550828310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc03e040 con 0x559df40692f0 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.765+0000 7f300f1bd740 1 -- 172.21.15.29:0/1550828310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 msgr2=0x559df4441b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009a20 tx=0x7f2ffc02ab90 comp rx=0 tx=0).stop 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 -- 172.21.15.29:0/1550828310 shutdown_connections 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40921b0 0x559df4439df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4442060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 --2- 172.21.15.29:0/1550828310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40692f0 0x559df4441b20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 -- 172.21.15.29:0/1550828310 >> 172.21.15.29:0/1550828310 conn(0x559df4086300 msgr2=0x559df44305c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 -- 172.21.15.29:0/1550828310 shutdown_connections 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 -- 172.21.15.29:0/1550828310 wait complete. 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 -- start start 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.766+0000 7f300f1bd740 1 Processor -- start 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4448b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30001aa140 con 0x559df40921b0 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a9a70 con 0x559df40714d0 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f150 con 0x559df40692f0 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f300f1bd740 10 client.0 _ll_register_callbacks cb 0x7f2ff8009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f30067ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44014/0 (socket says 172.21.15.29:44014) 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 -- 172.21.15.29:0/576071856 learned_addr learned my addr 172.21.15.29:0/576071856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f30067ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56424/0 (socket says 172.21.15.29:56424) 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f30057e8640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4448b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 -- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 msgr2=0x559df4448b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4448b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 -- 172.21.15.29:0/576071856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 msgr2=0x559df4449690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f3005fe9640 1 -- 172.21.15.29:0/576071856 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df4449df0 con 0x559df40692f0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.767+0000 7f30067ea640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f3005fe9640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033000 tx=0x7f2ffc032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64a8c2b774a22f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc059060 con 0x559df40692f0 2024-09-21T01:59:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc005040 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2ffc005470 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc03e070 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f300f1bd740 10 client.27493 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559df444b0e0 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc005040 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2ffc04c770 con 0x559df40692f0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.768+0000 7f300f1bd740 10 client.27493 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f300f1bd740 10 client.27493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f300f1bd740 20 client.27493 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df40261c0 0x559df44bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f300f1bd740 4 client.27493 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559df43aa0f0 con 0x559df40261c0 2024-09-21T01:59:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f30057e8640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df40261c0 0x559df44bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f2fe97fa640 10 client.27493 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.769+0000 7f30057e8640 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df40261c0 0x559df44bdc60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74263c5da3f30ae9 server_cookie=c96141a77ab90ff9 in_seq=0 out_seq=0 2024-09-21T01:59:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f2fe97fa640 1 -- 172.21.15.29:0/576071856 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559df43aa0f0 con 0x559df40261c0 2024-09-21T01:59:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f2fe97fa640 10 client.27493 handle_command_reply: tid=0 2024-09-21T01:59:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 client.27493 shutdown 2024-09-21T01:59:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 msgr2=0x559df4448460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033000 tx=0x7f2ffc032d30 comp rx=0 tx=0).stop 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 shutdown_connections 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559df40261c0 0x559df44bdc60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559df40921b0 0x559df4449690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559df40714d0 0x559df4448b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f2fe97fa640 0 client.27493 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 --2- 172.21.15.29:0/576071856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559df40692f0 0x559df4448460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 >> 172.21.15.29:0/576071856 conn(0x559df4086300 msgr2=0x559df44345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 shutdown_connections 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.777+0000 7f300f1bd740 1 -- 172.21.15.29:0/576071856 wait complete. 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300f1bd740 20 client.27493 trim_cache size 0 max 16384 2024-09-21T01:59:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 msgr2=0x7f2fec0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 0x7f2fec0b8bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012640 tx=0x7f2ff80094e0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 msgr2=0x7f30001a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c970 tx=0x7f2ff000ce30 comp rx=0 tx=0).stop 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 shutdown_connections 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2fec0b66b0 0x7f2fec0b8bc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30001071d0 0x7f30001a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3000104610 0x7f30001a0c60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 --2- 172.21.15.29:0/4286990414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f300006b710 0x7f30001a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 >> 172.21.15.29:0/4286990414 conn(0x7f30000fdee0 msgr2=0x7f30001055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 shutdown_connections 2024-09-21T01:59:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:15.778+0000 7f300824d640 1 -- 172.21.15.29:0/4286990414 wait complete. 2024-09-21T01:59:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:15 smithi190 ceph-mon[34368]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-21T01:59:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:15 smithi066 ceph-mon[35001]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-21T01:59:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:15 smithi029 ceph-mon[31228]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-21T01:59:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 640.0 s, 2091.2 tps, lat 48.267 ms stddev 51.793 2024-09-21T01:59:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:17 smithi190 ceph-mon[34368]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-09-21T01:59:18.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:17 smithi066 ceph-mon[35001]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-09-21T01:59:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:17 smithi029 ceph-mon[31228]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-09-21T01:59:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:19 smithi190 ceph-mon[34368]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 569 op/s 2024-09-21T01:59:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:19 smithi066 ceph-mon[35001]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 569 op/s 2024-09-21T01:59:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:19 smithi029 ceph-mon[31228]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 569 op/s 2024-09-21T01:59:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:21 smithi190 ceph-mon[34368]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 796 op/s 2024-09-21T01:59:22.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:21 smithi066 ceph-mon[35001]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 796 op/s 2024-09-21T01:59:22.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:21 smithi029 ceph-mon[31228]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 796 op/s 2024-09-21T01:59:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 645.0 s, 2095.8 tps, lat 47.355 ms stddev 58.256 2024-09-21T01:59:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:23 smithi190 ceph-mon[34368]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-09-21T01:59:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:23 smithi066 ceph-mon[35001]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-09-21T01:59:24.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:23 smithi029 ceph-mon[31228]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-09-21T01:59:25.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:59:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:26.082 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:25 smithi029 ceph-mon[31228]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-09-21T01:59:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3630876102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 msgr2=0x7f82ac11ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3630876102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac11ad40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009ca0 tx=0x7f82a402a8d0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3630876102 shutdown_connections 2024-09-21T01:59:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3630876102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac11ad40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3630876102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3630876102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3630876102 >> 172.21.15.29:0/3630876102 conn(0x7f82ac070ec0 msgr2=0x7f82ac0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.224+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3630876102 shutdown_connections 2024-09-21T01:59:26.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.225+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3630876102 wait complete. 2024-09-21T01:59:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.226+0000 7f82b1a8c640 1 Processor -- start 2024-09-21T01:59:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:25 smithi190 ceph-mon[34368]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-09-21T01:59:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 -- start start 2024-09-21T01:59:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac1aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac11e560 con 0x7f82ac10ad10 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac11e3e0 con 0x7f82ac10a3c0 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82b1a8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac11e6e0 con 0x7f82ac118900 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44526/0 (socket says 172.21.15.29:44526) 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38632/0 (socket says 172.21.15.29:38632) 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82ab7fe640 1 -- 172.21.15.29:0/3463287851 learned_addr learned my addr 172.21.15.29:0/3463287851 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.227+0000 7f82abfff640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac1aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 -- 172.21.15.29:0/3463287851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 msgr2=0x7f82ac1aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac1aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 -- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 msgr2=0x7f82ac1a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 -- 172.21.15.29:0/3463287851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1aaf00 con 0x7f82ac10a3c0 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82aaffd640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9f0 tx=0x7f829c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82a8ff9640 1 -- 172.21.15.29:0/3463287851 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c00ccc0 con 0x7f82ac10a3c0 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82a8ff9640 1 -- 172.21.15.29:0/3463287851 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0108e0 con 0x7f82ac10a3c0 2024-09-21T01:59:26.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82a8ff9640 1 -- 172.21.15.29:0/3463287851 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c018880 con 0x7f82ac10a3c0 2024-09-21T01:59:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1ab1f0 con 0x7f82ac10a3c0 2024-09-21T01:59:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.228+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1b1950 con 0x7f82ac10a3c0 2024-09-21T01:59:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.233+0000 7f82a8ff9640 1 -- 172.21.15.29:0/3463287851 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f829c018a20 con 0x7f82ac10a3c0 2024-09-21T01:59:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.233+0000 7f82b8000740 1 Processor -- start 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.233+0000 7f82b8000740 1 -- start start 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f71cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f71cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f71cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38640/0 (socket says 172.21.15.29:38640) 2024-09-21T01:59:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82aaffd640 1 -- 172.21.15.29:0/4003179067 learned_addr learned my addr 172.21.15.29:0/4003179067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564a93ee7e50 con 0x564a93ba9390 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564a93ed7280 con 0x564a93bca070 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a93edb090 con 0x564a93ba11b0 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.234+0000 7f82abfff640 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 -- 172.21.15.29:0/4003179067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 msgr2=0x564a93f71270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f71270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 -- 172.21.15.29:0/4003179067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 msgr2=0x564a93f71cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f71cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 -- 172.21.15.29:0/4003179067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a93c35650 con 0x564a93ba11b0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f717b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f829c0e50f0 tx=0x7f829c0edb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1584f7e143486369 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82a8ff9640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 0x7f82940b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f8291ffb640 1 -- 172.21.15.29:0/4003179067 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0fa070 con 0x564a93ba11b0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f8291ffb640 1 -- 172.21.15.29:0/4003179067 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0ad440 con 0x564a93ba11b0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f8291ffb640 1 -- 172.21.15.29:0/4003179067 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0ad0a0 con 0x564a93ba11b0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.235+0000 7f82abfff640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 0x7f82940b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82a8ff9640 1 -- 172.21.15.29:0/3463287851 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f829c0e76a0 con 0x7f82ac10a3c0 2024-09-21T01:59:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 msgr2=0x564a93f717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82abfff640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 0x7f82940b8b20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005e30 tx=0x7f82a4032fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f717b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f829c0e50f0 tx=0x7f829c0edb80 comp rx=0 tx=0).stop 2024-09-21T01:59:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 shutdown_connections 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f71cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f717b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 --2- 172.21.15.29:0/4003179067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f71270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.236+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 >> 172.21.15.29:0/4003179067 conn(0x564a93bbe460 msgr2=0x564a93f6f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.237+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 shutdown_connections 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.237+0000 7f82b8000740 1 -- 172.21.15.29:0/4003179067 wait complete. 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.237+0000 7f82b8000740 1 -- start start 2024-09-21T01:59:26.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.238+0000 7f82b8000740 1 Processor -- start 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f85570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564a93d38700 con 0x564a93ba9390 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564a93e72110 con 0x564a93bca070 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a93f88180 con 0x564a93ba11b0 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82b8000740 10 client.0 _ll_register_callbacks cb 0x564a93e8b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44552/0 (socket says 172.21.15.29:44552) 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 -- 172.21.15.29:0/1422588546 learned_addr learned my addr 172.21.15.29:0/1422588546 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 -- 172.21.15.29:0/1422588546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 msgr2=0x564a93f85030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f85030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 -- 172.21.15.29:0/1422588546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 msgr2=0x564a93f85570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f85570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.239+0000 7f82abfff640 1 -- 172.21.15.29:0/1422588546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a93e9fc40 con 0x564a93bca070 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f82abfff640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82a4035000 tx=0x7f82a402a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee6fa6540e7ca32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f828affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004250 con 0x564a93bca070 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f82a40043f0 con 0x564a93bca070 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c0ad440 con 0x564a93bca070 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f82a40045f0 con 0x564a93bca070 2024-09-21T01:59:26.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f82b8000740 10 client.27548 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564a93f93e50 con 0x564a93bca070 2024-09-21T01:59:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.240+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0ad440 con 0x564a93bca070 2024-09-21T01:59:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f82a405d020 con 0x564a93bca070 2024-09-21T01:59:26.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 10 client.27548 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 10 client.27548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 20 client.27548 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b889f0 0x564a93f97890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 4 client.27548 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a93ee9da0 con 0x564a93b889f0 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.242+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b889f0 0x564a93f97890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.243+0000 7f828affd640 10 client.27548 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:26.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.243+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b889f0 0x564a93f97890 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6abc55a93c3cc7 server_cookie=8609fb72da909ba1 in_seq=0 out_seq=0 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f828affd640 1 -- 172.21.15.29:0/1422588546 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a93ee9da0 con 0x564a93b889f0 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f828affd640 10 client.27548 handle_command_reply: tid=0 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 client.27548 shutdown 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 msgr2=0x564a93f85ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82a4035000 tx=0x7f82a402a8d0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 shutdown_connections 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b889f0 0x564a93f97890 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f85ab0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f85570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 --2- 172.21.15.29:0/1422588546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f85030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f828affd640 0 client.27548 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 >> 172.21.15.29:0/1422588546 conn(0x564a93bbe460 msgr2=0x564a93f6d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 shutdown_connections 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 1 -- 172.21.15.29:0/1422588546 wait complete. 2024-09-21T01:59:26.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.244+0000 7f82b8000740 20 client.27548 trim_cache size 0 max 16384 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.268+0000 7f82b8000740 1 Processor -- start 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.268+0000 7f82b8000740 1 -- start start 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b5a90 con 0x564a93ba9390 2024-09-21T01:59:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b7690 con 0x564a93bca070 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82b8000740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b6160 con 0x564a93ba11b0 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38664/0 (socket says 172.21.15.29:38664) 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82aaffd640 1 -- 172.21.15.29:0/2448310042 learned_addr learned my addr 172.21.15.29:0/2448310042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82abfff640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.269+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 -- 172.21.15.29:0/2448310042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 msgr2=0x564a93f69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f69df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 -- 172.21.15.29:0/2448310042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 msgr2=0x564a93f729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 -- 172.21.15.29:0/2448310042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a93f83f90 con 0x564a93ba9390 2024-09-21T01:59:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82abfff640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009f80 tx=0x7f82a0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a566d76aeff8bcc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.270+0000 7f82917fa640 1 -- 172.21.15.29:0/2448310042 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0014650 con 0x564a93ba9390 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82917fa640 1 -- 172.21.15.29:0/2448310042 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0014ca0 con 0x564a93ba9390 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82917fa640 1 -- 172.21.15.29:0/2448310042 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a001dd10 con 0x564a93ba9390 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 -- 172.21.15.29:0/2448310042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 msgr2=0x564a93f6f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009f80 tx=0x7f82a0011750 comp rx=0 tx=0).stop 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 -- 172.21.15.29:0/2448310042 shutdown_connections 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93bca070 0x564a93f729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba9390 0x564a93f6f060 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 --2- 172.21.15.29:0/2448310042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93ba11b0 0x564a93f69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.271+0000 7f82b8000740 1 -- 172.21.15.29:0/2448310042 >> 172.21.15.29:0/2448310042 conn(0x564a93bbe460 msgr2=0x564a93f722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.272+0000 7f82b8000740 1 -- 172.21.15.29:0/2448310042 shutdown_connections 2024-09-21T01:59:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.272+0000 7f82b8000740 1 -- 172.21.15.29:0/2448310042 wait complete. 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.272+0000 7f82b8000740 1 -- start start 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 Processor -- start 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 0x564a93f88f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93bca070 0x564a93f89480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b6f00 con 0x564a93ba11b0 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b6830 con 0x564a93ba9390 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.273+0000 7f82b8000740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b3af0 con 0x564a93bca070 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82b8000740 10 client.0 _ll_register_callbacks cb 0x7f82a40318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 0x564a93f88f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93bca070 0x564a93f89480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38670/0 (socket says 172.21.15.29:38670) 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 -- 172.21.15.29:0/2306188620 learned_addr learned my addr 172.21.15.29:0/2306188620 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 -- 172.21.15.29:0/2306188620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93bca070 msgr2=0x564a93f89480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93bca070 0x564a93f89480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 -- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 msgr2=0x564a93f88f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 0x564a93f88f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 -- 172.21.15.29:0/2306188620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a93f89a10 con 0x564a93ba11b0 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 0x564a93f88f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82ab7fe640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f829c0ad8c0 tx=0x7f829c0ad280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd0093d12dd8483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f828affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0bd400 con 0x564a93ba11b0 2024-09-21T01:59:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f829c0e51d0 con 0x564a93ba11b0 2024-09-21T01:59:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.274+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a0014ca0 con 0x564a93ba11b0 2024-09-21T01:59:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f829c0bd5f0 con 0x564a93ba11b0 2024-09-21T01:59:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 10 client.17730 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564a94001fe0 con 0x564a93ba11b0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0014ca0 con 0x564a93ba11b0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f829c0a9ba0 con 0x564a93ba11b0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 10 client.17730 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 20 client.17730 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b5dda0 0x564a940042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 4 client.17730 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a93ee5850 con 0x564a93b5dda0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.275+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b5dda0 0x564a940042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.276+0000 7f828affd640 10 client.17730 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.276+0000 7f82aaffd640 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b5dda0 0x564a940042d0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609e2f5d5ff7f33b server_cookie=7a0b42e20a7cc0c2 in_seq=0 out_seq=0 2024-09-21T01:59:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.276+0000 7f828affd640 1 -- 172.21.15.29:0/2306188620 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x564a93ee5850 con 0x564a93b5dda0 2024-09-21T01:59:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.276+0000 7f828affd640 10 client.17730 handle_command_reply: tid=0 2024-09-21T01:59:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.276+0000 7f82b8000740 1 client.17730 shutdown 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.277+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 msgr2=0x564a93f8da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.277+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f829c0ad8c0 tx=0x7f829c0ad280 comp rx=0 tx=0).stop 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.277+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 shutdown_connections 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.277+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x564a93b5dda0 0x564a940042d0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a93bca070 0x564a93f89480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564a93ba9390 0x564a93f88f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 --2- 172.21.15.29:0/2306188620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a93ba11b0 0x564a93f8da90 secure :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f829c0ad8c0 tx=0x7f829c0ad280 comp rx=0 tx=0).stop 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 >> 172.21.15.29:0/2306188620 conn(0x564a93bbe460 msgr2=0x564a93f72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f828affd640 0 client.17730 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 shutdown_connections 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 1 -- 172.21.15.29:0/2306188620 wait complete. 2024-09-21T01:59:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.278+0000 7f82b8000740 20 client.17730 trim_cache size 0 max 16384 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:59:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:59:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:59:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:59:26.290 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 msgr2=0x7f82940b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 0x7f82940b8b20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005e30 tx=0x7f82a4032fb0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 msgr2=0x7f82ac1a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9f0 tx=0x7f829c00edf0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 shutdown_connections 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f82940b6610 0x7f82940b8b20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82ac118900 0x7f82ac1aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac10ad10 0x7f82ac1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 --2- 172.21.15.29:0/3463287851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ac10a3c0 0x7f82ac1a4f90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 >> 172.21.15.29:0/3463287851 conn(0x7f82ac070ec0 msgr2=0x7f82ac118e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:26.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 shutdown_connections 2024-09-21T01:59:26.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:26.279+0000 7f82b1a8c640 1 -- 172.21.15.29:0/3463287851 wait complete. 2024-09-21T01:59:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:25 smithi066 ceph-mon[35001]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-09-21T01:59:27.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T01:59:27.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:59:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 650.0 s, 1923.6 tps, lat 52.102 ms stddev 58.574 2024-09-21T01:59:28.023 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:28.184 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:27 smithi029 ceph-mon[31228]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 MiB/s wr, 785 op/s 2024-09-21T01:59:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:27 smithi190 ceph-mon[34368]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 MiB/s wr, 785 op/s 2024-09-21T01:59:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:27 smithi066 ceph-mon[35001]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 MiB/s wr, 785 op/s 2024-09-21T01:59:28.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.620+0000 7f12e47b2640 1 -- 172.21.15.29:0/1269307456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 msgr2=0x7f12dc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.621+0000 7f12e47b2640 1 --2- 172.21.15.29:0/1269307456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc1087d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009900 tx=0x7f12cc02a410 comp rx=0 tx=0).stop 2024-09-21T01:59:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 -- 172.21.15.29:0/1269307456 shutdown_connections 2024-09-21T01:59:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 --2- 172.21.15.29:0/1269307456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 0x7f12dc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 --2- 172.21.15.29:0/1269307456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc1087d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 --2- 172.21.15.29:0/1269307456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 0x7f12dc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 -- 172.21.15.29:0/1269307456 >> 172.21.15.29:0/1269307456 conn(0x7f12dc079740 msgr2=0x7f12dc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 -- 172.21.15.29:0/1269307456 shutdown_connections 2024-09-21T01:59:28.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.623+0000 7f12e47b2640 1 -- 172.21.15.29:0/1269307456 wait complete. 2024-09-21T01:59:28.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 Processor -- start 2024-09-21T01:59:28.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 -- start start 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 0x7f12dc19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 0x7f12dc19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112980 con 0x7f12dc1083c0 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112800 con 0x7f12dc108d10 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e47b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112b00 con 0x7f12dc102a80 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e1d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e1d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38688/0 (socket says 172.21.15.29:38688) 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.624+0000 7f12e1d4d640 1 -- 172.21.15.29:0/977083718 learned_addr learned my addr 172.21.15.29:0/977083718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e2d4f640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 0x7f12dc19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e254e640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 0x7f12dc19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 -- 172.21.15.29:0/977083718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 msgr2=0x7f12dc19b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 0x7f12dc19b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 -- 172.21.15.29:0/977083718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 msgr2=0x7f12dc19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 0x7f12dc19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 -- 172.21.15.29:0/977083718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc1a63e0 con 0x7f12dc1083c0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e1d4d640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009850 tx=0x7f12cc0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc03f070 con 0x7f12dc1083c0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.625+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc1a6670 con 0x7f12dc1083c0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.626+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12cc004460 con 0x7f12dc1083c0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.626+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1a6ba0 con 0x7f12dc1083c0 2024-09-21T01:59:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.626+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc035030 con 0x7f12dc1083c0 2024-09-21T01:59:28.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.627+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc06bc90 con 0x7f12dc1083c0 2024-09-21T01:59:28.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.629+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f12cc042070 con 0x7f12dc1083c0 2024-09-21T01:59:28.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.630+0000 7f12d37fe640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 0x7f12c80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.630+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12cc107c30 con 0x7f12dc1083c0 2024-09-21T01:59:28.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.630+0000 7f12e254e640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 0x7f12c80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.631+0000 7f12e254e640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 0x7f12c80b8aa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12dc06c3e0 tx=0x7f12d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:28.635+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12cc049030 con 0x7f12dc1083c0 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2184491247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 msgr2=0x7fbda410a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2184491247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda410a810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009be0 tx=0x7fbd9c02a6c0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2184491247 shutdown_connections 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2184491247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda410a810 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2184491247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda41056e0 0x7fbda4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2184491247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda4102c20 0x7fbda41050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2184491247 >> 172.21.15.29:0/2184491247 conn(0x7fbda40fd690 msgr2=0x7fbda40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2184491247 shutdown_connections 2024-09-21T01:59:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.923+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2184491247 wait complete. 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.924+0000 7fbdaa52b640 1 Processor -- start 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.924+0000 7fbdaa52b640 1 -- start start 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.924+0000 7fbdaa52b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.924+0000 7fbdaa52b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda41056e0 0x7fbda4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbdaa52b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbdaa52b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbda410d120 con 0x7fbda41056e0 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbdaa52b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbda410cfa0 con 0x7fbda4108300 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbdaa52b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbda410d2a0 con 0x7fbda4102c20 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda8ac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37352/0 (socket says 172.21.15.29:37352) 2024-09-21T01:59:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 -- 172.21.15.29:0/2169934021 learned_addr learned my addr 172.21.15.29:0/2169934021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda37fe640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda41056e0 0x7fbda4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 -- 172.21.15.29:0/2169934021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 msgr2=0x7fbda4198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda4198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 -- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda41056e0 msgr2=0x7fbda4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda41056e0 0x7fbda4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 -- 172.21.15.29:0/2169934021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda4199720 con 0x7fbda4102c20 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.925+0000 7fbda3fff640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e9c0 tx=0x7fbd9400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.926+0000 7fbda17fa640 1 -- 172.21.15.29:0/2169934021 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400cde0 con 0x7fbda4102c20 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.926+0000 7fbda17fa640 1 -- 172.21.15.29:0/2169934021 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd940035f0 con 0x7fbda4102c20 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.926+0000 7fbda17fa640 1 -- 172.21.15.29:0/2169934021 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940216a0 con 0x7fbda4102c20 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.926+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda4199a10 con 0x7fbda4102c20 2024-09-21T01:59:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.926+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda406cfc0 con 0x7fbda4102c20 2024-09-21T01:59:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.928+0000 7fbdb149a740 1 Processor -- start 2024-09-21T01:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 -- start start 2024-09-21T01:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 0x556ef366d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef366dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ef35d76f0 con 0x556ef329c4d0 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ef35c6b20 con 0x556ef32942f0 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbdb149a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556ef35ca930 con 0x556ef32bd1b0 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda8ac8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 0x556ef366d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38726/0 (socket says 172.21.15.29:38726) 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda37fe640 1 -- 172.21.15.29:0/1649796639 learned_addr learned my addr 172.21.15.29:0/1649796639 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda8ac8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 0x556ef366d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37366/0 (socket says 172.21.15.29:37366) 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.929+0000 7fbda17fa640 1 -- 172.21.15.29:0/2169934021 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbd94004000 con 0x7fbda4102c20 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 -- 172.21.15.29:0/1649796639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 msgr2=0x556ef366d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 0x556ef366d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 -- 172.21.15.29:0/1649796639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 msgr2=0x556ef366dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef366dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 -- 172.21.15.29:0/1649796639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef358c530 con 0x556ef329c4d0 2024-09-21T01:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbda37fe640 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbd98004570 tx=0x7fbd98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ac5add1e51f884 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbd8a7fc640 1 -- 172.21.15.29:0/1649796639 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800be00 con 0x556ef329c4d0 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbd8a7fc640 1 -- 172.21.15.29:0/1649796639 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd98005020 con 0x556ef329c4d0 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.930+0000 7fbd8a7fc640 1 -- 172.21.15.29:0/1649796639 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98004c00 con 0x556ef329c4d0 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbda17fa640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 0x7fbd8c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbda8ac8640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 0x7fbd8c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbda17fa640 1 -- 172.21.15.29:0/2169934021 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbd940f8610 con 0x7fbda4102c20 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 -- 172.21.15.29:0/1649796639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 msgr2=0x556ef366e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbd98004570 tx=0x7fbd98005f90 comp rx=0 tx=0).stop 2024-09-21T01:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 -- 172.21.15.29:0/1649796639 shutdown_connections 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef366e820 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef366dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 --2- 172.21.15.29:0/1649796639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32bd1b0 0x556ef366d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 -- 172.21.15.29:0/1649796639 >> 172.21.15.29:0/1649796639 conn(0x556ef32b1300 msgr2=0x556ef36690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 -- 172.21.15.29:0/1649796639 shutdown_connections 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.931+0000 7fbdb149a740 1 -- 172.21.15.29:0/1649796639 wait complete. 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.932+0000 7fbda8ac8640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 0x7fbd8c0b8a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c006010 tx=0x7fbd9c0093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.932+0000 7fbdb149a740 1 -- start start 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.932+0000 7fbdb149a740 1 Processor -- start 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 0x556ef36774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3678040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ef34feab0 con 0x556ef32bd1b0 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ef356c730 con 0x556ef32942f0 2024-09-21T01:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556ef366f340 con 0x556ef329c4d0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbdb149a740 10 client.0 _ll_register_callbacks cb 0x556ef35c06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 0x556ef36774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44610/0 (socket says 172.21.15.29:44610) 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 0x556ef36774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37376/0 (socket says 172.21.15.29:37376) 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.933+0000 7fbda3fff640 1 -- 172.21.15.29:0/3556704030 learned_addr learned my addr 172.21.15.29:0/3556704030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda8ac8640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3678040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 -- 172.21.15.29:0/3556704030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 msgr2=0x556ef36774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 0x556ef36774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 -- 172.21.15.29:0/3556704030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 msgr2=0x556ef3678040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3678040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 -- 172.21.15.29:0/3556704030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef3328790 con 0x556ef32942f0 2024-09-21T01:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda8ac8640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3678040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:59:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd940bad80 tx=0x7fbd94003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7019f863b1dbfe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbd837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940bfce0 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.934+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbd940ba100 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.935+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd98005020 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.935+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbd940be2c0 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.935+0000 7fbdb149a740 10 client.27569 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.935+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556ef33a1620 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.935+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd98005020 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.936+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbd940bd480 con 0x556ef32942f0 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.936+0000 7fbdb149a740 10 client.27569 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.936+0000 7fbdb149a740 10 client.27569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.936+0000 7fbdb149a740 20 client.27569 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.936+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef327bd80 0x556ef3684680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.937+0000 7fbdb149a740 4 client.27569 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.937+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ef35d9640 con 0x556ef327bd80 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.937+0000 7fbda37fe640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef327bd80 0x556ef3684680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.937+0000 7fbd837fe640 10 client.27569 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.937+0000 7fbda37fe640 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef327bd80 0x556ef3684680 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2db1d4073518cdc server_cookie=474b32581680e52a in_seq=0 out_seq=0 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.938+0000 7fbd837fe640 1 -- 172.21.15.29:0/3556704030 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ef35d9640 con 0x556ef327bd80 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.938+0000 7fbd837fe640 10 client.27569 handle_command_reply: tid=0 2024-09-21T01:59:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.938+0000 7fbdb149a740 1 client.27569 shutdown 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 msgr2=0x556ef3676e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd940bad80 tx=0x7fbd94003cd0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 shutdown_connections 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef327bd80 0x556ef3684680 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3678040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef329c4d0 0x556ef36774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3556704030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32942f0 0x556ef3676e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 >> 172.21.15.29:0/3556704030 conn(0x556ef32b1300 msgr2=0x556ef365eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbd837fe640 0 client.27569 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 shutdown_connections 2024-09-21T01:59:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 1 -- 172.21.15.29:0/3556704030 wait complete. 2024-09-21T01:59:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.939+0000 7fbdb149a740 20 client.27569 trim_cache size 0 max 16384 2024-09-21T01:59:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.958+0000 7fbdb149a740 1 Processor -- start 2024-09-21T01:59:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.958+0000 7fbdb149a740 1 -- start start 2024-09-21T01:59:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef366a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3672c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a05c0 con 0x556ef32bd1b0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a21c0 con 0x556ef329c4d0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbdb149a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a0c90 con 0x556ef32942f0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda8ac8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3672c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44612/0 (socket says 172.21.15.29:44612) 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 -- 172.21.15.29:0/587781660 learned_addr learned my addr 172.21.15.29:0/587781660 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda3fff640 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef366a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 -- 172.21.15.29:0/587781660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 msgr2=0x556ef366a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef366a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 -- 172.21.15.29:0/587781660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 msgr2=0x556ef3672c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3672c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda37fe640 1 -- 172.21.15.29:0/587781660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef3680e50 con 0x556ef329c4d0 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.959+0000 7fbda3fff640 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef366a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.960+0000 7fbda37fe640 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800d010 tx=0x7fbd980041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6f5a52bf6c15f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.960+0000 7fbd89ffb640 1 -- 172.21.15.29:0/587781660 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802d070 con 0x556ef329c4d0 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.960+0000 7fbd89ffb640 1 -- 172.21.15.29:0/587781660 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd98016920 con 0x556ef329c4d0 2024-09-21T01:59:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.960+0000 7fbd89ffb640 1 -- 172.21.15.29:0/587781660 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98020420 con 0x556ef329c4d0 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- 172.21.15.29:0/587781660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 msgr2=0x556ef365b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800d010 tx=0x7fbd980041b0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- 172.21.15.29:0/587781660 shutdown_connections 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef32bd1b0 0x556ef3672c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef329c4d0 0x556ef365b5c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 --2- 172.21.15.29:0/587781660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef366a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- 172.21.15.29:0/587781660 >> 172.21.15.29:0/587781660 conn(0x556ef32b1300 msgr2=0x556ef366e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- 172.21.15.29:0/587781660 shutdown_connections 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- 172.21.15.29:0/587781660 wait complete. 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.961+0000 7fbdb149a740 1 -- start start 2024-09-21T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 Processor -- start 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef367db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32bd1b0 0x556ef367e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a1a30 con 0x556ef329c4d0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbda41a1360 con 0x556ef32bd1b0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4073c70 con 0x556ef32942f0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef367db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37386/0 (socket says 172.21.15.29:37386) 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbda3fff640 1 -- 172.21.15.29:0/3783355705 learned_addr learned my addr 172.21.15.29:0/3783355705 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbdb149a740 10 client.0 _ll_register_callbacks cb 0x7fbd940be100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.962+0000 7fbda8ac8640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32bd1b0 0x556ef367e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 -- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32bd1b0 msgr2=0x556ef367e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32bd1b0 0x556ef367e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 -- 172.21.15.29:0/3783355705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 msgr2=0x556ef367db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef367db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 -- 172.21.15.29:0/3783355705 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef36ed060 con 0x556ef32942f0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda37fe640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef367db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbda3fff640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd940ba7c0 tx=0x7fbd940bae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7f728808745c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbd837fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940be580 con 0x556ef32942f0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbd940e4020 con 0x556ef32942f0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.963+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd98016920 con 0x556ef32942f0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbd940beeb0 con 0x556ef32942f0 2024-09-21T01:59:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 10 client.27535 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556ef36ee340 con 0x556ef32942f0 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd940df020 con 0x556ef32942f0 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbd940bfb70 con 0x556ef32942f0 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 10 client.27535 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 10 client.27535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 20 client.27535 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef32511c0 0x556ef36f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 4 client.27535 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:59:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.964+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbd98016920 con 0x556ef32511c0 2024-09-21T01:59:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.966+0000 7fbda37fe640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef32511c0 0x556ef36f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.968+0000 7fbd837fe640 10 client.27535 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.968+0000 7fbda37fe640 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef32511c0 0x556ef36f0da0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b526e6cb4572731 server_cookie=e8c6d9ce8af36219 in_seq=0 out_seq=0 2024-09-21T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.977+0000 7fbd837fe640 1 -- 172.21.15.29:0/3783355705 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbd98016920 con 0x556ef32511c0 2024-09-21T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.977+0000 7fbd837fe640 10 client.27535 handle_command_reply: tid=0 2024-09-21T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.977+0000 7fbdb149a740 1 client.27535 shutdown 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.977+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 msgr2=0x556ef367d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.977+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd940ba7c0 tx=0x7fbd940bae70 comp rx=0 tx=0).stop 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 shutdown_connections 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556ef32511c0 0x556ef36f0da0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ef32bd1b0 0x556ef367e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ef329c4d0 0x556ef367db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 --2- 172.21.15.29:0/3783355705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ef32942f0 0x556ef367d650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 >> 172.21.15.29:0/3783355705 conn(0x556ef32b1300 msgr2=0x556ef3669c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbd837fe640 0 client.27535 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 shutdown_connections 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 1 -- 172.21.15.29:0/3783355705 wait complete. 2024-09-21T01:59:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.978+0000 7fbdb149a740 20 client.27535 trim_cache size 0 max 16384 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 msgr2=0x7fbd8c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 0x7fbd8c0b8a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c006010 tx=0x7fbd9c0093e0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 msgr2=0x7fbda4197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e9c0 tx=0x7fbd9400ee80 comp rx=0 tx=0).stop 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 shutdown_connections 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbd8c0b6550 0x7fbd8c0b8a60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4108300 0x7fbda4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbda41056e0 0x7fbda4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 --2- 172.21.15.29:0/2169934021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbda4102c20 0x7fbda4197ce0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e9c0 tx=0x7fbd9400ee80 comp rx=0 tx=0).stop 2024-09-21T01:59:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.980+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 >> 172.21.15.29:0/2169934021 conn(0x7fbda40fd690 msgr2=0x7fbda4103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.982+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 shutdown_connections 2024-09-21T01:59:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:28.982+0000 7fbdaa52b640 1 -- 172.21.15.29:0/2169934021 wait complete. 2024-09-21T01:59:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.030+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12dc02e670 con 0x7f12dc1083c0 2024-09-21T01:59:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.031+0000 7f12d37fe640 1 -- 172.21.15.29:0/977083718 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11977 (secure 0 0 0) 0x7f12cc0ce4d0 con 0x7f12dc1083c0 2024-09-21T01:59:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T01:59:29.033 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":47,"btime":"2024-09-21T01:59:00:830691+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:59:00.246820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T01:59:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 47 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 msgr2=0x7f12c80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 0x7f12c80b8aa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12dc06c3e0 tx=0x7f12d8005f90 comp rx=0 tx=0).stop 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 msgr2=0x7f12dc19b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009850 tx=0x7f12cc0305d0 comp rx=0 tx=0).stop 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 shutdown_connections 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f12c80b6590 0x7f12c80b8aa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:29.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12dc108d10 0x7f12dc19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:29.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12dc1083c0 0x7f12dc19b610 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:29.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 --2- 172.21.15.29:0/977083718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc102a80 0x7f12dc19b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:29.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 >> 172.21.15.29:0/977083718 conn(0x7f12dc079740 msgr2=0x7f12dc1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:29.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.034+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 shutdown_connections 2024-09-21T01:59:29.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:29.035+0000 7f12e47b2640 1 -- 172.21.15.29:0/977083718 wait complete. 2024-09-21T01:59:30.108 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T01:59:30.150 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:29 smithi029 ceph-mon[31228]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 MiB/s wr, 570 op/s 2024-09-21T01:59:30.150 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:29 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/977083718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:59:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:29 smithi190 ceph-mon[34368]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 MiB/s wr, 570 op/s 2024-09-21T01:59:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:29 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/977083718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:59:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:29 smithi066 ceph-mon[35001]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 MiB/s wr, 570 op/s 2024-09-21T01:59:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:29 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/977083718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:59:30.499 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1874930235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae8109e20 msgr2=0x7ffae8110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1874930235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae8109e20 0x7ffae8110650 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009ca0 tx=0x7ffae002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1874930235 shutdown_connections 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1874930235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae8109e20 0x7ffae8110650 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1874930235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae81094d0 0x7ffae81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1874930235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8103b90 0x7ffae8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.308+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1874930235 >> 172.21.15.29:0/1874930235 conn(0x7ffae80ff5e0 msgr2=0x7ffae8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.309+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1874930235 shutdown_connections 2024-09-21T01:59:31.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.309+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1874930235 wait complete. 2024-09-21T01:59:31.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.309+0000 7ffaefc8f640 1 Processor -- start 2024-09-21T01:59:31.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 -- start start 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae8103b90 0x7ffae819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8109e20 0x7ffae819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8113ae0 con 0x7ffae81094d0 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8113960 con 0x7ffae8109e20 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaefc8f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8113c60 con 0x7ffae8103b90 2024-09-21T01:59:31.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaeda2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae8103b90 0x7ffae819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaed22a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaed22a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38770/0 (socket says 172.21.15.29:38770) 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaed22a640 1 -- 172.21.15.29:0/1571764456 learned_addr learned my addr 172.21.15.29:0/1571764456 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.310+0000 7ffaee22c640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8109e20 0x7ffae819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 -- 172.21.15.29:0/1571764456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae8103b90 msgr2=0x7ffae819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae8103b90 0x7ffae819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 -- 172.21.15.29:0/1571764456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8109e20 msgr2=0x7ffae819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8109e20 0x7ffae819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 -- 172.21.15.29:0/1571764456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae81a03c0 con 0x7ffae81094d0 2024-09-21T01:59:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.311+0000 7ffaed22a640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e9f0 tx=0x7ffad800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.312+0000 7ffadeffd640 1 -- 172.21.15.29:0/1571764456 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad800ccc0 con 0x7ffae81094d0 2024-09-21T01:59:31.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.312+0000 7ffadeffd640 1 -- 172.21.15.29:0/1571764456 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad80108e0 con 0x7ffae81094d0 2024-09-21T01:59:31.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.312+0000 7ffadeffd640 1 -- 172.21.15.29:0/1571764456 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8018860 con 0x7ffae81094d0 2024-09-21T01:59:31.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.312+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae81a06b0 con 0x7ffae81094d0 2024-09-21T01:59:31.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.312+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81a6e10 con 0x7ffae81094d0 2024-09-21T01:59:31.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.316+0000 7ffadeffd640 1 -- 172.21.15.29:0/1571764456 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ffad8010430 con 0x7ffae81094d0 2024-09-21T01:59:31.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.316+0000 7ffaf6203740 1 Processor -- start 2024-09-21T01:59:31.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.316+0000 7ffaf6203740 1 -- start start 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d53bea0 con 0x55ff3d1f51b0 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d52b2d0 con 0x55ff3d1fd390 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaf6203740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d52f0e0 con 0x55ff3d21e070 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37402/0 (socket says 172.21.15.29:37402) 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaee22c640 1 -- 172.21.15.29:0/988352528 learned_addr learned my addr 172.21.15.29:0/988352528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:31.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffadeffd640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 0x7ffad40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.317+0000 7ffaed22a640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 -- 172.21.15.29:0/988352528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 msgr2=0x55ff3d5c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 -- 172.21.15.29:0/988352528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 msgr2=0x55ff3d5c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 -- 172.21.15.29:0/988352528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3d289650 con 0x55ff3d1fd390 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaee22c640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaee22c640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 0x7ffad40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffaed22a640 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5c6120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffad80e51d0 tx=0x7ffad80719d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433915302458fb6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffad3fff640 1 -- 172.21.15.29:0/988352528 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80ad160 con 0x55ff3d1fd390 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffad3fff640 1 -- 172.21.15.29:0/988352528 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad80ad320 con 0x55ff3d1fd390 2024-09-21T01:59:31.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.318+0000 7ffadeffd640 1 -- 172.21.15.29:0/1571764456 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffad80e7670 con 0x7ffae81094d0 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.319+0000 7ffad3fff640 1 -- 172.21.15.29:0/988352528 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80acf80 con 0x55ff3d1fd390 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 -- 172.21.15.29:0/988352528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 msgr2=0x55ff3d5c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5c6120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffad80e51d0 tx=0x7ffad80719d0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaee22c640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 0x7ffad40b8b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffae002f500 tx=0x7ffae002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 -- 172.21.15.29:0/988352528 shutdown_connections 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5c6120 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 --2- 172.21.15.29:0/988352528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d21e070 0x55ff3d5c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 -- 172.21.15.29:0/988352528 >> 172.21.15.29:0/988352528 conn(0x55ff3d212460 msgr2=0x55ff3d5c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 -- 172.21.15.29:0/988352528 shutdown_connections 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.320+0000 7ffaf6203740 1 -- 172.21.15.29:0/988352528 wait complete. 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.321+0000 7ffaf6203740 1 -- start start 2024-09-21T01:59:31.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.321+0000 7ffaf6203740 1 Processor -- start 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 0x55ff3d5dd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d38c700 con 0x55ff3d21e070 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d4c6160 con 0x55ff3d1fd390 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ff3d5d3ec0 con 0x55ff3d1f51b0 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaf6203740 10 client.0 _ll_register_callbacks cb 0x55ff3d4df9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 0x55ff3d5dd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 0x55ff3d5dd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38796/0 (socket says 172.21.15.29:38796) 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaee22c640 1 -- 172.21.15.29:0/4166945205 learned_addr learned my addr 172.21.15.29:0/4166945205 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 -- 172.21.15.29:0/4166945205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 msgr2=0x55ff3d5d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 -- 172.21.15.29:0/4166945205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 msgr2=0x55ff3d5dd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 0x55ff3d5dd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.322+0000 7ffaed22a640 1 -- 172.21.15.29:0/4166945205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3d4f3c90 con 0x55ff3d1fd390 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffaed22a640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5d1240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e9c0 tx=0x7ffad80096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e689393f5bf987d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80acf80 con 0x55ff3d1fd390 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffad80717b0 con 0x55ff3d1fd390 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad80ad320 con 0x55ff3d1fd390 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffad80ad160 con 0x55ff3d1fd390 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffaf6203740 10 client.27587 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:31.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ff3d5dfb30 con 0x55ff3d1fd390 2024-09-21T01:59:31.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.323+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad80ad320 con 0x55ff3d1fd390 2024-09-21T01:59:31.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ffad80b6da0 con 0x55ff3d1fd390 2024-09-21T01:59:31.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 10 client.27587 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:31.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 10 client.27587 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 20 client.27587 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1dc9f0 0x55ff3d5e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 4 client.27587 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff3d53ddf0 con 0x55ff3d1dc9f0 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1dc9f0 0x55ff3d5e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffad17fa640 10 client.27587 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:59:31.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.324+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1dc9f0 0x55ff3d5e2e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1825028b6b00793e server_cookie=b0fadbd287030513 in_seq=0 out_seq=0 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffad17fa640 1 -- 172.21.15.29:0/4166945205 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff3d53ddf0 con 0x55ff3d1dc9f0 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffad17fa640 10 client.27587 handle_command_reply: tid=0 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 client.27587 shutdown 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 msgr2=0x55ff3d5d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5d1240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e9c0 tx=0x7ffad80096a0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 shutdown_connections 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1dc9f0 0x55ff3d5e2e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d21e070 0x55ff3d5dd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d1fd390 0x55ff3d5d1240 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4166945205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 >> 172.21.15.29:0/4166945205 conn(0x55ff3d212460 msgr2=0x55ff3d5c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.325+0000 7ffad17fa640 0 client.27587 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.326+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 shutdown_connections 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.326+0000 7ffaf6203740 1 -- 172.21.15.29:0/4166945205 wait complete. 2024-09-21T01:59:31.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.326+0000 7ffaf6203740 20 client.27587 trim_cache size 0 max 16384 2024-09-21T01:59:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.343+0000 7ffaf6203740 1 Processor -- start 2024-09-21T01:59:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.343+0000 7ffaf6203740 1 -- start start 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1fd390 0x55ff3d5dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81af380 con 0x55ff3d1fd390 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81b0f80 con 0x55ff3d21e070 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaf6203740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81afa50 con 0x55ff3d1f51b0 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44674/0 (socket says 172.21.15.29:44674) 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaee22c640 1 -- 172.21.15.29:0/4088457812 learned_addr learned my addr 172.21.15.29:0/4088457812 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:31.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaed22a640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1fd390 0x55ff3d5dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.344+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 -- 172.21.15.29:0/4088457812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 msgr2=0x55ff3d5cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 -- 172.21.15.29:0/4088457812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1fd390 msgr2=0x55ff3d5dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1fd390 0x55ff3d5dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 -- 172.21.15.29:0/4088457812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3d5d1fc0 con 0x55ff3d21e070 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffaee22c640 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffae0031630 tx=0x7ffae0005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df1e35ccf45a66f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffad37fe640 1 -- 172.21.15.29:0/4088457812 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0045070 con 0x55ff3d21e070 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffad37fe640 1 -- 172.21.15.29:0/4088457812 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae0036040 con 0x55ff3d21e070 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.345+0000 7ffad37fe640 1 -- 172.21.15.29:0/4088457812 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae002fc70 con 0x55ff3d21e070 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 -- 172.21.15.29:0/4088457812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 msgr2=0x55ff3d5c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffae0031630 tx=0x7ffae0005d80 comp rx=0 tx=0).stop 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 -- 172.21.15.29:0/4088457812 shutdown_connections 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d5c6de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1fd390 0x55ff3d5dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 --2- 172.21.15.29:0/4088457812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1f51b0 0x55ff3d5cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 -- 172.21.15.29:0/4088457812 >> 172.21.15.29:0/4088457812 conn(0x55ff3d212460 msgr2=0x55ff3d5c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.346+0000 7ffaf6203740 1 -- 172.21.15.29:0/4088457812 shutdown_connections 2024-09-21T01:59:31.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 -- 172.21.15.29:0/4088457812 wait complete. 2024-09-21T01:59:31.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 -- start start 2024-09-21T01:59:31.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 Processor -- start 2024-09-21T01:59:31.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d644dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 0x55ff3d6453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.347+0000 7ffaf6203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaf6203740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81b07f0 con 0x55ff3d1f51b0 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaf6203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81b0120 con 0x55ff3d21e070 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaf6203740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81ad360 con 0x55ff3d1fd390 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaf6203740 10 client.0 _ll_register_callbacks cb 0x7ffae4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaeda2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d644dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:44684/0 (socket says 172.21.15.29:44684) 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 -- 172.21.15.29:0/59750160 learned_addr learned my addr 172.21.15.29:0/59750160 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaed22a640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 0x55ff3d6453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 -- 172.21.15.29:0/59750160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 msgr2=0x55ff3d6453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 0x55ff3d6453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 -- 172.21.15.29:0/59750160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 msgr2=0x55ff3d644dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:31.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d644dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 -- 172.21.15.29:0/59750160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3d64aa40 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaed22a640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 0x55ff3d6453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffaee22c640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffae0004af0 tx=0x7ffae0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e2ecd79acd67d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.348+0000 7ffad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0045040 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ffae0004450 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae0036040 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffae002f660 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 10 client.27593 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ff3d651e30 con 0x55ff3d21e070 2024-09-21T01:59:31.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae0036040 con 0x55ff3d21e070 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ffae0030cb0 con 0x55ff3d21e070 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 10 client.27593 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 10 client.27593 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 20 client.27593 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.349+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1b1da0 0x55ff3d654570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.350+0000 7ffaf6203740 4 client.27593 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.350+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff3d5398a0 con 0x55ff3d1b1da0 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.350+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1b1da0 0x55ff3d654570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.350+0000 7ffad17fa640 10 client.27593 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.350+0000 7ffaeda2b640 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1b1da0 0x55ff3d654570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0721b8245122b58 server_cookie=9ee27af5526599fc in_seq=0 out_seq=0 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffad17fa640 1 -- 172.21.15.29:0/59750160 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff3d5398a0 con 0x55ff3d1b1da0 2024-09-21T01:59:31.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffad17fa640 10 client.27593 handle_command_reply: tid=0 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 client.27593 shutdown 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 msgr2=0x55ff3d64a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffae0004af0 tx=0x7ffae0005df0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 shutdown_connections 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ff3d1b1da0 0x55ff3d654570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3d21e070 0x55ff3d64a3d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ff3d1fd390 0x55ff3d6453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 --2- 172.21.15.29:0/59750160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ff3d1f51b0 0x55ff3d644dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 >> 172.21.15.29:0/59750160 conn(0x55ff3d212460 msgr2=0x55ff3d5c65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffad17fa640 0 client.27593 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 shutdown_connections 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 1 -- 172.21.15.29:0/59750160 wait complete. 2024-09-21T01:59:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.351+0000 7ffaf6203740 20 client.27593 trim_cache size 0 max 16384 2024-09-21T01:59:31.357 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:59:31.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 msgr2=0x7ffad40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 0x7ffad40b8b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffae002f500 tx=0x7ffae002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 msgr2=0x7ffae819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e9f0 tx=0x7ffad800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 shutdown_connections 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ffad40b6660 0x7ffad40b8b70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae8109e20 0x7ffae819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffae81094d0 0x7ffae819a9f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 --2- 172.21.15.29:0/1571764456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffae8103b90 0x7ffae819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 >> 172.21.15.29:0/1571764456 conn(0x7ffae80ff5e0 msgr2=0x7ffae8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 shutdown_connections 2024-09-21T01:59:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:31.352+0000 7ffaefc8f640 1 -- 172.21.15.29:0/1571764456 wait complete. 2024-09-21T01:59:32.186 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:31 smithi029 ceph-mon[31228]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-09-21T01:59:32.187 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T01:59:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:31 smithi190 ceph-mon[34368]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-09-21T01:59:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:31 smithi066 ceph-mon[35001]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-09-21T01:59:32.621 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 655.0 s, 1219.0 tps, lat 81.835 ms stddev 83.955 2024-09-21T01:59:34.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.032+0000 7f53075d3640 1 -- 172.21.15.29:0/1432856285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 msgr2=0x7f5300075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.032+0000 7f53075d3640 1 --2- 172.21.15.29:0/1432856285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 0x7f5300075b40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009900 tx=0x7f52fc02a410 comp rx=0 tx=0).stop 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.033+0000 7f53075d3640 1 -- 172.21.15.29:0/1432856285 shutdown_connections 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.033+0000 7f53075d3640 1 --2- 172.21.15.29:0/1432856285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f5300112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.033+0000 7f53075d3640 1 --2- 172.21.15.29:0/1432856285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300076080 0x7f530010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.033+0000 7f53075d3640 1 --2- 172.21.15.29:0/1432856285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 0x7f5300075b40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.033+0000 7f53075d3640 1 -- 172.21.15.29:0/1432856285 >> 172.21.15.29:0/1432856285 conn(0x7f53000ff610 msgr2=0x7f5300101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.034+0000 7f53075d3640 1 -- 172.21.15.29:0/1432856285 shutdown_connections 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.034+0000 7f53075d3640 1 -- 172.21.15.29:0/1432856285 wait complete. 2024-09-21T01:59:34.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.035+0000 7f53075d3640 1 Processor -- start 2024-09-21T01:59:34.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.035+0000 7f53075d3640 1 -- start start 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.035+0000 7f53075d3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 0x7f530019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f53075d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300076080 0x7f530019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f53075d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f53075d3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5300115ca0 con 0x7f5300075730 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f53075d3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5300115b20 con 0x7f5300110160 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f53075d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5300115e20 con 0x7f5300076080 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51240/0 (socket says 172.21.15.29:51240) 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 -- 172.21.15.29:0/806149715 learned_addr learned my addr 172.21.15.29:0/806149715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 -- 172.21.15.29:0/806149715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300076080 msgr2=0x7f530019ccb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300076080 0x7f530019ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 -- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 msgr2=0x7f530019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 0x7f530019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.036+0000 7f5305b70640 1 -- 172.21.15.29:0/806149715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53001a2680 con 0x7f5300110160 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f5305b70640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52f800eb10 tx=0x7f52f800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f52f67fc640 1 -- 172.21.15.29:0/806149715 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8004b20 con 0x7f5300110160 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f52f67fc640 1 -- 172.21.15.29:0/806149715 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f8004cc0 con 0x7f5300110160 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f52f67fc640 1 -- 172.21.15.29:0/806149715 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8010720 con 0x7f5300110160 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001a2970 con 0x7f5300110160 2024-09-21T01:59:34.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.037+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001a90d0 con 0x7f5300110160 2024-09-21T01:59:34.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.041+0000 7f52f67fc640 1 -- 172.21.15.29:0/806149715 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f52f8005020 con 0x7f5300110160 2024-09-21T01:59:34.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.041+0000 7f530db47740 1 Processor -- start 2024-09-21T01:59:34.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 -- start start 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f231a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f231b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649f2290ea0 con 0x5649f1f52390 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649f22802d0 con 0x5649f1f73070 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530db47740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649f22840e0 con 0x5649f1f4a1b0 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530536f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530536f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:50968/0 (socket says 172.21.15.29:50968) 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.042+0000 7f530536f640 1 -- 172.21.15.29:0/2434636128 learned_addr learned my addr 172.21.15.29:0/2434636128 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:34.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 -- 172.21.15.29:0/2434636128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 msgr2=0x5649f231a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f231a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 -- 172.21.15.29:0/2434636128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 msgr2=0x5649f231b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f52f67fc640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 0x7f52ec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f231b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 -- 172.21.15.29:0/2434636128 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f1fde650 con 0x5649f1f4a1b0 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f52f67fc640 1 -- 172.21.15.29:0/806149715 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52f80e7910 con 0x7f5300110160 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f530536f640 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005f70 tx=0x7f52fc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b170155da6711a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f5305b70640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 0x7f52ec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.043+0000 7f52eb7fe640 1 -- 172.21.15.29:0/2434636128 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc040070 con 0x5649f1f4a1b0 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.044+0000 7f52eb7fe640 1 -- 172.21.15.29:0/2434636128 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc004040 con 0x5649f1f4a1b0 2024-09-21T01:59:34.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.044+0000 7f52eb7fe640 1 -- 172.21.15.29:0/2434636128 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc035040 con 0x5649f1f4a1b0 2024-09-21T01:59:34.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.044+0000 7f5305b70640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 0x7f52ec0b8aa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f52f80aad20 tx=0x7f52f80058a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.044+0000 7f530db47740 1 -- 172.21.15.29:0/2434636128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 msgr2=0x5649f231abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.044+0000 7f530db47740 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005f70 tx=0x7f52fc000bc0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 -- 172.21.15.29:0/2434636128 shutdown_connections 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f231b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f231abe0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 --2- 172.21.15.29:0/2434636128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f231a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 -- 172.21.15.29:0/2434636128 >> 172.21.15.29:0/2434636128 conn(0x5649f1f67460 msgr2=0x5649f2318590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 -- 172.21.15.29:0/2434636128 shutdown_connections 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 -- 172.21.15.29:0/2434636128 wait complete. 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 -- start start 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.045+0000 7f530db47740 1 Processor -- start 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f232ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f2334350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5649f20e1700 con 0x5649f1f52390 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5649f221b160 con 0x5649f1f73070 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649f2328e70 con 0x5649f1f4a1b0 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530db47740 10 client.0 _ll_register_callbacks cb 0x5649f22349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49690/0 (socket says 172.21.15.29:49690) 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f530536f640 1 -- 172.21.15.29:0/792539444 learned_addr learned my addr 172.21.15.29:0/792539444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f5304b6e640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f232ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.046+0000 7f5304b6e640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f2334350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 -- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 msgr2=0x5649f232ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f232ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 -- 172.21.15.29:0/792539444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 msgr2=0x5649f2334350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f2334350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 -- 172.21.15.29:0/792539444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f2248c90 con 0x5649f1f52390 2024-09-21T01:59:34.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f530536f640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f52fc033000 tx=0x7f52fc032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6f987ad14ccc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.047+0000 7f52e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:34.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.048+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc040040 con 0x5649f1f52390 2024-09-21T01:59:34.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.049+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52fc0314a0 con 0x5649f1f52390 2024-09-21T01:59:34.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.049+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc004040 con 0x5649f1f52390 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.049+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f52fc035040 con 0x5649f1f52390 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.049+0000 7f530db47740 10 client.17772 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.049+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649f233d2a0 con 0x5649f1f52390 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc004040 con 0x5649f1f52390 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f52fc0507c0 con 0x5649f1f52390 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f530db47740 10 client.17772 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f530db47740 10 client.17772 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f530db47740 20 client.17772 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.050+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f319f0 0x5649f2340600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.051+0000 7f530db47740 4 client.17772 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.051+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f2292df0 con 0x5649f1f319f0 2024-09-21T01:59:34.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.051+0000 7f5304b6e640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f319f0 0x5649f2340600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.051+0000 7f5304b6e640 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f319f0 0x5649f2340600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c055c6987d6e85b6 server_cookie=c4a1df2c2c0160e7 in_seq=0 out_seq=0 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.051+0000 7f52e8ff9640 10 client.17772 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f52e8ff9640 1 -- 172.21.15.29:0/792539444 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649f2292df0 con 0x5649f1f319f0 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f52e8ff9640 10 client.17772 handle_command_reply: tid=0 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 client.17772 shutdown 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 msgr2=0x5649f232f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f52fc033000 tx=0x7f52fc032ed0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 shutdown_connections 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f2334350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232f330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f319f0 0x5649f2340600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 --2- 172.21.15.29:0/792539444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f232ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.052+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 >> 172.21.15.29:0/792539444 conn(0x5649f1f67460 msgr2=0x5649f231c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.053+0000 7f52e8ff9640 0 client.17772 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:59:34.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.053+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 shutdown_connections 2024-09-21T01:59:34.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.053+0000 7f530db47740 1 -- 172.21.15.29:0/792539444 wait complete. 2024-09-21T01:59:34.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.053+0000 7f530db47740 20 client.17772 trim_cache size 0 max 16384 2024-09-21T01:59:34.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 Processor -- start 2024-09-21T01:59:34.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 -- start start 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f4a1b0 0x5649f23281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f73070 0x5649f232f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53001ad320 con 0x5649f1f52390 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53001aef20 con 0x5649f1f4a1b0 2024-09-21T01:59:34.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530db47740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53001ad9f0 con 0x5649f1f73070 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49694/0 (socket says 172.21.15.29:49694) 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f530536f640 1 -- 172.21.15.29:0/1983100584 learned_addr learned my addr 172.21.15.29:0/1983100584 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f5304b6e640 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f73070 0x5649f232f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.070+0000 7f5304b6e640 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f4a1b0 0x5649f23281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 -- 172.21.15.29:0/1983100584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f73070 msgr2=0x5649f232f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f73070 0x5649f232f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 -- 172.21.15.29:0/1983100584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f4a1b0 msgr2=0x5649f23281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f4a1b0 0x5649f23281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 -- 172.21.15.29:0/1983100584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f231e7b0 con 0x5649f1f52390 2024-09-21T01:59:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.071+0000 7f530536f640 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005850 tx=0x7f52fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7ccd34b918f363 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.072+0000 7f52eaffd640 1 -- 172.21.15.29:0/1983100584 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc004040 con 0x5649f1f52390 2024-09-21T01:59:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.072+0000 7f52eaffd640 1 -- 172.21.15.29:0/1983100584 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc050480 con 0x5649f1f52390 2024-09-21T01:59:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.072+0000 7f52eaffd640 1 -- 172.21.15.29:0/1983100584 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc003bf0 con 0x5649f1f52390 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 -- 172.21.15.29:0/1983100584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 msgr2=0x5649f232ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005850 tx=0x7f52fc005880 comp rx=0 tx=0).stop 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 -- 172.21.15.29:0/1983100584 shutdown_connections 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f73070 0x5649f232f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f232ed50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 --2- 172.21.15.29:0/1983100584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f4a1b0 0x5649f23281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 -- 172.21.15.29:0/1983100584 >> 172.21.15.29:0/1983100584 conn(0x5649f1f67460 msgr2=0x5649f232d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.073+0000 7f530db47740 1 -- 172.21.15.29:0/1983100584 shutdown_connections 2024-09-21T01:59:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.074+0000 7f530db47740 1 -- 172.21.15.29:0/1983100584 wait complete. 2024-09-21T01:59:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.075+0000 7f530db47740 1 -- start start 2024-09-21T01:59:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.075+0000 7f530db47740 1 Processor -- start 2024-09-21T01:59:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.075+0000 7f530db47740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f2336a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.075+0000 7f530db47740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.075+0000 7f530db47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f23a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530db47740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53001ae790 con 0x5649f1f52390 2024-09-21T01:59:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530db47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53001ae0c0 con 0x5649f1f73070 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530db47740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53001ab320 con 0x5649f1f4a1b0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530db47740 10 client.0 _ll_register_callbacks cb 0x7f52fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49706/0 (socket says 172.21.15.29:49706) 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 -- 172.21.15.29:0/396955107 learned_addr learned my addr 172.21.15.29:0/396955107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f5304b6e640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f2336a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f5304b6e640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f23a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 -- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 msgr2=0x5649f2336a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f2336a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 -- 172.21.15.29:0/396955107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 msgr2=0x5649f23a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f23a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 -- 172.21.15.29:0/396955107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f23377c0 con 0x5649f1f52390 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f530536f640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009850 tx=0x7f52fc02a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f877ad2610364ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.076+0000 7f52e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.077+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc040030 con 0x5649f1f52390 2024-09-21T01:59:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.077+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc050480 con 0x5649f1f52390 2024-09-21T01:59:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.079+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f52fc0316b0 con 0x5649f1f52390 2024-09-21T01:59:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.079+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f52fc071050 con 0x5649f1f52390 2024-09-21T01:59:34.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.079+0000 7f530db47740 10 client.17778 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:34.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.079+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649f23ab2f0 con 0x5649f1f52390 2024-09-21T01:59:34.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.080+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc003e20 con 0x5649f1f52390 2024-09-21T01:59:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f52fc0459f0 con 0x5649f1f52390 2024-09-21T01:59:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 10 client.17778 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 10 client.17778 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T01:59:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 20 client.17778 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f06da0 0x5649f23ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 4 client.17778 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:59:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f52fc050480 con 0x5649f1f06da0 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.081+0000 7f5304b6e640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f06da0 0x5649f23ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f52e8ff9640 10 client.17778 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f5304b6e640 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f06da0 0x5649f23ad910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8faa64adb8c7be5 server_cookie=2fc5ace1769c11fc in_seq=0 out_seq=0 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f52e8ff9640 1 -- 172.21.15.29:0/396955107 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f52fc050480 con 0x5649f1f06da0 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f52e8ff9640 10 client.17778 handle_command_reply: tid=0 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 client.17778 shutdown 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 msgr2=0x5649f2336f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009850 tx=0x7f52fc02a9b0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 shutdown_connections 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649f1f73070 0x5649f23a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.082+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5649f1f52390 0x5649f2336f40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5649f1f06da0 0x5649f23ad910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 1 --2- 172.21.15.29:0/396955107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649f1f4a1b0 0x5649f2336a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f52e8ff9640 0 client.17778 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T01:59:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 >> 172.21.15.29:0/396955107 conn(0x5649f1f67460 msgr2=0x5649f2317e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 shutdown_connections 2024-09-21T01:59:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 1 -- 172.21.15.29:0/396955107 wait complete. 2024-09-21T01:59:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.083+0000 7f530db47740 20 client.17778 trim_cache size 0 max 16384 2024-09-21T01:59:34.086 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 msgr2=0x7f52ec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 0x7f52ec0b8aa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f52f80aad20 tx=0x7f52f80058a0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 msgr2=0x7f53001a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52f800eb10 tx=0x7f52f800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 shutdown_connections 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f52ec0b6590 0x7f52ec0b8aa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5300110160 0x7f53001a1f50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300076080 0x7f530019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 --2- 172.21.15.29:0/806149715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5300075730 0x7f530019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 >> 172.21.15.29:0/806149715 conn(0x7f53000ff610 msgr2=0x7f5300109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:34.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 shutdown_connections 2024-09-21T01:59:34.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:34.084+0000 7f53075d3640 1 -- 172.21.15.29:0/806149715 wait complete. 2024-09-21T01:59:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:33 smithi190 ceph-mon[34368]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-21T01:59:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:33 smithi066 ceph-mon[35001]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-21T01:59:34.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:33 smithi029 ceph-mon[31228]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-21T01:59:35.322 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T01:59:35.717 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:36.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:35 smithi029 ceph-mon[31228]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-09-21T01:59:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:35 smithi190 ceph-mon[34368]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-09-21T01:59:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:35 smithi066 ceph-mon[35001]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-09-21T01:59:37.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 -- 172.21.15.29:0/38997074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c1083a0 msgr2=0x7f118c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 --2- 172.21.15.29:0/38997074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c1083a0 0x7f118c1087b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a410 comp rx=0 tx=0).stop 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 -- 172.21.15.29:0/38997074 shutdown_connections 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 --2- 172.21.15.29:0/38997074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 0x7f118c06ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 --2- 172.21.15.29:0/38997074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c1083a0 0x7f118c1087b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 --2- 172.21.15.29:0/38997074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c102a60 0x7f118c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 -- 172.21.15.29:0/38997074 >> 172.21.15.29:0/38997074 conn(0x7f118c079730 msgr2=0x7f118c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 -- 172.21.15.29:0/38997074 shutdown_connections 2024-09-21T01:59:37.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.007+0000 7f119087b640 1 -- 172.21.15.29:0/38997074 wait complete. 2024-09-21T01:59:37.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.008+0000 7f119087b640 1 Processor -- start 2024-09-21T01:59:37.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.008+0000 7f119087b640 1 -- start start 2024-09-21T01:59:37.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f119087b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f119087b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 0x7f118c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f119087b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 0x7f118c1a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f118a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f118a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51028/0 (socket says 172.21.15.29:51028) 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 0x7f118c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 0x7f118c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49734/0 (socket says 172.21.15.29:49734) 2024-09-21T01:59:37.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f118a59c640 1 -- 172.21.15.29:0/184595639 learned_addr learned my addr 172.21.15.29:0/184595639 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:37.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f118c115db0 con 0x7f118c1083a0 2024-09-21T01:59:37.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f118ad9d640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 0x7f118c1a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.009+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f118c115c30 con 0x7f118c108cf0 2024-09-21T01:59:37.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f118c115f30 con 0x7f118c102a60 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 -- 172.21.15.29:0/184595639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 msgr2=0x7f118c1a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 0x7f118c1a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 -- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 msgr2=0x7f118c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 0x7f118c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c1a27f0 con 0x7f118c102a60 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.010+0000 7f118a59c640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b4a0 tx=0x7f117c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.011+0000 7f117b7fe640 1 -- 172.21.15.29:0/184595639 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c014070 con 0x7f118c102a60 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.011+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c1a2ae0 con 0x7f118c102a60 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.011+0000 7f117b7fe640 1 -- 172.21.15.29:0/184595639 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c010ac0 con 0x7f118c102a60 2024-09-21T01:59:37.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.011+0000 7f117b7fe640 1 -- 172.21.15.29:0/184595639 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c017c30 con 0x7f118c102a60 2024-09-21T01:59:37.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.011+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c1a9240 con 0x7f118c102a60 2024-09-21T01:59:37.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.013+0000 7f117b7fe640 1 -- 172.21.15.29:0/184595639 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f117c017070 con 0x7f118c102a60 2024-09-21T01:59:37.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.014+0000 7f1196def740 1 Processor -- start 2024-09-21T01:59:37.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.015+0000 7f117b7fe640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 0x7f11740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.015+0000 7f117b7fe640 1 -- 172.21.15.29:0/184595639 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f117c0e6840 con 0x7f118c102a60 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.015+0000 7f1196def740 1 -- start start 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee1a390 0x555b1f1e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b1f158ea0 con 0x555b1ee1a390 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b1f1482d0 con 0x555b1ee3b070 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.016+0000 7f1196def740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b1f14c0e0 con 0x555b1ee121b0 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51312/0 (socket says 172.21.15.29:51312) 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 -- 172.21.15.29:0/2370167551 learned_addr learned my addr 172.21.15.29:0/2370167551 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118ad9d640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 0x7f11740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee1a390 0x555b1f1e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 -- 172.21.15.29:0/2370167551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 msgr2=0x555b1f1e2be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e2be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 -- 172.21.15.29:0/2370167551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee1a390 msgr2=0x555b1f1e3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee1a390 0x555b1f1e3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.017+0000 7f118a59c640 1 -- 172.21.15.29:0/2370167551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1eea6650 con 0x555b1ee3b070 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.018+0000 7f118a59c640 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b470 tx=0x7f117c0047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459251ecfc394378 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.018+0000 7f118ad9d640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 0x7f11740b8a20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f118400bd60 tx=0x7f1184009c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.018+0000 7f1178ff9640 1 -- 172.21.15.29:0/2370167551 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c01e070 con 0x555b1ee3b070 2024-09-21T01:59:37.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.018+0000 7f1178ff9640 1 -- 172.21.15.29:0/2370167551 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c0b1460 con 0x555b1ee3b070 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.019+0000 7f1178ff9640 1 -- 172.21.15.29:0/2370167551 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c0abef0 con 0x555b1ee3b070 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.019+0000 7f1196def740 1 -- 172.21.15.29:0/2370167551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 msgr2=0x555b1f1e26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.019+0000 7f1196def740 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b470 tx=0x7f117c0047d0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.019+0000 7f1196def740 1 -- 172.21.15.29:0/2370167551 shutdown_connections 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.019+0000 7f1196def740 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee1a390 0x555b1f1e3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 --2- 172.21.15.29:0/2370167551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee3b070 0x555b1f1e26a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 -- 172.21.15.29:0/2370167551 >> 172.21.15.29:0/2370167551 conn(0x555b1ee2f460 msgr2=0x555b1f1e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 -- 172.21.15.29:0/2370167551 shutdown_connections 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 -- 172.21.15.29:0/2370167551 wait complete. 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 -- start start 2024-09-21T01:59:37.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.020+0000 7f1196def740 1 Processor -- start 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 0x555b1f1edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 0x555b1f1ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f1fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b1efa9700 con 0x555b1ee3b070 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555b1f0e3160 con 0x555b1ee121b0 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555b1f1f0d60 con 0x555b1ee1a390 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1196def740 10 client.0 _ll_register_callbacks cb 0x555b1f0fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 0x555b1f1ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 0x555b1f1ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51038/0 (socket says 172.21.15.29:51038) 2024-09-21T01:59:37.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f1189d9b640 1 -- 172.21.15.29:0/1113163628 learned_addr learned my addr 172.21.15.29:0/1113163628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.021+0000 7f118ad9d640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f1fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 0x555b1f1edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 -- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 msgr2=0x555b1f1ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 0x555b1f1ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 -- 172.21.15.29:0/1113163628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 msgr2=0x555b1f1fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f1fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 -- 172.21.15.29:0/1113163628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1f110c90 con 0x555b1ee121b0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f118a59c640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 0x555b1f1edd10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b470 tx=0x7f117c017650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70147e606299a49b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f1171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c01e030 con 0x555b1ee121b0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f117c010610 con 0x555b1ee121b0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.022+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c0b1460 con 0x555b1ee121b0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f117c0b63a0 con 0x555b1ee121b0 2024-09-21T01:59:37.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 10 client.27617 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555b1f1fca30 con 0x555b1ee121b0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c0b1460 con 0x555b1ee121b0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f117c0a8580 con 0x555b1ee121b0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 10 client.27617 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 10 client.27617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 20 client.27617 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edf99f0 0x555b1f1ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 4 client.27617 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1f15adf0 con 0x555b1edf99f0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.023+0000 7f1189d9b640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edf99f0 0x555b1f1ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.024+0000 7f1171ffb640 10 client.27617 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.024+0000 7f1189d9b640 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edf99f0 0x555b1f1ffea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d90b471f82ab1cba server_cookie=738235de85362c8b in_seq=0 out_seq=0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1171ffb640 1 -- 172.21.15.29:0/1113163628 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b1f15adf0 con 0x555b1edf99f0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1171ffb640 10 client.27617 handle_command_reply: tid=0 2024-09-21T01:59:37.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 client.27617 shutdown 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 msgr2=0x555b1f1edd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 0x555b1f1edd10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b470 tx=0x7f117c017650 comp rx=0 tx=0).stop 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 shutdown_connections 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edf99f0 0x555b1f1ffea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f1fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee1a390 0x555b1f1ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 --2- 172.21.15.29:0/1113163628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee121b0 0x555b1f1edd10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 >> 172.21.15.29:0/1113163628 conn(0x555b1ee2f460 msgr2=0x555b1f1e36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1171ffb640 0 client.27617 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 shutdown_connections 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 1 -- 172.21.15.29:0/1113163628 wait complete. 2024-09-21T01:59:37.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.025+0000 7f1196def740 20 client.27617 trim_cache size 0 max 16384 2024-09-21T01:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.041+0000 7f1196def740 1 Processor -- start 2024-09-21T01:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.042+0000 7f1196def740 1 -- start start 2024-09-21T01:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.042+0000 7f1196def740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.042+0000 7f1196def740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f200eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f1196def740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f2013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f118a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f118a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51042/0 (socket says 172.21.15.29:51042) 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f118a59c640 1 -- 172.21.15.29:0/2126619156 learned_addr learned my addr 172.21.15.29:0/2126619156 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f200eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f118c075160 con 0x555b1ee3b070 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1aabe0 con 0x555b1ee1a390 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1a96b0 con 0x555b1ee121b0 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.043+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f2013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 -- 172.21.15.29:0/2126619156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 msgr2=0x555b1f1e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 -- 172.21.15.29:0/2126619156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 msgr2=0x555b1f2013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f2013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 -- 172.21.15.29:0/2126619156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1f201930 con 0x555b1ee1a390 2024-09-21T01:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f118a59c640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f200eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f1180005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8af519c19743c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1173fff640 1 -- 172.21.15.29:0/2126619156 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180049070 con 0x555b1ee1a390 2024-09-21T01:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1173fff640 1 -- 172.21.15.29:0/2126619156 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11800047a0 con 0x555b1ee1a390 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.044+0000 7f1173fff640 1 -- 172.21.15.29:0/2126619156 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180035040 con 0x555b1ee1a390 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 msgr2=0x555b1f200eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f200eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f1180005a20 comp rx=0 tx=0).stop 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 shutdown_connections 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee3b070 0x555b1f2013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f200eb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 --2- 172.21.15.29:0/2126619156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee121b0 0x555b1f1e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 >> 172.21.15.29:0/2126619156 conn(0x555b1ee2f460 msgr2=0x555b1f1e36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 shutdown_connections 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.045+0000 7f1196def740 1 -- 172.21.15.29:0/2126619156 wait complete. 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.046+0000 7f1196def740 1 -- start start 2024-09-21T01:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.046+0000 7f1196def740 1 Processor -- start 2024-09-21T01:59:37.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.046+0000 7f1196def740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 0x555b1f202ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f2033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 0x555b1f202ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 0x555b1f202ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49770/0 (socket says 172.21.15.29:49770) 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118a59c640 1 -- 172.21.15.29:0/2719299230 learned_addr learned my addr 172.21.15.29:0/2719299230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f2033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 0x555b1f268e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1aa450 con 0x555b1ee121b0 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1a9d80 con 0x555b1ee1a390 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f118c108870 con 0x555b1ee3b070 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 0x555b1f268e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1196def740 10 client.0 _ll_register_callbacks cb 0x7f1184009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 -- 172.21.15.29:0/2719299230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 msgr2=0x555b1f2033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f2033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 -- 172.21.15.29:0/2719299230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 msgr2=0x555b1f202ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 0x555b1f202ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f118ad9d640 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1f203c30 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.047+0000 7f1189d9b640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f2033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f118ad9d640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 0x555b1f268e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1184009e90 tx=0x7f1184004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbf1c1b10a0b163 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118402c070 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1184026620 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11800047a0 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1184014040 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1196def740 10 client.27580 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.048+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555b1f26a320 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11800047a0 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1184025750 con 0x555b1ee3b070 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 10 client.27580 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 10 client.27580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T01:59:37.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 20 client.27580 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edceda0 0x555b1f26cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 4 client.27580 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.049+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b1f1568a0 con 0x555b1edceda0 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f118a59c640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edceda0 0x555b1f26cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f1171ffb640 10 client.27580 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f118a59c640 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edceda0 0x555b1f26cb10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=267f8784e336a496 server_cookie=e0b1403f272087d3 in_seq=0 out_seq=0 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f1171ffb640 1 -- 172.21.15.29:0/2719299230 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b1f1568a0 con 0x555b1edceda0 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f1171ffb640 10 client.27580 handle_command_reply: tid=0 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.050+0000 7f1196def740 1 client.27580 shutdown 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 msgr2=0x555b1f268e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 0x555b1f268e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1184009e90 tx=0x7f1184004a40 comp rx=0 tx=0).stop 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 shutdown_connections 2024-09-21T01:59:37.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x555b1edceda0 0x555b1f26cb10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b1ee3b070 0x555b1f268e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b1ee1a390 0x555b1f2033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 --2- 172.21.15.29:0/2719299230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b1ee121b0 0x555b1f202ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 >> 172.21.15.29:0/2719299230 conn(0x555b1ee2f460 msgr2=0x555b1f1ef120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1171ffb640 0 client.27580 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 shutdown_connections 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 1 -- 172.21.15.29:0/2719299230 wait complete. 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.051+0000 7f1196def740 20 client.27580 trim_cache size 0 max 16384 2024-09-21T01:59:37.057 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:59:37.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 msgr2=0x7f11740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 0x7f11740b8a20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f118400bd60 tx=0x7f1184009c30 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 msgr2=0x7f118c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b4a0 tx=0x7f117c00b960 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 shutdown_connections 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f11740b6510 0x7f11740b8a20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f118c108cf0 0x7f118c1a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f118c1083a0 0x7f118c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 --2- 172.21.15.29:0/184595639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f118c102a60 0x7f118c19c880 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 >> 172.21.15.29:0/184595639 conn(0x7f118c079730 msgr2=0x7f118c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 shutdown_connections 2024-09-21T01:59:37.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:37.052+0000 7f119087b640 1 -- 172.21.15.29:0/184595639 wait complete. 2024-09-21T01:59:37.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T01:59:37.969 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 660.0 s, 1295.4 tps, lat 77.221 ms stddev 89.676 2024-09-21T01:59:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:37 smithi190 ceph-mon[34368]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 725 op/s 2024-09-21T01:59:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:37 smithi066 ceph-mon[35001]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 725 op/s 2024-09-21T01:59:38.311 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:38.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:37 smithi029 ceph-mon[31228]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 725 op/s 2024-09-21T01:59:39.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.276+0000 7fce7ba68640 1 -- 172.21.15.29:0/4281799392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 msgr2=0x7fce74105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.276+0000 7fce7ba68640 1 --2- 172.21.15.29:0/4281799392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce74105800 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fce7001a910 tx=0x7fce7003b470 comp rx=0 tx=0).stop 2024-09-21T01:59:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.281+0000 7fce7ba68640 1 -- 172.21.15.29:0/4281799392 shutdown_connections 2024-09-21T01:59:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.281+0000 7fce7ba68640 1 --2- 172.21.15.29:0/4281799392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74102630 0x7fce74106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.281+0000 7fce7ba68640 1 --2- 172.21.15.29:0/4281799392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74101ce0 0x7fce741020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.281+0000 7fce7ba68640 1 --2- 172.21.15.29:0/4281799392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce74105800 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.281+0000 7fce7ba68640 1 -- 172.21.15.29:0/4281799392 >> 172.21.15.29:0/4281799392 conn(0x7fce74077c60 msgr2=0x7fce740780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.283+0000 7fce7ba68640 1 -- 172.21.15.29:0/4281799392 shutdown_connections 2024-09-21T01:59:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.283+0000 7fce7ba68640 1 -- 172.21.15.29:0/4281799392 wait complete. 2024-09-21T01:59:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.285+0000 7fce7ba68640 1 Processor -- start 2024-09-21T01:59:39.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.285+0000 7fce7ba68640 1 -- start start 2024-09-21T01:59:39.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.286+0000 7fce7ba68640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74101ce0 0x7fce74198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.286+0000 7fce7ba68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 0x7fce741989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.286+0000 7fce7ba68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce7419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.286+0000 7fce79003640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 0x7fce741989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.286+0000 7fce79003640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 0x7fce741989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51068/0 (socket says 172.21.15.29:51068) 2024-09-21T01:59:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.287+0000 7fce79003640 1 -- 172.21.15.29:0/261595293 learned_addr learned my addr 172.21.15.29:0/261595293 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.287+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410b990 con 0x7fce741053f0 2024-09-21T01:59:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.287+0000 7fce7a005640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce7419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.287+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410b810 con 0x7fce74101ce0 2024-09-21T01:59:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.287+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410bb10 con 0x7fce74102630 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce79804640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74101ce0 0x7fce74198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce7a005640 1 -- 172.21.15.29:0/261595293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 msgr2=0x7fce741989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce7a005640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 0x7fce741989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce7a005640 1 -- 172.21.15.29:0/261595293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74101ce0 msgr2=0x7fce74198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce7a005640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74101ce0 0x7fce74198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.289+0000 7fce7a005640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7419e3c0 con 0x7fce741053f0 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.290+0000 7fce7a005640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce7419dc90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b3e0 tx=0x7fce6400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.290+0000 7fce6affd640 1 -- 172.21.15.29:0/261595293 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64015020 con 0x7fce741053f0 2024-09-21T01:59:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.290+0000 7fce6affd640 1 -- 172.21.15.29:0/261595293 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64011ce0 con 0x7fce741053f0 2024-09-21T01:59:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.291+0000 7fce6affd640 1 -- 172.21.15.29:0/261595293 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64010e00 con 0x7fce741053f0 2024-09-21T01:59:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.291+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7419e6b0 con 0x7fce741053f0 2024-09-21T01:59:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.291+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a4e10 con 0x7fce741053f0 2024-09-21T01:59:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.296+0000 7fce81fdc740 1 Processor -- start 2024-09-21T01:59:39.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.299+0000 7fce6affd640 1 -- 172.21.15.29:0/261595293 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fce64018020 con 0x7fce741053f0 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce6affd640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 0x7fce600b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce81fdc740 1 -- start start 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce6affd640 1 -- 172.21.15.29:0/261595293 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fce640e6870 con 0x7fce741053f0 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45921b0 0x5556f4962be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f4963120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.301+0000 7fce81fdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5556f48d8ea0 con 0x5556f45bb070 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.302+0000 7fce81fdc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5556f48c82d0 con 0x5556f45921b0 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.302+0000 7fce81fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5556f48cc0e0 con 0x5556f459a390 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.303+0000 7fce7a005640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f4963120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49796/0 (socket says 172.21.15.29:49796) 2024-09-21T01:59:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 -- 172.21.15.29:0/3545902172 learned_addr learned my addr 172.21.15.29:0/3545902172 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79003640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 0x7fce600b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 -- 172.21.15.29:0/3545902172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 msgr2=0x5556f4963120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f4963120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.305+0000 7fce79804640 1 -- 172.21.15.29:0/3545902172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45921b0 msgr2=0x5556f4962be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.306+0000 7fce79804640 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45921b0 0x5556f4962be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.306+0000 7fce79804640 1 -- 172.21.15.29:0/3545902172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f4626650 con 0x5556f45bb070 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.307+0000 7fce79804640 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fce7003b9e0 tx=0x7fce70004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5667dce220432ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.307+0000 7fce5ffff640 1 -- 172.21.15.29:0/3545902172 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70046070 con 0x5556f45bb070 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.307+0000 7fce5ffff640 1 -- 172.21.15.29:0/3545902172 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70004af0 con 0x5556f45bb070 2024-09-21T01:59:39.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.307+0000 7fce5ffff640 1 -- 172.21.15.29:0/3545902172 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700035f0 con 0x5556f45bb070 2024-09-21T01:59:39.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.309+0000 7fce81fdc740 1 -- 172.21.15.29:0/3545902172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 msgr2=0x5556f49626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.309+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fce7003b9e0 tx=0x7fce70004900 comp rx=0 tx=0).stop 2024-09-21T01:59:39.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.310+0000 7fce79003640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 0x7fce600b8bb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5556f4961320 tx=0x7fce6c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 -- 172.21.15.29:0/3545902172 shutdown_connections 2024-09-21T01:59:39.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f4963120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45921b0 0x5556f4962be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3545902172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45bb070 0x5556f49626a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 -- 172.21.15.29:0/3545902172 >> 172.21.15.29:0/3545902172 conn(0x5556f45af460 msgr2=0x5556f4960590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 -- 172.21.15.29:0/3545902172 shutdown_connections 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.311+0000 7fce81fdc740 1 -- 172.21.15.29:0/3545902172 wait complete. 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.312+0000 7fce81fdc740 1 -- start start 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.312+0000 7fce81fdc740 1 Processor -- start 2024-09-21T01:59:39.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f496ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f498b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5556f4729700 con 0x5556f45921b0 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5556f4863160 con 0x5556f45bb070 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5556f4981e30 con 0x5556f459a390 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce81fdc740 10 client.0 _ll_register_callbacks cb 0x5556f487c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:39.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce79003640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce79003640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51080/0 (socket says 172.21.15.29:51080) 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.313+0000 7fce79003640 1 -- 172.21.15.29:0/3939493440 learned_addr learned my addr 172.21.15.29:0/3939493440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce7a005640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f498b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 -- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 msgr2=0x5556f498b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f498b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 -- 172.21.15.29:0/3939493440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 msgr2=0x5556f496ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f496ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 -- 172.21.15.29:0/3939493440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f4890c90 con 0x5556f459a390 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce79003640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fce6c014de0 tx=0x7fce6c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b8e1df5708be38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.314+0000 7fce5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c022020 con 0x5556f459a390 2024-09-21T01:59:39.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce70004af0 con 0x5556f459a390 2024-09-21T01:59:39.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fce6c013d80 con 0x5556f459a390 2024-09-21T01:59:39.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fce6c01a040 con 0x5556f459a390 2024-09-21T01:59:39.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce81fdc740 10 client.27586 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:39.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5556f49723d0 con 0x5556f459a390 2024-09-21T01:59:39.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70004af0 con 0x5556f459a390 2024-09-21T01:59:39.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.315+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fce6c01d030 con 0x5556f459a390 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 10 client.27586 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 10 client.27586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 20 client.27586 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f45799f0 0x5556f4975d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 4 client.27586 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.316+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f48dadf0 con 0x5556f45799f0 2024-09-21T01:59:39.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.318+0000 7fce79804640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f45799f0 0x5556f4975d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.321+0000 7fce79804640 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f45799f0 0x5556f4975d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=737e82ea9f9195ff server_cookie=31f422295e0dd309 in_seq=0 out_seq=0 2024-09-21T01:59:39.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.321+0000 7fce5d7fa640 10 client.27586 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.321+0000 7fce5d7fa640 1 -- 172.21.15.29:0/3939493440 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556f48dadf0 con 0x5556f45799f0 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.321+0000 7fce5d7fa640 10 client.27586 handle_command_reply: tid=0 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.321+0000 7fce81fdc740 1 client.27586 shutdown 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 msgr2=0x5556f496e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fce6c014de0 tx=0x7fce6c004ad0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 shutdown_connections 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f45799f0 0x5556f4975d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f498b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f459a390 0x5556f496e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 --2- 172.21.15.29:0/3939493440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f496ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 >> 172.21.15.29:0/3939493440 conn(0x5556f45af460 msgr2=0x5556f4965880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce5d7fa640 0 client.27586 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 shutdown_connections 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 1 -- 172.21.15.29:0/3939493440 wait complete. 2024-09-21T01:59:39.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.322+0000 7fce81fdc740 20 client.27586 trim_cache size 0 max 16384 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 Processor -- start 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 -- start start 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f4981540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45bb070 0x5556f498b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce74075180 con 0x5556f45921b0 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce741a6660 con 0x5556f459a390 2024-09-21T01:59:39.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce81fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce741a5130 con 0x5556f45bb070 2024-09-21T01:59:39.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce79003640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce79003640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51390/0 (socket says 172.21.15.29:51390) 2024-09-21T01:59:39.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce79003640 1 -- 172.21.15.29:0/4233929846 learned_addr learned my addr 172.21.15.29:0/4233929846 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:39.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.341+0000 7fce7a005640 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45bb070 0x5556f498b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 -- 172.21.15.29:0/4233929846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45bb070 msgr2=0x5556f498b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45bb070 0x5556f498b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 -- 172.21.15.29:0/4233929846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 msgr2=0x5556f4981540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f4981540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 -- 172.21.15.29:0/4233929846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f4960f00 con 0x5556f459a390 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.342+0000 7fce79003640 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0266f0 tx=0x7fce6c014af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7399468f596b92cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.343+0000 7fce5f7fe640 1 -- 172.21.15.29:0/4233929846 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c022070 con 0x5556f459a390 2024-09-21T01:59:39.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.343+0000 7fce5f7fe640 1 -- 172.21.15.29:0/4233929846 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c026770 con 0x5556f459a390 2024-09-21T01:59:39.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.343+0000 7fce5f7fe640 1 -- 172.21.15.29:0/4233929846 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c029680 con 0x5556f459a390 2024-09-21T01:59:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 -- 172.21.15.29:0/4233929846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 msgr2=0x5556f496e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0266f0 tx=0x7fce6c014af0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 -- 172.21.15.29:0/4233929846 shutdown_connections 2024-09-21T01:59:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45bb070 0x5556f498b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f459a390 0x5556f496e270 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 --2- 172.21.15.29:0/4233929846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f45921b0 0x5556f4981540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 -- 172.21.15.29:0/4233929846 >> 172.21.15.29:0/4233929846 conn(0x5556f45af460 msgr2=0x5556f496db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 -- 172.21.15.29:0/4233929846 shutdown_connections 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.344+0000 7fce81fdc740 1 -- 172.21.15.29:0/4233929846 wait complete. 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.345+0000 7fce81fdc740 1 -- start start 2024-09-21T01:59:39.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.345+0000 7fce81fdc740 1 Processor -- start 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45921b0 0x5556f4983860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f4a0b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce741a5ed0 con 0x5556f459a390 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce741a5800 con 0x5556f45bb070 2024-09-21T01:59:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce79003640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce7a005640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f4a0b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce79003640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49822/0 (socket says 172.21.15.29:49822) 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce79003640 1 -- 172.21.15.29:0/892442947 learned_addr learned my addr 172.21.15.29:0/892442947 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce74106210 con 0x5556f45921b0 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.346+0000 7fce79804640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45921b0 0x5556f4983860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce81fdc740 10 client.0 _ll_register_callbacks cb 0x7fce6c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 -- 172.21.15.29:0/892442947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45921b0 msgr2=0x5556f4983860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45921b0 0x5556f4983860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 -- 172.21.15.29:0/892442947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 msgr2=0x5556f4983de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 -- 172.21.15.29:0/892442947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f4a0bb30 con 0x5556f45bb070 2024-09-21T01:59:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce79003640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce7a005640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f4a0b520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce640e43d0 tx=0x7fce640ac860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f938fb09b2f280ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640a8180 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fce640b1250 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556f4969960 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fce640d00b0 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.347+0000 7fce81fdc740 10 client.27635 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5556f4a0cdd0 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5556f4969960 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fce64022060 con 0x5556f45bb070 2024-09-21T01:59:39.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 10 client.27635 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 10 client.27635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 20 client.27635 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f454eda0 0x5556f4a0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 4 client.27635 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556f495b9e0 con 0x5556f454eda0 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.348+0000 7fce79804640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f454eda0 0x5556f4a0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.349+0000 7fce79804640 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f454eda0 0x5556f4a0f3d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56baf8e7bf6c0aa0 server_cookie=6105cc210323904e in_seq=0 out_seq=0 2024-09-21T01:59:39.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.350+0000 7fce5d7fa640 10 client.27635 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.350+0000 7fce5d7fa640 1 -- 172.21.15.29:0/892442947 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556f495b9e0 con 0x5556f454eda0 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.350+0000 7fce5d7fa640 10 client.27635 handle_command_reply: tid=0 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 client.27635 shutdown 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 msgr2=0x5556f4a0b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f4a0b520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce640e43d0 tx=0x7fce640ac860 comp rx=0 tx=0).stop 2024-09-21T01:59:39.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 shutdown_connections 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5556f454eda0 0x5556f4a0f3d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556f45bb070 0x5556f4a0b520 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556f459a390 0x5556f4983de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 --2- 172.21.15.29:0/892442947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5556f45921b0 0x5556f4983860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce5d7fa640 0 client.27635 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T01:59:39.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 >> 172.21.15.29:0/892442947 conn(0x5556f45af460 msgr2=0x5556f4963680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 shutdown_connections 2024-09-21T01:59:39.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 1 -- 172.21.15.29:0/892442947 wait complete. 2024-09-21T01:59:39.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.351+0000 7fce81fdc740 20 client.27635 trim_cache size 0 max 16384 2024-09-21T01:59:39.360 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 msgr2=0x7fce600b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 0x7fce600b8bb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5556f4961320 tx=0x7fce6c00b040 comp rx=0 tx=0).stop 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 msgr2=0x7fce7419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce7419dc90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b3e0 tx=0x7fce6400b8a0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 shutdown_connections 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fce600b66a0 0x7fce600b8bb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce741053f0 0x7fce7419dc90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce74102630 0x7fce741989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 --2- 172.21.15.29:0/261595293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce74101ce0 0x7fce74198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:39.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 >> 172.21.15.29:0/261595293 conn(0x7fce74077c60 msgr2=0x7fce74100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:39.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 shutdown_connections 2024-09-21T01:59:39.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:39.352+0000 7fce7ba68640 1 -- 172.21.15.29:0/261595293 wait complete. 2024-09-21T01:59:40.045 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:39 smithi029 ceph-mon[31228]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 479 op/s 2024-09-21T01:59:40.081 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T01:59:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:39 smithi190 ceph-mon[34368]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 479 op/s 2024-09-21T01:59:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:39 smithi066 ceph-mon[35001]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 479 op/s 2024-09-21T01:59:40.500 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:41.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 -- 172.21.15.29:0/765990635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9901049b0 msgr2=0x7fc990104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 --2- 172.21.15.29:0/765990635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9901049b0 0x7fc990104dc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc980009860 tx=0x7fc98002a340 comp rx=0 tx=0).stop 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 -- 172.21.15.29:0/765990635 shutdown_connections 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 --2- 172.21.15.29:0/765990635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc99010de10 0x7fc990110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 --2- 172.21.15.29:0/765990635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 0x7fc99010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 --2- 172.21.15.29:0/765990635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9901049b0 0x7fc990104dc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 -- 172.21.15.29:0/765990635 >> 172.21.15.29:0/765990635 conn(0x7fc990079730 msgr2=0x7fc990079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 -- 172.21.15.29:0/765990635 shutdown_connections 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.082+0000 7fc994ebc640 1 -- 172.21.15.29:0/765990635 wait complete. 2024-09-21T01:59:41.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.083+0000 7fc994ebc640 1 Processor -- start 2024-09-21T01:59:41.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.084+0000 7fc994ebc640 1 -- start start 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9901049b0 0x7fc99019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 0x7fc99019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc990113a70 con 0x7fc990105300 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901138f0 con 0x7fc9901049b0 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc994ebc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc990113bf0 con 0x7fc99010de10 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9901049b0 0x7fc99019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc98ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 0x7fc99019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc98ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51116/0 (socket says 172.21.15.29:51116) 2024-09-21T01:59:41.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.085+0000 7fc98ed9d640 1 -- 172.21.15.29:0/1722169732 learned_addr learned my addr 172.21.15.29:0/1722169732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:41.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 -- 172.21.15.29:0/1722169732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9901049b0 msgr2=0x7fc99019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9901049b0 0x7fc99019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 -- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 msgr2=0x7fc99019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 0x7fc99019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 -- 172.21.15.29:0/1722169732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a0490 con 0x7fc99010de10 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc98ed9d640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc98800eb10 tx=0x7fc98800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc97f7fe640 1 -- 172.21.15.29:0/1722169732 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988004b20 con 0x7fc99010de10 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc97f7fe640 1 -- 172.21.15.29:0/1722169732 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc988004cc0 con 0x7fc99010de10 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc97f7fe640 1 -- 172.21.15.29:0/1722169732 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988010670 con 0x7fc99010de10 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a0780 con 0x7fc99010de10 2024-09-21T01:59:41.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.086+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a6ee0 con 0x7fc99010de10 2024-09-21T01:59:41.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.089+0000 7fc99b430740 1 Processor -- start 2024-09-21T01:59:41.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc97f7fe640 1 -- 172.21.15.29:0/1722169732 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc988005020 con 0x7fc99010de10 2024-09-21T01:59:41.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 -- start start 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 0x561c6f55abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f192390 0x561c6f55b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561c6f4d0ea0 con 0x561c6f1b3070 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561c6f4c02d0 con 0x561c6f192390 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc99b430740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561c6f4c40e0 con 0x561c6f18a1b0 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49848/0 (socket says 172.21.15.29:49848) 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc98dd9b640 1 -- 172.21.15.29:0/3185144549 learned_addr learned my addr 172.21.15.29:0/3185144549 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:41.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc98e59c640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 0x561c6f55abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.090+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f192390 0x561c6f55b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 -- 172.21.15.29:0/3185144549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f192390 msgr2=0x561c6f55b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc97f7fe640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 0x7fc9780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f192390 0x561c6f55b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 -- 172.21.15.29:0/3185144549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 msgr2=0x561c6f55a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 -- 172.21.15.29:0/3185144549 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6f21e650 con 0x561c6f18a1b0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98ed9d640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 0x7fc9780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc97f7fe640 1 -- 172.21.15.29:0/1722169732 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc9880e7880 con 0x7fc99010de10 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc98e59c640 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 0x561c6f55abe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc98002a860 tx=0x7fc980005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c13876a1bf30fed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc97cff9640 1 -- 172.21.15.29:0/3185144549 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980041020 con 0x561c6f18a1b0 2024-09-21T01:59:41.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.091+0000 7fc97cff9640 1 -- 172.21.15.29:0/3185144549 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc980030dd0 con 0x561c6f18a1b0 2024-09-21T01:59:41.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.092+0000 7fc97cff9640 1 -- 172.21.15.29:0/3185144549 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980037040 con 0x561c6f18a1b0 2024-09-21T01:59:41.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 -- 172.21.15.29:0/3185144549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 msgr2=0x561c6f55abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 0x561c6f55abe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc98002a860 tx=0x7fc980005d10 comp rx=0 tx=0).stop 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc98ed9d640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 0x7fc9780b8a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc98800f010 tx=0x7fc988013940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 -- 172.21.15.29:0/3185144549 shutdown_connections 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f192390 0x561c6f55b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f18a1b0 0x561c6f55abe0 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc98002a860 tx=0x7fc980005d10 comp rx=0 tx=0).stop 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 --2- 172.21.15.29:0/3185144549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f55a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.093+0000 7fc99b430740 1 -- 172.21.15.29:0/3185144549 >> 172.21.15.29:0/3185144549 conn(0x561c6f1a7460 msgr2=0x561c6f558590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.094+0000 7fc99b430740 1 -- 172.21.15.29:0/3185144549 shutdown_connections 2024-09-21T01:59:41.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.094+0000 7fc99b430740 1 -- 172.21.15.29:0/3185144549 wait complete. 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.094+0000 7fc99b430740 1 -- start start 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 Processor -- start 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f1b3070 0x561c6f566340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f572110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49856/0 (socket says 172.21.15.29:49856) 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc98dd9b640 1 -- 172.21.15.29:0/4119051522 learned_addr learned my addr 172.21.15.29:0/4119051522 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561c6f321700 con 0x561c6f192390 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561c6f45b160 con 0x561c6f1719f0 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561c6f568e20 con 0x561c6f1b3070 2024-09-21T01:59:41.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc98e59c640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f1b3070 0x561c6f566340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.095+0000 7fc99b430740 10 client.0 _ll_register_callbacks cb 0x561c6f4749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 -- 172.21.15.29:0/4119051522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f1b3070 msgr2=0x561c6f566340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f1b3070 0x561c6f566340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 -- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 msgr2=0x561c6f572110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f572110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6f488c90 con 0x561c6f192390 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc984013d80 tx=0x7fc98400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917f93f0d7977774 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc975ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004020 con 0x561c6f192390 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc9840041c0 con 0x561c6f192390 2024-09-21T01:59:41.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.096+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc980030dd0 con 0x561c6f192390 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.097+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc9840043c0 con 0x561c6f192390 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.097+0000 7fc99b430740 10 client.17817 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.097+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c6f574b90 con 0x561c6f192390 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc980030dd0 con 0x561c6f192390 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fc9840105f0 con 0x561c6f192390 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 10 client.17817 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 20 client.17817 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f146da0 0x561c6f5781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 4 client.17817 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c6f4d2df0 con 0x561c6f146da0 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.098+0000 7fc98e59c640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f146da0 0x561c6f5781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.099+0000 7fc98e59c640 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f146da0 0x561c6f5781b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7561b386ed58743 server_cookie=91583eb12b9912cb in_seq=0 out_seq=0 2024-09-21T01:59:41.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.099+0000 7fc975ffb640 10 client.17817 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc975ffb640 1 -- 172.21.15.29:0/4119051522 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c6f4d2df0 con 0x561c6f146da0 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc975ffb640 10 client.17817 handle_command_reply: tid=0 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 client.17817 shutdown 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 msgr2=0x561c6f565e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc984013d80 tx=0x7fc98400f790 comp rx=0 tx=0).stop 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 shutdown_connections 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f146da0 0x561c6f5781b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f572110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f1b3070 0x561c6f566340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 --2- 172.21.15.29:0/4119051522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f192390 0x561c6f565e00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 >> 172.21.15.29:0/4119051522 conn(0x561c6f1a7460 msgr2=0x561c6f55d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc975ffb640 0 client.17817 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:41.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 shutdown_connections 2024-09-21T01:59:41.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 1 -- 172.21.15.29:0/4119051522 wait complete. 2024-09-21T01:59:41.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.100+0000 7fc99b430740 20 client.17817 trim_cache size 0 max 16384 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.117+0000 7fc99b430740 1 Processor -- start 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.117+0000 7fc99b430740 1 -- start start 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f55d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f55dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901af3c0 con 0x561c6f1b3070 2024-09-21T01:59:41.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901b0fc0 con 0x561c6f1719f0 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc99b430740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901afa90 con 0x561c6f192390 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49870/0 (socket says 172.21.15.29:49870) 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2425614448 learned_addr learned my addr 172.21.15.29:0/2425614448 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98e59c640 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f55dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f55d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2425614448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 msgr2=0x561c6f55dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f55dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2425614448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 msgr2=0x561c6f55d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f55d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.118+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2425614448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6f55c990 con 0x561c6f1b3070 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.119+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc984004610 tx=0x7fc984012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5407f0645a88706 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.119+0000 7fc977fff640 1 -- 172.21.15.29:0/2425614448 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984010630 con 0x561c6f1b3070 2024-09-21T01:59:41.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.119+0000 7fc977fff640 1 -- 172.21.15.29:0/2425614448 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98401e030 con 0x561c6f1b3070 2024-09-21T01:59:41.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.119+0000 7fc977fff640 1 -- 172.21.15.29:0/2425614448 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984023410 con 0x561c6f1b3070 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.120+0000 7fc99b430740 1 -- 172.21.15.29:0/2425614448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 msgr2=0x561c6f562ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.120+0000 7fc99b430740 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc984004610 tx=0x7fc984012040 comp rx=0 tx=0).stop 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 -- 172.21.15.29:0/2425614448 shutdown_connections 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f562ff0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f55dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 --2- 172.21.15.29:0/2425614448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f55d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 -- 172.21.15.29:0/2425614448 >> 172.21.15.29:0/2425614448 conn(0x561c6f1a7460 msgr2=0x561c6f564680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 -- 172.21.15.29:0/2425614448 shutdown_connections 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.121+0000 7fc99b430740 1 -- 172.21.15.29:0/2425614448 wait complete. 2024-09-21T01:59:41.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 -- start start 2024-09-21T01:59:41.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 Processor -- start 2024-09-21T01:59:41.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f56e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f56e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901b0830 con 0x561c6f1b3070 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901b0160 con 0x561c6f1719f0 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ad3a0 con 0x561c6f192390 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc99b430740 10 client.0 _ll_register_callbacks cb 0x7fc984013990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51458/0 (socket says 172.21.15.29:51458) 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2508020046 learned_addr learned my addr 172.21.15.29:0/2508020046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f56e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.122+0000 7fc98e59c640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f56e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2508020046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 msgr2=0x561c6f56e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f56e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2508020046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 msgr2=0x561c6f56e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:41.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f56e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 -- 172.21.15.29:0/2508020046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6f56eb70 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc98dd9b640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc98400fd10 tx=0x7fc984009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b93870426bc545e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc975ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984010640 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc98402c450 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98401e030 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc9840107e0 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.123+0000 7fc99b430740 10 client.27650 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c6f5e6f80 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98401e030 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fc98403d040 con 0x561c6f1719f0 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 10 client.27650 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:41.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 10 client.27650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 20 client.27650 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f5e7570 0x561c6f5e9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 4 client.27650 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c6f4ce8a0 con 0x561c6f5e7570 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.124+0000 7fc98e59c640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f5e7570 0x561c6f5e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc975ffb640 10 client.27650 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc98e59c640 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f5e7570 0x561c6f5e9990 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a788e065f867c886 server_cookie=6ec8ac743642ef69 in_seq=0 out_seq=0 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc975ffb640 1 -- 172.21.15.29:0/2508020046 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c6f4ce8a0 con 0x561c6f5e7570 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc975ffb640 10 client.27650 handle_command_reply: tid=0 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 client.27650 shutdown 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 msgr2=0x561c6f5de1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc98400fd10 tx=0x7fc984009640 comp rx=0 tx=0).stop 2024-09-21T01:59:41.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 shutdown_connections 2024-09-21T01:59:41.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561c6f5e7570 0x561c6f5e9990 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561c6f1b3070 0x561c6f56e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c6f192390 0x561c6f56e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 --2- 172.21.15.29:0/2508020046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561c6f1719f0 0x561c6f5de1d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.125+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 >> 172.21.15.29:0/2508020046 conn(0x561c6f1a7460 msgr2=0x561c6f564280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.126+0000 7fc975ffb640 0 client.27650 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.126+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 shutdown_connections 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc99b430740 1 -- 172.21.15.29:0/2508020046 wait complete. 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc99b430740 20 client.27650 trim_cache size 0 max 16384 2024-09-21T01:59:41.130 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 msgr2=0x7fc9780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 0x7fc9780b8a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc98800f010 tx=0x7fc988013940 comp rx=0 tx=0).stop 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 msgr2=0x7fc99019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc98800eb10 tx=0x7fc98800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 shutdown_connections 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc9780b6510 0x7fc9780b8a20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99010de10 0x7fc99019fd60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc990105300 0x7fc99019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 --2- 172.21.15.29:0/1722169732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9901049b0 0x7fc99019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:41.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 >> 172.21.15.29:0/1722169732 conn(0x7fc990079730 msgr2=0x7fc9901008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:41.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 shutdown_connections 2024-09-21T01:59:41.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:41.127+0000 7fc994ebc640 1 -- 172.21.15.29:0/1722169732 wait complete. 2024-09-21T01:59:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 -- 172.21.15.29:0/696857223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e8104740 msgr2=0x7f57e8104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/696857223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e8104740 0x7f57e8104b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d002a400 comp rx=0 tx=0).stop 2024-09-21T01:59:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 -- 172.21.15.29:0/696857223 shutdown_connections 2024-09-21T01:59:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/696857223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 0x7f57e810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/696857223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e8104740 0x7f57e8104b50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/696857223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e81036c0 0x7f57e8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 -- 172.21.15.29:0/696857223 >> 172.21.15.29:0/696857223 conn(0x7f57e80fefe0 msgr2=0x7f57e81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 -- 172.21.15.29:0/696857223 shutdown_connections 2024-09-21T01:59:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.141+0000 7f57ec9f9640 1 -- 172.21.15.29:0/696857223 wait complete. 2024-09-21T01:59:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.142+0000 7f57ec9f9640 1 Processor -- start 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.142+0000 7f57ec9f9640 1 -- start start 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e81036c0 0x7f57e8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 0x7f57e819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57e810f340 con 0x7f57e81036c0 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57e810f1c0 con 0x7f57e8104740 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ec9f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57e810f4c0 con 0x7f57e81050e0 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57670/0 (socket says 172.21.15.29:57670) 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ddd9b640 1 -- 172.21.15.29:0/4193601566 learned_addr learned my addr 172.21.15.29:0/4193601566 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 0x7f57e819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57e659c640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e81036c0 0x7f57e8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.143+0000 7f57ddd9b640 1 -- 172.21.15.29:0/4193601566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 msgr2=0x7f57e819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 0x7f57e819b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ddd9b640 1 -- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e81036c0 msgr2=0x7f57e8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e81036c0 0x7f57e8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ddd9b640 1 -- 172.21.15.29:0/4193601566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e819b950 con 0x7f57e8104740 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f57d0031440 tx=0x7f57d0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57dffff640 1 -- 172.21.15.29:0/4193601566 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d003f070 con 0x7f57e8104740 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e819bbe0 con 0x7f57e8104740 2024-09-21T01:59:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57dffff640 1 -- 172.21.15.29:0/4193601566 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0031c50 con 0x7f57e8104740 2024-09-21T01:59:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.144+0000 7f57dffff640 1 -- 172.21.15.29:0/4193601566 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0035030 con 0x7f57e8104740 2024-09-21T01:59:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.145+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e819bef0 con 0x7f57e8104740 2024-09-21T01:59:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.148+0000 7f57dffff640 1 -- 172.21.15.29:0/4193601566 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f57d0042070 con 0x7f57e8104740 2024-09-21T01:59:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 Processor -- start 2024-09-21T01:59:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 -- start start 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90184d0 0x557da93e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557da93536f0 con 0x557da90102f0 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557da9342b20 con 0x557da90184d0 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57f3a42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557da9346930 con 0x557da90391b0 2024-09-21T01:59:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37268/0 (socket says 172.21.15.29:37268) 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57e6d9d640 1 -- 172.21.15.29:0/2315595380 learned_addr learned my addr 172.21.15.29:0/2315595380 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90184d0 0x557da93e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.149+0000 7f57e659c640 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 -- 172.21.15.29:0/2315595380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90184d0 msgr2=0x557da93e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90184d0 0x557da93e2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 -- 172.21.15.29:0/2315595380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 msgr2=0x557da93e14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93e14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 -- 172.21.15.29:0/2315595380 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da9308530 con 0x557da90391b0 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57dffff640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 0x7f57d40bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e659c640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 0x7f57d40bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f57e0010370 tx=0x7f57e0010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dda11e6783e6d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57dffff640 1 -- 172.21.15.29:0/4193601566 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f57d0107a40 con 0x7f57e8104740 2024-09-21T01:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57dcd99640 1 -- 172.21.15.29:0/2315595380 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0016070 con 0x557da90391b0 2024-09-21T01:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57dcd99640 1 -- 172.21.15.29:0/2315595380 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e0004d20 con 0x557da90391b0 2024-09-21T01:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.150+0000 7f57dcd99640 1 -- 172.21.15.29:0/2315595380 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0005420 con 0x557da90391b0 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 -- 172.21.15.29:0/2315595380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 msgr2=0x557da93e0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f57e0010370 tx=0x7f57e0010830 comp rx=0 tx=0).stop 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57e659c640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 0x7f57d40bceb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009d70 tx=0x7f57d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 -- 172.21.15.29:0/2315595380 shutdown_connections 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90184d0 0x557da93e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93e14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 --2- 172.21.15.29:0/2315595380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93e0e50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 -- 172.21.15.29:0/2315595380 >> 172.21.15.29:0/2315595380 conn(0x557da902d300 msgr2=0x557da93dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 -- 172.21.15.29:0/2315595380 shutdown_connections 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.151+0000 7f57f3a42740 1 -- 172.21.15.29:0/2315595380 wait complete. 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 -- start start 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 Processor -- start 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 0x557da93ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557da927aab0 con 0x557da90184d0 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557da92e8730 con 0x557da90102f0 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557da93e2b30 con 0x557da90391b0 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51440/0 (socket says 172.21.15.29:51440) 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57ddd9b640 1 -- 172.21.15.29:0/1860948891 learned_addr learned my addr 172.21.15.29:0/1860948891 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57e659c640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 0x557da93ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.152+0000 7f57f3a42740 10 client.0 _ll_register_callbacks cb 0x557da933c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 -- 172.21.15.29:0/1860948891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 msgr2=0x557da93eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 -- 172.21.15.29:0/1860948891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 msgr2=0x557da93ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 -- 172.21.15.29:0/1860948891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da90a4790 con 0x557da90102f0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.153+0000 7f57e659c640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 0x557da93ea660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012cb0 tx=0x7f57d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bc37394cade150 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004030 con 0x557da90102f0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f57d8018030 con 0x557da90102f0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e0004d20 con 0x557da90102f0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f57d80041d0 con 0x557da90102f0 2024-09-21T01:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57f3a42740 10 client.27662 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557da911d620 con 0x557da90102f0 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.155+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e0004d20 con 0x557da90102f0 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f57d8003920 con 0x557da90102f0 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 10 client.27662 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 10 client.27662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 20 client.27662 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da8ff7d80 0x557da93f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 4 client.27662 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557da9355640 con 0x557da8ff7d80 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da8ff7d80 0x557da93f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57cdffb640 10 client.27662 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.156+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da8ff7d80 0x557da93f83b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0ecce1ad1f5140 server_cookie=8864c8477fa0d29e in_seq=0 out_seq=0 2024-09-21T01:59:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.157+0000 7f57cdffb640 1 -- 172.21.15.29:0/1860948891 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557da9355640 con 0x557da8ff7d80 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.157+0000 7f57cdffb640 10 client.27662 handle_command_reply: tid=0 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.157+0000 7f57f3a42740 1 client.27662 shutdown 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 msgr2=0x557da93ea660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 0x557da93ea660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012cb0 tx=0x7f57d8004730 comp rx=0 tx=0).stop 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 shutdown_connections 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da8ff7d80 0x557da93f83b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90391b0 0x557da93eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 --2- 172.21.15.29:0/1860948891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da90102f0 0x557da93ea660 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57cdffb640 0 client.27662 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 >> 172.21.15.29:0/1860948891 conn(0x557da902d300 msgr2=0x557da93daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 shutdown_connections 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 1 -- 172.21.15.29:0/1860948891 wait complete. 2024-09-21T01:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.158+0000 7f57f3a42740 20 client.27662 trim_cache size 0 max 16384 2024-09-21T01:59:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 Processor -- start 2024-09-21T01:59:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 -- start start 2024-09-21T01:59:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90102f0 0x557da93f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800e470 con 0x557da90184d0 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8010070 con 0x557da8ff7d80 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.177+0000 7f57f3a42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800eb40 con 0x557da90102f0 2024-09-21T01:59:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57696/0 (socket says 172.21.15.29:57696) 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 -- 172.21.15.29:0/3997184612 learned_addr learned my addr 172.21.15.29:0/3997184612 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90102f0 0x557da93f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 -- 172.21.15.29:0/3997184612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90102f0 msgr2=0x557da93f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90102f0 0x557da93f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 -- 172.21.15.29:0/3997184612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 msgr2=0x557da93e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.178+0000 7f57e659c640 1 -- 172.21.15.29:0/3997184612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da93dc760 con 0x557da8ff7d80 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.180+0000 7f57e659c640 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012cb0 tx=0x7f57d8012d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64af6cdedf97950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.180+0000 7f57cffff640 1 -- 172.21.15.29:0/3997184612 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8038030 con 0x557da8ff7d80 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.180+0000 7f57cffff640 1 -- 172.21.15.29:0/3997184612 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8020740 con 0x557da8ff7d80 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.180+0000 7f57cffff640 1 -- 172.21.15.29:0/3997184612 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80041d0 con 0x557da8ff7d80 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- 172.21.15.29:0/3997184612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 msgr2=0x557da93dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012cb0 tx=0x7f57d8012d30 comp rx=0 tx=0).stop 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- 172.21.15.29:0/3997184612 shutdown_connections 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90184d0 0x557da93e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90102f0 0x557da93f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3997184612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93dc220 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- 172.21.15.29:0/3997184612 >> 172.21.15.29:0/3997184612 conn(0x557da902d300 msgr2=0x557da93f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- 172.21.15.29:0/3997184612 shutdown_connections 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- 172.21.15.29:0/3997184612 wait complete. 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.181+0000 7f57f3a42740 1 -- start start 2024-09-21T01:59:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 Processor -- start 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 0x557da93f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57ddd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51448/0 (socket says 172.21.15.29:51448) 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57ddd9b640 1 -- 172.21.15.29:0/3516284820 learned_addr learned my addr 172.21.15.29:0/3516284820 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800f8e0 con 0x557da90102f0 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57e659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800f210 con 0x557da8ff7d80 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.182+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800c410 con 0x557da90184d0 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 0x557da93f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57f3a42740 10 client.0 _ll_register_callbacks cb 0x7f57d8009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 -- 172.21.15.29:0/3516284820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 msgr2=0x557da93f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 0x557da93f07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 -- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 msgr2=0x557da93ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da93f0f00 con 0x557da90102f0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57e6d9d640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 0x557da93f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57e659c640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57ddd9b640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f57d0105550 tx=0x7f57d00f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc25f0226d0df19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.183+0000 7f57cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0049040 con 0x557da90102f0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f57d00f10b0 con 0x557da90102f0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d8020740 con 0x557da90102f0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f57d00cd350 con 0x557da90102f0 2024-09-21T01:59:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57f3a42740 10 client.17841 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557da93f22c0 con 0x557da90102f0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8020740 con 0x557da90102f0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.184+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f57d003b460 con 0x557da90102f0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 10 client.17841 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 10 client.17841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 20 client.17841 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da90391b0 0x557da9464e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 4 client.17841 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557da93e2d80 con 0x557da90391b0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57e659c640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da90391b0 0x557da9464e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57e659c640 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da90391b0 0x557da9464e40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412528d9d8a664ca server_cookie=c8ac7a0ac45204dd in_seq=0 out_seq=0 2024-09-21T01:59:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.185+0000 7f57cdffb640 10 client.17841 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57cdffb640 1 -- 172.21.15.29:0/3516284820 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557da93e2d80 con 0x557da90391b0 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57cdffb640 10 client.17841 handle_command_reply: tid=0 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 client.17841 shutdown 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 msgr2=0x557da93efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f57d0105550 tx=0x7f57d00f1040 comp rx=0 tx=0).stop 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 shutdown_connections 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557da90391b0 0x557da9464e40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557da90184d0 0x557da93f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557da90102f0 0x557da93efd10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 --2- 172.21.15.29:0/3516284820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557da8ff7d80 0x557da93ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57cdffb640 0 client.17841 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 >> 172.21.15.29:0/3516284820 conn(0x557da902d300 msgr2=0x557da93f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 shutdown_connections 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 1 -- 172.21.15.29:0/3516284820 wait complete. 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.193+0000 7f57f3a42740 20 client.17841 trim_cache size 0 max 16384 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:59:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 msgr2=0x7f57d40bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 0x7f57d40bceb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009d70 tx=0x7f57d8005f90 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 msgr2=0x7f57e819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f57d0031440 tx=0x7f57d0003c10 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 shutdown_connections 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f57d40ba9a0 0x7f57d40bceb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e81050e0 0x7f57e819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e8104740 0x7f57e819a580 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 --2- 172.21.15.29:0/4193601566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e81036c0 0x7f57e8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 >> 172.21.15.29:0/4193601566 conn(0x7f57e80fefe0 msgr2=0x7f57e8107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.194+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 shutdown_connections 2024-09-21T01:59:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:42.195+0000 7f57ec9f9640 1 -- 172.21.15.29:0/4193601566 wait complete. 2024-09-21T01:59:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:41 smithi190 ceph-mon[34368]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 666 op/s 2024-09-21T01:59:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:41 smithi066 ceph-mon[35001]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 666 op/s 2024-09-21T01:59:42.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:41 smithi029 ceph-mon[31228]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 666 op/s 2024-09-21T01:59:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 665.0 s, 2009.4 tps, lat 50.108 ms stddev 54.518 2024-09-21T01:59:43.031 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T01:59:43.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T01:59:43.438 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:44.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 -- 172.21.15.29:0/881278115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2240956c0 msgr2=0x7fe224099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 --2- 172.21.15.29:0/881278115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2240956c0 0x7fe224099bf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe22c069e80 tx=0x7fe22c0980d0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 -- 172.21.15.29:0/881278115 shutdown_connections 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 --2- 172.21.15.29:0/881278115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 0x7fe22409a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 --2- 172.21.15.29:0/881278115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2240956c0 0x7fe224099bf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 --2- 172.21.15.29:0/881278115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe224094d70 0x7fe224095180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.045+0000 7fe22b59e640 1 -- 172.21.15.29:0/881278115 >> 172.21.15.29:0/881278115 conn(0x7fe22400cd80 msgr2=0x7fe22400d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.046+0000 7fe22b59e640 1 -- 172.21.15.29:0/881278115 shutdown_connections 2024-09-21T01:59:44.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.046+0000 7fe22b59e640 1 -- 172.21.15.29:0/881278115 wait complete. 2024-09-21T01:59:44.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.047+0000 7fe22b59e640 1 Processor -- start 2024-09-21T01:59:44.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.047+0000 7fe22b59e640 1 -- start start 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 0x7fe224133070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 0x7fe2241388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240ac580 con 0x7fe22409a1b0 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240ac400 con 0x7fe224094d70 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22b59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240ac700 con 0x7fe2240956c0 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 0x7fe224133070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37314/0 (socket says 172.21.15.29:37314) 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 0x7fe224133070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57724/0 (socket says 172.21.15.29:57724) 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 -- 172.21.15.29:0/3257294399 learned_addr learned my addr 172.21.15.29:0/3257294399 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 0x7fe2241388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 -- 172.21.15.29:0/3257294399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 msgr2=0x7fe224133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 0x7fe224133070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 -- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 msgr2=0x7fe2241388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 0x7fe2241388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.048+0000 7fe229d9b640 1 -- 172.21.15.29:0/3257294399 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe224138fe0 con 0x7fe2240956c0 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe229d9b640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0986c0 tx=0x7fe22c098d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe21b7fe640 1 -- 172.21.15.29:0/3257294399 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0af070 con 0x7fe2240956c0 2024-09-21T01:59:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe224139270 con 0x7fe2240956c0 2024-09-21T01:59:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe21b7fe640 1 -- 172.21.15.29:0/3257294399 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe22c0a1e50 con 0x7fe2240956c0 2024-09-21T01:59:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe21b7fe640 1 -- 172.21.15.29:0/3257294399 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0aa410 con 0x7fe2240956c0 2024-09-21T01:59:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.049+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22413fa30 con 0x7fe2240956c0 2024-09-21T01:59:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.052+0000 7fe21b7fe640 1 -- 172.21.15.29:0/3257294399 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe22c06a760 con 0x7fe2240956c0 2024-09-21T01:59:44.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 Processor -- start 2024-09-21T01:59:44.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 -- start start 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe21b7fe640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 0x7fe2140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 0x7fe2140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563a6e833e60 con 0x563a6e4ed490 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563a6e823290 con 0x563a6e516350 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe236eb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563a6e8270a0 con 0x563a6e4f5670 2024-09-21T01:59:44.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.055+0000 7fe21b7fe640 1 -- 172.21.15.29:0/3257294399 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe22c174270 con 0x7fe2240956c0 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37322/0 (socket says 172.21.15.29:37322) 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 -- 172.21.15.29:0/1996297831 learned_addr learned my addr 172.21.15.29:0/1996297831 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe229d9b640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 0x7fe2140b8a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22a59c640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 -- 172.21.15.29:0/1996297831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 msgr2=0x563a6e8bd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8bd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 -- 172.21.15.29:0/1996297831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 msgr2=0x563a6e8bdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8bdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22ad9d640 1 -- 172.21.15.29:0/1996297831 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6e581930 con 0x563a6e4f5670 2024-09-21T01:59:44.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.056+0000 7fe22a59c640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8bd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:59:44.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.057+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe22001d400 tx=0x7fe22001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9b5d55c7ac1720b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.057+0000 7fe218ff9640 1 -- 172.21.15.29:0/1996297831 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220017040 con 0x563a6e4f5670 2024-09-21T01:59:44.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.057+0000 7fe218ff9640 1 -- 172.21.15.29:0/1996297831 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe220022030 con 0x563a6e4f5670 2024-09-21T01:59:44.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.057+0000 7fe218ff9640 1 -- 172.21.15.29:0/1996297831 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22000f880 con 0x563a6e4f5670 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 -- 172.21.15.29:0/1996297831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 msgr2=0x563a6e8be090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe22001d400 tx=0x7fe22001c000 comp rx=0 tx=0).stop 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 -- 172.21.15.29:0/1996297831 shutdown_connections 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8be090 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8bdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 --2- 172.21.15.29:0/1996297831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8bd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 -- 172.21.15.29:0/1996297831 >> 172.21.15.29:0/1996297831 conn(0x563a6e50a740 msgr2=0x563a6e8bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 -- 172.21.15.29:0/1996297831 shutdown_connections 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.058+0000 7fe236eb7740 1 -- 172.21.15.29:0/1996297831 wait complete. 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.059+0000 7fe236eb7740 1 -- start start 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.059+0000 7fe236eb7740 1 Processor -- start 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4ed490 0x563a6e8c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 0x563a6e8c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563a6e684890 con 0x563a6e4f5670 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563a6e7bdd40 con 0x563a6e4ed490 2024-09-21T01:59:44.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563a6e8cbdf0 con 0x563a6e516350 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe236eb7740 10 client.0 _ll_register_callbacks cb 0x563a6e821c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:37336/0 (socket says 172.21.15.29:37336) 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 -- 172.21.15.29:0/2601733314 learned_addr learned my addr 172.21.15.29:0/2601733314 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 -- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4ed490 msgr2=0x563a6e8c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe229d9b640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 0x563a6e8c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4ed490 0x563a6e8c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 -- 172.21.15.29:0/2601733314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 msgr2=0x563a6e8c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 0x563a6e8c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.060+0000 7fe22ad9d640 1 -- 172.21.15.29:0/2601733314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6e8e6740 con 0x563a6e516350 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe229d9b640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 0x563a6e8c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:59:44.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe22001d3d0 tx=0x7fe2200055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bed5fe98ab2944b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220017070 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe220004250 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe220022030 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe220004450 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe236eb7740 10 client.27637 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563a6e8e8fc0 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.061+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe220022030 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe22002e030 con 0x563a6e516350 2024-09-21T01:59:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 10 client.27637 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 10 client.27637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 20 client.27637 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4d4cd0 0x563a6e8ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 4 client.27637 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6e835db0 con 0x563a6e4d4cd0 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe22a59c640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4d4cd0 0x563a6e8ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe22a59c640 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4d4cd0 0x563a6e8ec5a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e259b87ad759bd39 server_cookie=64ebb23fc859b49e in_seq=0 out_seq=0 2024-09-21T01:59:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.062+0000 7fe211ffb640 10 client.27637 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:44.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe211ffb640 1 -- 172.21.15.29:0/2601733314 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a6e835db0 con 0x563a6e4d4cd0 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe211ffb640 10 client.27637 handle_command_reply: tid=0 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 client.27637 shutdown 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 msgr2=0x563a6e8d5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe22001d3d0 tx=0x7fe2200055e0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 shutdown_connections 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4d4cd0 0x563a6e8ec5a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d5140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4f5670 0x563a6e8c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2601733314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4ed490 0x563a6e8c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.063+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 >> 172.21.15.29:0/2601733314 conn(0x563a6e50a740 msgr2=0x563a6e8c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.064+0000 7fe211ffb640 0 client.27637 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.064+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 shutdown_connections 2024-09-21T01:59:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.064+0000 7fe236eb7740 1 -- 172.21.15.29:0/2601733314 wait complete. 2024-09-21T01:59:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.064+0000 7fe236eb7740 20 client.27637 trim_cache size 0 max 16384 2024-09-21T01:59:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.090+0000 7fe236eb7740 1 Processor -- start 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 -- start start 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe224147de0 con 0x563a6e4ed490 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241499e0 con 0x563a6e516350 2024-09-21T01:59:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.092+0000 7fe236eb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241484b0 con 0x563a6e4f5670 2024-09-21T01:59:44.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.094+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.099+0000 7fe22ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57748/0 (socket says 172.21.15.29:57748) 2024-09-21T01:59:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.099+0000 7fe22ad9d640 1 -- 172.21.15.29:0/3605330866 learned_addr learned my addr 172.21.15.29:0/3605330866 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.100+0000 7fe22ad9d640 1 -- 172.21.15.29:0/3605330866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 msgr2=0x563a6e8c78c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:59:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.100+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8c78c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.100+0000 7fe22ad9d640 1 -- 172.21.15.29:0/3605330866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 msgr2=0x563a6e8d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.100+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.100+0000 7fe22ad9d640 1 -- 172.21.15.29:0/3605330866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6e8cb780 con 0x563a6e516350 2024-09-21T01:59:44.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.103+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe220009530 tx=0x7fe220009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b99ea0e5d7a62e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.119+0000 7fe213fff640 1 -- 172.21.15.29:0/3605330866 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220015040 con 0x563a6e516350 2024-09-21T01:59:44.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.119+0000 7fe213fff640 1 -- 172.21.15.29:0/3605330866 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe22003e020 con 0x563a6e516350 2024-09-21T01:59:44.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.119+0000 7fe213fff640 1 -- 172.21.15.29:0/3605330866 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220017040 con 0x563a6e516350 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 -- 172.21.15.29:0/3605330866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 msgr2=0x563a6e8c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe220009530 tx=0x7fe220009890 comp rx=0 tx=0).stop 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 -- 172.21.15.29:0/3605330866 shutdown_connections 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e516350 0x563a6e8c7e00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e4f5670 0x563a6e8c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 --2- 172.21.15.29:0/3605330866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e8d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.123+0000 7fe236eb7740 1 -- 172.21.15.29:0/3605330866 >> 172.21.15.29:0/3605330866 conn(0x563a6e50a740 msgr2=0x563a6e8c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.134+0000 7fe236eb7740 1 -- 172.21.15.29:0/3605330866 shutdown_connections 2024-09-21T01:59:44.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.137+0000 7fe236eb7740 1 -- 172.21.15.29:0/3605330866 wait complete. 2024-09-21T01:59:44.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.138+0000 7fe236eb7740 1 -- start start 2024-09-21T01:59:44.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.138+0000 7fe236eb7740 1 Processor -- start 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.139+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e9526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.139+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.139+0000 7fe236eb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.139+0000 7fe236eb7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe224149250 con 0x563a6e4ed490 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe236eb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe224148b80 con 0x563a6e4f5670 2024-09-21T01:59:44.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe236eb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe224145dc0 con 0x563a6e516350 2024-09-21T01:59:44.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe229d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe229d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57750/0 (socket says 172.21.15.29:57750) 2024-09-21T01:59:44.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe229d9b640 1 -- 172.21.15.29:0/2292599736 learned_addr learned my addr 172.21.15.29:0/2292599736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:44.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe236eb7740 10 client.0 _ll_register_callbacks cb 0x7fe22c069360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:44.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.140+0000 7fe22ad9d640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.141+0000 7fe22a59c640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e9526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.142+0000 7fe229d9b640 1 -- 172.21.15.29:0/2292599736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 msgr2=0x563a6e8d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.142+0000 7fe229d9b640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.142+0000 7fe229d9b640 1 -- 172.21.15.29:0/2292599736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 msgr2=0x563a6e9526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:44.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.142+0000 7fe229d9b640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e9526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.142+0000 7fe229d9b640 1 -- 172.21.15.29:0/2292599736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6e8d1c80 con 0x563a6e4f5670 2024-09-21T01:59:44.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.147+0000 7fe229d9b640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe22c052910 tx=0x7fe22c143c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6a89370f03d3de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:44.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.147+0000 7fe211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:44.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.148+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c161070 con 0x563a6e4f5670 2024-09-21T01:59:44.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.151+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe22c0a1ba0 con 0x563a6e4f5670 2024-09-21T01:59:44.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.152+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe22003e020 con 0x563a6e4f5670 2024-09-21T01:59:44.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.156+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe22c139bd0 con 0x563a6e4f5670 2024-09-21T01:59:44.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.157+0000 7fe236eb7740 10 client.27680 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:44.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.157+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563a6e967fb0 con 0x563a6e4f5670 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.158+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe22003e020 con 0x563a6e4f5670 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe22c135ed0 con 0x563a6e4f5670 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 10 client.27680 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 20 client.27680 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:44.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4aa080 0x563a6e96a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:44.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 4 client.27680 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T01:59:44.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a6e831860 con 0x563a6e4aa080 2024-09-21T01:59:44.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.159+0000 7fe22a59c640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4aa080 0x563a6e96a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.165+0000 7fe22a59c640 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4aa080 0x563a6e96a780 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348b7cce113036be server_cookie=60fe08e187fba16b in_seq=0 out_seq=0 2024-09-21T01:59:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.165+0000 7fe211ffb640 10 client.27680 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:44.166 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:43 smithi029 ceph-mon[31228]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-21T01:59:44.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.168+0000 7fe211ffb640 1 -- 172.21.15.29:0/2292599736 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a6e831860 con 0x563a6e4aa080 2024-09-21T01:59:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.168+0000 7fe211ffb640 10 client.27680 handle_command_reply: tid=0 2024-09-21T01:59:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.168+0000 7fe236eb7740 1 client.27680 shutdown 2024-09-21T01:59:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.169+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 msgr2=0x563a6e8d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.169+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe22c052910 tx=0x7fe22c143c80 comp rx=0 tx=0).stop 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.172+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 shutdown_connections 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.172+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x563a6e4aa080 0x563a6e96a780 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.172+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563a6e516350 0x563a6e8d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.172+0000 7fe211ffb640 0 client.27680 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a6e4f5670 0x563a6e8d10a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 1 --2- 172.21.15.29:0/2292599736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a6e4ed490 0x563a6e9526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 >> 172.21.15.29:0/2292599736 conn(0x563a6e50a740 msgr2=0x563a6e8c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 shutdown_connections 2024-09-21T01:59:44.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 1 -- 172.21.15.29:0/2292599736 wait complete. 2024-09-21T01:59:44.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.173+0000 7fe236eb7740 20 client.27680 trim_cache size 0 max 16384 2024-09-21T01:59:44.175 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:59:44.175 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T01:59:44.176 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "5aa0003d-8663-4473-83a4-1d4c3aa57a8a", 2024-09-21T01:59:44.176 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T01:59:44.176 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:59:44.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 msgr2=0x7fe2140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 0x7fe2140b8a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220011440 comp rx=0 tx=0).stop 2024-09-21T01:59:44.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 msgr2=0x7fe224133610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:44.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0986c0 tx=0x7fe22c098d40 comp rx=0 tx=0).stop 2024-09-21T01:59:44.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 shutdown_connections 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe2140b6510 0x7fe2140b8a20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22409a1b0 0x7fe2241388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2240956c0 0x7fe224133610 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.177+0000 7fe22b59e640 1 --2- 172.21.15.29:0/3257294399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe224094d70 0x7fe224133070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.178+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 >> 172.21.15.29:0/3257294399 conn(0x7fe22400cd80 msgr2=0x7fe224097bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.178+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 shutdown_connections 2024-09-21T01:59:44.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:44.179+0000 7fe22b59e640 1 -- 172.21.15.29:0/3257294399 wait complete. 2024-09-21T01:59:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:43 smithi190 ceph-mon[34368]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-21T01:59:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:43 smithi066 ceph-mon[35001]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-21T01:59:44.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T01:59:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T01:59:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:44 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.285 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:45.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T01:59:45.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:59:45.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:59:45.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:44 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.347 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.348 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:44 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 -- 172.21.15.29:0/1915159062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac109c10 msgr2=0x7f0cac110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 --2- 172.21.15.29:0/1915159062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac109c10 0x7f0cac110440 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca402a8e0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 -- 172.21.15.29:0/1915159062 shutdown_connections 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 --2- 172.21.15.29:0/1915159062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac109c10 0x7f0cac110440 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 --2- 172.21.15.29:0/1915159062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac1092c0 0x7f0cac1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 --2- 172.21.15.29:0/1915159062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac103990 0x7f0cac103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 -- 172.21.15.29:0/1915159062 >> 172.21.15.29:0/1915159062 conn(0x7f0cac079600 msgr2=0x7f0cac079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 -- 172.21.15.29:0/1915159062 shutdown_connections 2024-09-21T01:59:45.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.889+0000 7f0cab59e640 1 -- 172.21.15.29:0/1915159062 wait complete. 2024-09-21T01:59:45.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.890+0000 7f0cab59e640 1 Processor -- start 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.890+0000 7f0cab59e640 1 -- start start 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 0x7f0cac1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 0x7f0cac198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac109c10 0x7f0cac19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac111780 con 0x7f0cac1092c0 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac111600 con 0x7f0cac103990 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0cab59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac111900 con 0x7f0cac109c10 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0ca9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 0x7f0cac198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 0x7f0cac1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0ca9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 0x7f0cac198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51496/0 (socket says 172.21.15.29:51496) 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/3309689761 learned_addr learned my addr 172.21.15.29:0/3309689761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caad9d640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac109c10 0x7f0cac19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 -- 172.21.15.29:0/3309689761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac109c10 msgr2=0x7f0cac19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac109c10 0x7f0cac19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 -- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 msgr2=0x7f0cac198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 0x7f0cac198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 -- 172.21.15.29:0/3309689761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac19e040 con 0x7f0cac103990 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 0x7f0cac1980d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009fd0 tx=0x7f0c9c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.891+0000 7f0c9b7fe640 1 -- 172.21.15.29:0/3309689761 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c015020 con 0x7f0cac103990 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.892+0000 7f0c9b7fe640 1 -- 172.21.15.29:0/3309689761 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c011ce0 con 0x7f0cac103990 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.892+0000 7f0c9b7fe640 1 -- 172.21.15.29:0/3309689761 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c010de0 con 0x7f0cac103990 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.892+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac19e330 con 0x7f0cac103990 2024-09-21T01:59:45.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.892+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac1a4a90 con 0x7f0cac103990 2024-09-21T01:59:45.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.895+0000 7f0cb72ea740 1 Processor -- start 2024-09-21T01:59:45.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 -- start start 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c54001b0 0x55f1c57d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 0x55f1c57d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f1c5746e50 con 0x55f1c5429070 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f1c5736280 con 0x55f1c5408390 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.896+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f1c573a090 con 0x55f1c54001b0 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0caad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 0x55f1c57d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0caad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51502/0 (socket says 172.21.15.29:51502) 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0caad9d640 1 -- 172.21.15.29:0/384537027 learned_addr learned my addr 172.21.15.29:0/384537027 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/384537027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c54001b0 msgr2=0x55f1c57d0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c54001b0 0x55f1c57d0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/384537027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 msgr2=0x55f1c57d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/384537027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c5494650 con 0x55f1c5408390 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.897+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 0x55f1c57d1290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009c20 tx=0x7f0ca0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef39414af2fa0833 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.898+0000 7f0c98ff9640 1 -- 172.21.15.29:0/384537027 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0005220 con 0x55f1c5408390 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.898+0000 7f0c98ff9640 1 -- 172.21.15.29:0/384537027 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca0004c50 con 0x55f1c5408390 2024-09-21T01:59:45.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.898+0000 7f0c98ff9640 1 -- 172.21.15.29:0/384537027 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0019650 con 0x55f1c5408390 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.898+0000 7f0c9b7fe640 1 -- 172.21.15.29:0/3309689761 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0c9c018020 con 0x7f0cac103990 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 -- 172.21.15.29:0/384537027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 msgr2=0x55f1c57d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 0x55f1c57d1290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009c20 tx=0x7f0ca0011810 comp rx=0 tx=0).stop 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 -- 172.21.15.29:0/384537027 shutdown_connections 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5408390 0x55f1c57d1290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c54001b0 0x55f1c57d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/384537027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0cb72ea740 1 -- 172.21.15.29:0/384537027 >> 172.21.15.29:0/384537027 conn(0x55f1c541d460 msgr2=0x55f1c57ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0c9b7fe640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 0x7f0c940bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0caad9d640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 0x7f0c940bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.899+0000 7f0c9b7fe640 1 -- 172.21.15.29:0/3309689761 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c9c0e6a70 con 0x7f0cac103990 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.900+0000 7f0caad9d640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 0x7f0c940bcf70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca40093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.900+0000 7f0cb72ea740 1 -- 172.21.15.29:0/384537027 shutdown_connections 2024-09-21T01:59:45.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.900+0000 7f0cb72ea740 1 -- 172.21.15.29:0/384537027 wait complete. 2024-09-21T01:59:45.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.901+0000 7f0cb72ea740 1 -- start start 2024-09-21T01:59:45.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.901+0000 7f0cb72ea740 1 Processor -- start 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.901+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 0x55f1c57dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5429070 0x55f1c57dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 0x55f1c57dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 0x55f1c57dbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51512/0 (socket says 172.21.15.29:51512) 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0caa59c640 1 -- 172.21.15.29:0/493505392 learned_addr learned my addr 172.21.15.29:0/493505392 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f1c5597700 con 0x55f1c54001b0 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0caad9d640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5429070 0x55f1c57dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f1c56d1110 con 0x55f1c5429070 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f1c57deff0 con 0x55f1c5408390 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0cb72ea740 10 client.0 _ll_register_callbacks cb 0x55f1c56ea990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5429070 msgr2=0x55f1c57dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.902+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5429070 0x55f1c57dc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/493505392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 msgr2=0x55f1c57dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 0x55f1c57dbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0ca9d9b640 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c56fec40 con 0x55f1c5408390 2024-09-21T01:59:45.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57dc410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ca001b040 tx=0x7f0ca00050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309de9bedc430361 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0c91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:45.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0029070 con 0x55f1c5408390 2024-09-21T01:59:45.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca0004c50 con 0x55f1c5408390 2024-09-21T01:59:45.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0ca0023630 con 0x55f1c5408390 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0ca0035050 con 0x55f1c5408390 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0cb72ea740 10 client.27646 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.903+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f1c57eac30 con 0x55f1c5408390 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca0004c50 con 0x55f1c5408390 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0ca00196e0 con 0x55f1c5408390 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 10 client.27646 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 10 client.27646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 20 client.27646 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53e79f0 0x55f1c57edff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 4 client.27646 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.904+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c5748da0 con 0x55f1c53e79f0 2024-09-21T01:59:45.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.905+0000 7f0caa59c640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53e79f0 0x55f1c57edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.906+0000 7f0c91ffb640 10 client.27646 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:45.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.906+0000 7f0caa59c640 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53e79f0 0x55f1c57edff0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60cc94bcab72df6 server_cookie=49946e17b93f2b5d in_seq=0 out_seq=0 2024-09-21T01:59:45.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.906+0000 7f0c91ffb640 1 -- 172.21.15.29:0/493505392 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1c5748da0 con 0x55f1c53e79f0 2024-09-21T01:59:45.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.906+0000 7f0c91ffb640 10 client.27646 handle_command_reply: tid=0 2024-09-21T01:59:45.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 client.27646 shutdown 2024-09-21T01:59:45.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 msgr2=0x55f1c57dc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57dc410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ca001b040 tx=0x7f0ca00050b0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 shutdown_connections 2024-09-21T01:59:45.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53e79f0 0x55f1c57edff0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c5429070 0x55f1c57dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57dc410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/493505392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c54001b0 0x55f1c57dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 >> 172.21.15.29:0/493505392 conn(0x55f1c541d460 msgr2=0x55f1c57d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.907+0000 7f0c91ffb640 0 client.27646 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.908+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 shutdown_connections 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.908+0000 7f0cb72ea740 1 -- 172.21.15.29:0/493505392 wait complete. 2024-09-21T01:59:45.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.908+0000 7f0cb72ea740 20 client.27646 trim_cache size 0 max 16384 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 Processor -- start 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 -- start start 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ad070 con 0x55f1c5429070 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1aec70 con 0x55f1c54001b0 2024-09-21T01:59:45.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.932+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ad740 con 0x55f1c5408390 2024-09-21T01:59:45.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51522/0 (socket says 172.21.15.29:51522) 2024-09-21T01:59:45.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caad9d640 1 -- 172.21.15.29:0/2182224065 learned_addr learned my addr 172.21.15.29:0/2182224065 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 -- 172.21.15.29:0/2182224065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 msgr2=0x55f1c57ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 -- 172.21.15.29:0/2182224065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 msgr2=0x55f1c57d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 -- 172.21.15.29:0/2182224065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c57ee450 con 0x55f1c54001b0 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0caa59c640 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57dadc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0acb40 tx=0x7f0c9c0e43f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac3b09dfff7f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0c93fff640 1 -- 172.21.15.29:0/2182224065 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0d4170 con 0x55f1c54001b0 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0c93fff640 1 -- 172.21.15.29:0/2182224065 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f1c5748da0 con 0x55f1c54001b0 2024-09-21T01:59:45.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.933+0000 7f0c93fff640 1 -- 172.21.15.29:0/2182224065 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0a8e20 con 0x55f1c54001b0 2024-09-21T01:59:45.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.934+0000 7f0cb72ea740 1 -- 172.21.15.29:0/2182224065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 msgr2=0x55f1c57dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.934+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57dadc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0acb40 tx=0x7f0c9c0e43f0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.936+0000 7f0cb72ea740 1 -- 172.21.15.29:0/2182224065 shutdown_connections 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.936+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.936+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.936+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/2182224065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57dadc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.936+0000 7f0cb72ea740 1 -- 172.21.15.29:0/2182224065 >> 172.21.15.29:0/2182224065 conn(0x55f1c541d460 msgr2=0x55f1c57dd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.937+0000 7f0cb72ea740 1 -- 172.21.15.29:0/2182224065 shutdown_connections 2024-09-21T01:59:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.937+0000 7f0cb72ea740 1 -- 172.21.15.29:0/2182224065 wait complete. 2024-09-21T01:59:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 -- start start 2024-09-21T01:59:45.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 Processor -- start 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ae4e0 con 0x55f1c5429070 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ade10 con 0x55f1c54001b0 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ab050 con 0x55f1c5408390 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0cb72ea740 10 client.0 _ll_register_callbacks cb 0x7f0ca0004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.938+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57816/0 (socket says 172.21.15.29:57816) 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 -- 172.21.15.29:0/3058922076 learned_addr learned my addr 172.21.15.29:0/3058922076 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:45.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caad9d640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 -- 172.21.15.29:0/3058922076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 msgr2=0x55f1c57e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 -- 172.21.15.29:0/3058922076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 msgr2=0x55f1c57e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57e23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 -- 172.21.15.29:0/3058922076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c57e2a70 con 0x55f1c54001b0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caad9d640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0caa59c640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0d3040 tx=0x7f0c9c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4a873df0c3966f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0c91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0b1690 con 0x55f1c54001b0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c9c0a8e20 con 0x55f1c54001b0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.939+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f1c5748da0 con 0x55f1c54001b0 2024-09-21T01:59:45.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0c9c0b65d0 con 0x55f1c54001b0 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 10 client.27698 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f1c5858710 con 0x55f1c54001b0 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f1c5748da0 con 0x55f1c54001b0 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0c9c0b1240 con 0x55f1c54001b0 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 10 client.27698 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 10 client.27698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 20 client.27698 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53bcda0 0x55f1c585aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 4 client.27698 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1c5744850 con 0x55f1c53bcda0 2024-09-21T01:59:45.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.940+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53bcda0 0x55f1c585aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.941+0000 7f0c91ffb640 10 client.27698 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.941+0000 7f0ca9d9b640 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53bcda0 0x55f1c585aeb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3984e408a11b43e7 server_cookie=549dddbfbe1cb6e5 in_seq=0 out_seq=0 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.942+0000 7f0c91ffb640 1 -- 172.21.15.29:0/3058922076 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55f1c5744850 con 0x55f1c53bcda0 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.942+0000 7f0c91ffb640 10 client.27698 handle_command_reply: tid=0 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.942+0000 7f0cb72ea740 1 client.27698 shutdown 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.942+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 msgr2=0x55f1c57e4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.942+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0d3040 tx=0x7f0c9c00bf10 comp rx=0 tx=0).stop 2024-09-21T01:59:45.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 shutdown_connections 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f1c53bcda0 0x55f1c585aeb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1c5429070 0x55f1c57e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1c5408390 0x55f1c57e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 --2- 172.21.15.29:0/3058922076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f1c54001b0 0x55f1c57e4970 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 >> 172.21.15.29:0/3058922076 conn(0x55f1c541d460 msgr2=0x55f1c57e7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0c91ffb640 0 client.27698 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 shutdown_connections 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 1 -- 172.21.15.29:0/3058922076 wait complete. 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.943+0000 7f0cb72ea740 20 client.27698 trim_cache size 0 max 16384 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T01:59:45.946 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "5aa0003d-8663-4473-83a4-1d4c3aa57a8a", 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.947 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.948 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T01:59:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.950 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.951 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T01:59:45.953 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T01:59:45.954 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T01:59:45.955 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T01:59:45.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 msgr2=0x7f0c940bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 0x7f0c940bcf70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca40093e0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 msgr2=0x7f0cac1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:45.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 0x7f0cac1980d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009fd0 tx=0x7f0c9c00b870 comp rx=0 tx=0).stop 2024-09-21T01:59:45.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 shutdown_connections 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c940baa60 0x7f0c940bcf70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cac109c10 0x7f0cac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cac1092c0 0x7f0cac198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 --2- 172.21.15.29:0/3309689761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cac103990 0x7f0cac1980d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 >> 172.21.15.29:0/3309689761 conn(0x7f0cac079600 msgr2=0x7f0cac1072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 shutdown_connections 2024-09-21T01:59:45.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T01:59:45.944+0000 7f0cab59e640 1 -- 172.21.15.29:0/3309689761 wait complete. 2024-09-21T01:59:46.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T01:59:46.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-09-21T01:59:46.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:46.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:46.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T01:59:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-09-21T01:59:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:46.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:46.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T01:59:47.229 INFO:tasks.cephfs.filesystem:scrub status for tag:5aa0003d-8663-4473-83a4-1d4c3aa57a8a - {'path': '/', 'tag': '5aa0003d-8663-4473-83a4-1d4c3aa57a8a', 'options': 'recursive,force'} 2024-09-21T01:59:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 670.0 s, 1571.2 tps, lat 62.938 ms stddev 70.784 2024-09-21T01:59:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:47 smithi190 ceph-mon[34368]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 771 op/s 2024-09-21T01:59:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:47 smithi066 ceph-mon[35001]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 771 op/s 2024-09-21T01:59:48.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:47 smithi029 ceph-mon[31228]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 771 op/s 2024-09-21T01:59:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:49 smithi190 ceph-mon[34368]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-09-21T01:59:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:49 smithi066 ceph-mon[35001]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-09-21T01:59:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:49 smithi029 ceph-mon[31228]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-09-21T01:59:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:51 smithi190 ceph-mon[34368]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.4 MiB/s wr, 821 op/s 2024-09-21T01:59:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:51 smithi066 ceph-mon[35001]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.4 MiB/s wr, 821 op/s 2024-09-21T01:59:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:51 smithi029 ceph-mon[31228]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.4 MiB/s wr, 821 op/s 2024-09-21T01:59:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 675.0 s, 2161.6 tps, lat 46.666 ms stddev 59.375 2024-09-21T01:59:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:53 smithi190 ceph-mon[34368]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-09-21T01:59:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:53 smithi066 ceph-mon[35001]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-09-21T01:59:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:53 smithi029 ceph-mon[31228]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-09-21T01:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.342+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/3496236688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 msgr2=0x7fdb84102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.342+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/3496236688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 0x7fdb84102a40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009900 tx=0x7fdb7402a410 comp rx=0 tx=0).stop 2024-09-21T01:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/3496236688 shutdown_connections 2024-09-21T01:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/3496236688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb8410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/3496236688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb84103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/3496236688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 0x7fdb84102a40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/3496236688 >> 172.21.15.29:0/3496236688 conn(0x7fdb840fdf30 msgr2=0x7fdb841003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/3496236688 shutdown_connections 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.343+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/3496236688 wait complete. 2024-09-21T01:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.344+0000 7fdb8c3e6640 1 Processor -- start 2024-09-21T01:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.344+0000 7fdb8c3e6640 1 -- start start 2024-09-21T01:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.344+0000 7fdb8c3e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 0x7fdb84197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.344+0000 7fdb8c3e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb8c3e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb84198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb8c3e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410d150 con 0x7fdb84103ff0 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb8c3e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410cfd0 con 0x7fdb841025d0 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb8c3e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410d2d0 con 0x7fdb84103650 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb8a983640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb84198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48230/0 (socket says 172.21.15.29:48230) 2024-09-21T01:59:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 -- 172.21.15.29:0/1802855253 learned_addr learned my addr 172.21.15.29:0/1802855253 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 -- 172.21.15.29:0/1802855253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 msgr2=0x7fdb84197d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 0x7fdb84197d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 -- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 msgr2=0x7fdb84198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb84198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.345+0000 7fdb89981640 1 -- 172.21.15.29:0/1802855253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb84199780 con 0x7fdb84103650 2024-09-21T01:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.346+0000 7fdb8a983640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb84198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.346+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ebc0 tx=0x7fdb7c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.346+0000 7fdb7b7fe640 1 -- 172.21.15.29:0/1802855253 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c010430 con 0x7fdb84103650 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.347+0000 7fdb7b7fe640 1 -- 172.21.15.29:0/1802855253 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c010a80 con 0x7fdb84103650 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.347+0000 7fdb7b7fe640 1 -- 172.21.15.29:0/1802855253 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c018af0 con 0x7fdb84103650 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.347+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb84199a70 con 0x7fdb84103650 2024-09-21T01:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.347+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8419ff80 con 0x7fdb84103650 2024-09-21T01:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.350+0000 7fdb7b7fe640 1 -- 172.21.15.29:0/1802855253 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdb7c010c30 con 0x7fdb84103650 2024-09-21T01:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.350+0000 7fdb9342d740 1 Processor -- start 2024-09-21T01:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 -- start start 2024-09-21T01:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772104d0 0x55e8775e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e87754b6f0 con 0x55e8772104d0 2024-09-21T01:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e87753ab20 con 0x55e8772082f0 2024-09-21T01:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb9342d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e87753e930 con 0x55e8772311b0 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb8a983640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775d71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49134/0 (socket says 172.21.15.29:49134) 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb8a182640 1 -- 172.21.15.29:0/2525364085 learned_addr learned my addr 172.21.15.29:0/2525364085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.351+0000 7fdb8a983640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48246/0 (socket says 172.21.15.29:48246) 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb89981640 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772104d0 0x55e8775e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb8a983640 1 -- 172.21.15.29:0/2525364085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 msgr2=0x55e8775d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb8a983640 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb8a983640 1 -- 172.21.15.29:0/2525364085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772104d0 msgr2=0x55e8775e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb8a983640 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772104d0 0x55e8775e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb8a983640 1 -- 172.21.15.29:0/2525364085 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e877500530 con 0x55e8772311b0 2024-09-21T01:59:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.352+0000 7fdb7b7fe640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 0x7fdb700b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb8a983640 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000d380 tx=0x7fdb80009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822423bd9cbe9e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb7b7fe640 1 -- 172.21.15.29:0/1802855253 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb7c0e7bb0 con 0x7fdb84103650 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb78bfe640 1 -- 172.21.15.29:0/2525364085 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80004050 con 0x55e8772311b0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb78bfe640 1 -- 172.21.15.29:0/2525364085 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800054d0 con 0x55e8772311b0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb78bfe640 1 -- 172.21.15.29:0/2525364085 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80004d40 con 0x55e8772311b0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.353+0000 7fdb8a182640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 0x7fdb700b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 -- 172.21.15.29:0/2525364085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 msgr2=0x55e8775d6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb8a182640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 0x7fdb700b8b20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb74031440 tx=0x7fdb74009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000d380 tx=0x7fdb80009740 comp rx=0 tx=0).stop 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 -- 172.21.15.29:0/2525364085 shutdown_connections 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772104d0 0x55e8775e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2525364085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772311b0 0x55e8775d6cb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 -- 172.21.15.29:0/2525364085 >> 172.21.15.29:0/2525364085 conn(0x55e877225300 msgr2=0x55e8775d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 -- 172.21.15.29:0/2525364085 shutdown_connections 2024-09-21T01:59:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.354+0000 7fdb9342d740 1 -- 172.21.15.29:0/2525364085 wait complete. 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb9342d740 1 -- start start 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb9342d740 1 Processor -- start 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775eae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 0x55e8775eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.355+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775eae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49144/0 (socket says 172.21.15.29:49144) 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ec010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb9342d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e877472ab0 con 0x55e8772311b0 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb9342d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e8774e0730 con 0x55e8772082f0 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb9342d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e8775e93e0 con 0x55e8772104d0 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb8a182640 1 -- 172.21.15.29:0/1802542612 learned_addr learned my addr 172.21.15.29:0/1802542612 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 0x55e8775eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb8a983640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb9342d740 10 client.0 _ll_register_callbacks cb 0x55e8775346b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 -- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 msgr2=0x55e8775eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775eae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 -- 172.21.15.29:0/1802542612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 msgr2=0x55e8775ec010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ec010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.356+0000 7fdb89981640 1 -- 172.21.15.29:0/1802542612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87729c790 con 0x55e8772104d0 2024-09-21T01:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb89981640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 0x55e8775eb510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c017040 tx=0x7fdb7c009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7add7e12db8d391 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0adac0 con 0x55e8772104d0 2024-09-21T01:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb7c0ad150 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb800054d0 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c0ad460 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb9342d740 10 client.27661 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.357+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e877315620 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800054d0 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fdb7c0d1070 con 0x55e8772104d0 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 10 client.27661 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 10 client.27661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 20 client.27661 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771efd80 0x55e8775f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 4 client.27661 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e87754d640 con 0x55e8771efd80 2024-09-21T01:59:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.358+0000 7fdb8a182640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771efd80 0x55e8775f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.359+0000 7fdb8a182640 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771efd80 0x55e8775f85d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a5fa61531ba00f server_cookie=f8e7060a2ae2effc in_seq=0 out_seq=0 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.359+0000 7fdb6dffb640 10 client.27661 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb6dffb640 1 -- 172.21.15.29:0/1802542612 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e87754d640 con 0x55e8771efd80 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb6dffb640 10 client.27661 handle_command_reply: tid=0 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 client.27661 shutdown 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 msgr2=0x55e8775eb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 0x55e8775eb510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c017040 tx=0x7fdb7c009200 comp rx=0 tx=0).stop 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 shutdown_connections 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771efd80 0x55e8775f85d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ec010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb6dffb640 0 client.27661 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772104d0 0x55e8775eb510 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1802542612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772082f0 0x55e8775eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.360+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 >> 172.21.15.29:0/1802542612 conn(0x55e877225300 msgr2=0x55e8775d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.361+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 shutdown_connections 2024-09-21T01:59:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.361+0000 7fdb9342d740 1 -- 172.21.15.29:0/1802542612 wait complete. 2024-09-21T01:59:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.361+0000 7fdb9342d740 20 client.27661 trim_cache size 0 max 16384 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.383+0000 7fdb9342d740 1 Processor -- start 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.383+0000 7fdb9342d740 1 -- start start 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841a8de0 con 0x55e8772311b0 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841aa9e0 con 0x55e8772104d0 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb9342d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841a94b0 con 0x55e8772082f0 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb89981640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48266/0 (socket says 172.21.15.29:48266) 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a182640 1 -- 172.21.15.29:0/1238893699 learned_addr learned my addr 172.21.15.29:0/1238893699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 -- 172.21.15.29:0/1238893699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 msgr2=0x55e8775d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 -- 172.21.15.29:0/1238893699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 msgr2=0x55e8775d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 -- 172.21.15.29:0/1238893699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8775d6970 con 0x55e8772311b0 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb89981640 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.384+0000 7fdb8a983640 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775e4c70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009c40 tx=0x7fdb80005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1b695264d30117 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.385+0000 7fdb6ffff640 1 -- 172.21.15.29:0/1238893699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb800051a0 con 0x55e8772311b0 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.385+0000 7fdb6ffff640 1 -- 172.21.15.29:0/1238893699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800035f0 con 0x55e8772311b0 2024-09-21T01:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.385+0000 7fdb6ffff640 1 -- 172.21.15.29:0/1238893699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8002a630 con 0x55e8772311b0 2024-09-21T01:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.386+0000 7fdb9342d740 1 -- 172.21.15.29:0/1238893699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 msgr2=0x55e8775e4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.386+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775e4c70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009c40 tx=0x7fdb80005850 comp rx=0 tx=0).stop 2024-09-21T01:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.386+0000 7fdb9342d740 1 -- 172.21.15.29:0/1238893699 shutdown_connections 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.386+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775e4c70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 --2- 172.21.15.29:0/1238893699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 -- 172.21.15.29:0/1238893699 >> 172.21.15.29:0/1238893699 conn(0x55e877225300 msgr2=0x55e8775d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 -- 172.21.15.29:0/1238893699 shutdown_connections 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 -- 172.21.15.29:0/1238893699 wait complete. 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.387+0000 7fdb9342d740 1 -- start start 2024-09-21T01:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 Processor -- start 2024-09-21T01:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841aa250 con 0x55e8772311b0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841a9b80 con 0x55e8772104d0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841a6dc0 con 0x55e8772082f0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb9342d740 10 client.0 _ll_register_callbacks cb 0x7fdb7c0aa430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.388+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a983640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48280/0 (socket says 172.21.15.29:48280) 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 -- 172.21.15.29:0/2859321976 learned_addr learned my addr 172.21.15.29:0/2859321976 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb89981640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 -- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 msgr2=0x55e8775f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 -- 172.21.15.29:0/2859321976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 msgr2=0x55e8775ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 -- 172.21.15.29:0/2859321976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8775eef00 con 0x55e8772082f0 2024-09-21T01:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a983640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb8a182640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb740099f0 tx=0x7fdb7403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef6dff627fb03e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.389+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74035040 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb800035f0 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdb74005250 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdb74005450 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 10 client.27667 fetch_fsmap learned FSMap version 47 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e877662fd0 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800035f0 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fdb74058610 con 0x55e8772082f0 2024-09-21T01:59:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 10 client.27667 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 10 client.27667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 20 client.27667 populate_metadata read hostname 'smithi029' 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771c51c0 0x55e877665a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 4 client.27667 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.390+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8775490f0 con 0x55e8771c51c0 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.392+0000 7fdb89981640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771c51c0 0x55e877665a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.392+0000 7fdb89981640 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771c51c0 0x55e877665a30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e65ca68e002ab5f server_cookie=8569527ff5abad64 in_seq=0 out_seq=0 2024-09-21T01:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.392+0000 7fdb6dffb640 10 client.27667 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb6dffb640 1 -- 172.21.15.29:0/2859321976 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8775490f0 con 0x55e8771c51c0 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb6dffb640 10 client.27667 handle_command_reply: tid=0 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 client.27667 shutdown 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 msgr2=0x55e8775f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb740099f0 tx=0x7fdb7403d000 comp rx=0 tx=0).stop 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 shutdown_connections 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e8771c51c0 0x55e877665a30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb6dffb640 0 client.27667 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e8772311b0 0x55e8775ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8772104d0 0x55e8775f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 --2- 172.21.15.29:0/2859321976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8772082f0 0x55e8775f27b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 >> 172.21.15.29:0/2859321976 conn(0x55e877225300 msgr2=0x55e8775d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 shutdown_connections 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.400+0000 7fdb9342d740 1 -- 172.21.15.29:0/2859321976 wait complete. 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb9342d740 20 client.27667 trim_cache size 0 max 16384 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T01:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 msgr2=0x7fdb700b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 0x7fdb700b8b20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb74031440 tx=0x7fdb74009520 comp rx=0 tx=0).stop 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 msgr2=0x7fdb841983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ebc0 tx=0x7fdb7c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 shutdown_connections 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdb700b6610 0x7fdb700b8b20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb84103ff0 0x7fdb84198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb84103650 0x7fdb841983b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 --2- 172.21.15.29:0/1802855253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb841025d0 0x7fdb84197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 >> 172.21.15.29:0/1802855253 conn(0x7fdb840fdf30 msgr2=0x7fdb84106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 shutdown_connections 2024-09-21T01:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T01:59:55.401+0000 7fdb8c3e6640 1 -- 172.21.15.29:0/1802855253 wait complete. 2024-09-21T01:59:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:55 smithi190 ceph-mon[34368]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 707 op/s 2024-09-21T01:59:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:55 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:55 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:55 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:55 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:55 smithi066 ceph-mon[35001]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 707 op/s 2024-09-21T01:59:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:55 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:55 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:55 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:55 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:56.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:55 smithi029 ceph-mon[31228]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 707 op/s 2024-09-21T01:59:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:55 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:55 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:55 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:56.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:55 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:57.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:56 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T01:59:57.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:56 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T01:59:57.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:56 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T01:59:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 680.0 s, 2730.6 tps, lat 36.628 ms stddev 44.560 2024-09-21T01:59:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:57 smithi190 ceph-mon[34368]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 855 op/s 2024-09-21T01:59:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:57 smithi066 ceph-mon[35001]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 855 op/s 2024-09-21T01:59:58.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:57 smithi029 ceph-mon[31228]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 855 op/s 2024-09-21T02:00:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 01:59:59 smithi190 ceph-mon[34368]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T02:00:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 01:59:59 smithi066 ceph-mon[35001]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T02:00:00.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 01:59:59 smithi029 ceph-mon[31228]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: Health detail: HEALTH_WARN 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: [WRN] BLUESTORE_SLOW_OP_ALERT: 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.1 observed slow operation indications in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.2 observed slow operation indications in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.4 observed slow operation indications in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.5 observed slow operation indications in BlueStore 2024-09-21T02:00:01.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.7 observed slow operation indications in BlueStore 2024-09-21T02:00:01.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:00 smithi066 ceph-mon[35001]: osd.10 observed slow operation indications in BlueStore 2024-09-21T02:00:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: Health detail: HEALTH_WARN 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: [WRN] BLUESTORE_SLOW_OP_ALERT: 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.1 observed slow operation indications in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.2 observed slow operation indications in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.4 observed slow operation indications in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.5 observed slow operation indications in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.7 observed slow operation indications in BlueStore 2024-09-21T02:00:01.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:00 smithi029 ceph-mon[31228]: osd.10 observed slow operation indications in BlueStore 2024-09-21T02:00:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: Health detail: HEALTH_WARN 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: [WRN] BLUESTORE_SLOW_OP_ALERT: 6 OSD(s) experiencing slow operations in BlueStore 2024-09-21T02:00:01.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.1 observed slow operation indications in BlueStore 2024-09-21T02:00:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.2 observed slow operation indications in BlueStore 2024-09-21T02:00:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.4 observed slow operation indications in BlueStore 2024-09-21T02:00:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.5 observed slow operation indications in BlueStore 2024-09-21T02:00:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.7 observed slow operation indications in BlueStore 2024-09-21T02:00:01.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:00 smithi190 ceph-mon[34368]: osd.10 observed slow operation indications in BlueStore 2024-09-21T02:00:02.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:01 smithi066 ceph-mon[35001]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 767 op/s 2024-09-21T02:00:02.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:01 smithi029 ceph-mon[31228]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 767 op/s 2024-09-21T02:00:02.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:01 smithi190 ceph-mon[34368]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 767 op/s 2024-09-21T02:00:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 685.0 s, 2325.4 tps, lat 42.840 ms stddev 57.157 2024-09-21T02:00:04.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:03 smithi066 ceph-mon[35001]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 682 op/s 2024-09-21T02:00:04.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:03 smithi029 ceph-mon[31228]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 682 op/s 2024-09-21T02:00:04.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:03 smithi190 ceph-mon[34368]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 682 op/s 2024-09-21T02:00:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:05 smithi066 ceph-mon[35001]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-21T02:00:06.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:05 smithi029 ceph-mon[31228]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-21T02:00:06.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:05 smithi190 ceph-mon[34368]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-21T02:00:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 690.0 s, 1386.0 tps, lat 71.945 ms stddev 77.151 2024-09-21T02:00:08.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:07 smithi066 ceph-mon[35001]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-21T02:00:08.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:07 smithi029 ceph-mon[31228]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-21T02:00:08.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:07 smithi190 ceph-mon[34368]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-21T02:00:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 -- 172.21.15.29:0/2073183349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 msgr2=0x7fa0d0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2073183349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d0103a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009860 tx=0x7fa0c402a340 comp rx=0 tx=0).stop 2024-09-21T02:00:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 -- 172.21.15.29:0/2073183349 shutdown_connections 2024-09-21T02:00:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2073183349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 0x7fa0d010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2073183349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d0103a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2073183349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 -- 172.21.15.29:0/2073183349 >> 172.21.15.29:0/2073183349 conn(0x7fa0d00fdee0 msgr2=0x7fa0d0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 -- 172.21.15.29:0/2073183349 shutdown_connections 2024-09-21T02:00:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.542+0000 7fa0d5227640 1 -- 172.21.15.29:0/2073183349 wait complete. 2024-09-21T02:00:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.543+0000 7fa0d5227640 1 Processor -- start 2024-09-21T02:00:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.543+0000 7fa0d5227640 1 -- start start 2024-09-21T02:00:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.543+0000 7fa0d5227640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.543+0000 7fa0d5227640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0d5227640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 0x7fa0d01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0d5227640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010e220 con 0x7fa0d0103fc0 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0d5227640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010e0a0 con 0x7fa0d01025a0 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0d5227640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010e3a0 con 0x7fa0d0103620 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52286/0 (socket says 172.21.15.29:52286) 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 -- 172.21.15.29:0/2316843828 learned_addr learned my addr 172.21.15.29:0/2316843828 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 0x7fa0d01a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 -- 172.21.15.29:0/2316843828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 msgr2=0x7fa0d019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 -- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 msgr2=0x7fa0d01a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 0x7fa0d01a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 -- 172.21.15.29:0/2316843828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d01a1a30 con 0x7fa0d01025a0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.544+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000d950 tx=0x7fa0c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.545+0000 7fa0b7fff640 1 -- 172.21.15.29:0/2316843828 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0017030 con 0x7fa0d01025a0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.545+0000 7fa0b7fff640 1 -- 172.21.15.29:0/2316843828 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c0004020 con 0x7fa0d01025a0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.545+0000 7fa0b7fff640 1 -- 172.21.15.29:0/2316843828 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00047a0 con 0x7fa0d01025a0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.545+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d01a1d20 con 0x7fa0d01025a0 2024-09-21T02:00:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.545+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d01a21b0 con 0x7fa0d01025a0 2024-09-21T02:00:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 Processor -- start 2024-09-21T02:00:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 -- start start 2024-09-21T02:00:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0b7fff640 1 -- 172.21.15.29:0/2316843828 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa0c0004940 con 0x7fa0d01025a0 2024-09-21T02:00:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa365260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa3658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa2d36f0 con 0x55c0f9f902f0 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa2c2b20 con 0x55c0f9f984d0 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0dc269740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa2c6930 con 0x55c0f9fb91b0 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52288/0 (socket says 172.21.15.29:52288) 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0cf59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa365260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.549+0000 7fa0ce59c640 1 -- 172.21.15.29:0/433127075 learned_addr learned my addr 172.21.15.29:0/433127075 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa3658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 -- 172.21.15.29:0/433127075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 msgr2=0x55c0fa365260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa365260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 -- 172.21.15.29:0/433127075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 msgr2=0x55c0fa3658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa3658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 -- 172.21.15.29:0/433127075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fa288530 con 0x55c0f9f984d0 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa3658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa365260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0b7fff640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 0x7fa0b80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009780 tx=0x7fa0c4030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd397e8fde65b909 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0b7fff640 1 -- 172.21.15.29:0/2316843828 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa0c00e6cb0 con 0x7fa0d01025a0 2024-09-21T02:00:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 0x7fa0b80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0b57fa640 1 -- 172.21.15.29:0/433127075 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4037040 con 0x55c0f9f984d0 2024-09-21T02:00:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0b57fa640 1 -- 172.21.15.29:0/433127075 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.550+0000 7fa0b57fa640 1 -- 172.21.15.29:0/433127075 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40053c0 con 0x55c0f9f984d0 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 -- 172.21.15.29:0/433127075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 msgr2=0x55c0fa366420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 0x7fa0b80b8ae0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005e20 tx=0x7fa0c8011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009780 tx=0x7fa0c4030b50 comp rx=0 tx=0).stop 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 -- 172.21.15.29:0/433127075 shutdown_connections 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa366420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa3658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.551+0000 7fa0dc269740 1 --2- 172.21.15.29:0/433127075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa365260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.552+0000 7fa0dc269740 1 -- 172.21.15.29:0/433127075 >> 172.21.15.29:0/433127075 conn(0x55c0f9fad300 msgr2=0x55c0fa35c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.552+0000 7fa0dc269740 1 -- 172.21.15.29:0/433127075 shutdown_connections 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.552+0000 7fa0dc269740 1 -- 172.21.15.29:0/433127075 wait complete. 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.552+0000 7fa0dc269740 1 -- start start 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.552+0000 7fa0dc269740 1 Processor -- start 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa1faab0 con 0x55c0f9f984d0 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa268730 con 0x55c0f9fb91b0 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c0fa366ee0 con 0x55c0f9f902f0 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.553+0000 7fa0dc269740 10 client.0 _ll_register_callbacks cb 0x55c0fa2bc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0cf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46034/0 (socket says 172.21.15.29:46034) 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 -- 172.21.15.29:0/3290647695 learned_addr learned my addr 172.21.15.29:0/3290647695 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 -- 172.21.15.29:0/3290647695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 msgr2=0x55c0fa36e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 -- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 msgr2=0x55c0fa36fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0ce59c640 1 -- 172.21.15.29:0/3290647695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fa024790 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.554+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.555+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004650 tx=0x7fa0c40040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f65001f02593b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.555+0000 7fa0b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.555+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4037050 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.555+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.555+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa0c4005140 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa0c4005340 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 10 client.17898 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0fa09d620 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa0c40036c0 con 0x55c0f9f984d0 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 10 client.17898 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 10 client.17898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 20 client.17898 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f77d80 0x55c0fa37c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 4 client.17898 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.556+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0fa2d5640 con 0x55c0f9f77d80 2024-09-21T02:00:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.557+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f77d80 0x55c0fa37c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.557+0000 7fa0b2ff5640 10 client.17898 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.557+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f77d80 0x55c0fa37c220 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77993d587cb1c801 server_cookie=c9b72cac8ad28ec3 in_seq=0 out_seq=0 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/3290647695 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0fa2d5640 con 0x55c0f9f77d80 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0b2ff5640 10 client.17898 handle_command_reply: tid=0 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 client.17898 shutdown 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 msgr2=0x55c0fa36f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004650 tx=0x7fa0c40040d0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 shutdown_connections 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f77d80 0x55c0fa37c220 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f984d0 0x55c0fa36f040 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 --2- 172.21.15.29:0/3290647695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 >> 172.21.15.29:0/3290647695 conn(0x55c0f9fad300 msgr2=0x55c0fa35aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.558+0000 7fa0b2ff5640 0 client.17898 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.559+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 shutdown_connections 2024-09-21T02:00:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.559+0000 7fa0dc269740 1 -- 172.21.15.29:0/3290647695 wait complete. 2024-09-21T02:00:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.559+0000 7fa0dc269740 20 client.17898 trim_cache size 0 max 16384 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 Processor -- start 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 -- start start 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a8d50 con 0x55c0f9fb91b0 2024-09-21T02:00:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01aa950 con 0x55c0f9f984d0 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0dc269740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a9420 con 0x55c0f9f902f0 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0cf59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52306/0 (socket says 172.21.15.29:52306) 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0ce59c640 1 -- 172.21.15.29:0/1596771280 learned_addr learned my addr 172.21.15.29:0/1596771280 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.580+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.581+0000 7fa0ce59c640 1 -- 172.21.15.29:0/1596771280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 msgr2=0x55c0fa36ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.581+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.581+0000 7fa0ce59c640 1 -- 172.21.15.29:0/1596771280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9fb91b0 msgr2=0x55c0fa36e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.581+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.581+0000 7fa0ce59c640 1 -- 172.21.15.29:0/1596771280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fa3580a0 con 0x55c0f9f984d0 2024-09-21T02:00:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.583+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004620 tx=0x7fa0c40303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe8d2700cfc6a088 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0b4ff9640 1 -- 172.21.15.29:0/1596771280 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4037040 con 0x55c0f9f984d0 2024-09-21T02:00:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0b4ff9640 1 -- 172.21.15.29:0/1596771280 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0b4ff9640 1 -- 172.21.15.29:0/1596771280 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4003850 con 0x55c0f9f984d0 2024-09-21T02:00:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 -- 172.21.15.29:0/1596771280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 msgr2=0x55c0fa36d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004620 tx=0x7fa0c40303b0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 -- 172.21.15.29:0/1596771280 shutdown_connections 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9fb91b0 0x55c0fa36e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa36d410 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.584+0000 7fa0dc269740 1 --2- 172.21.15.29:0/1596771280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9f902f0 0x55c0fa36ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 -- 172.21.15.29:0/1596771280 >> 172.21.15.29:0/1596771280 conn(0x55c0f9fad300 msgr2=0x55c0fa3660a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 -- 172.21.15.29:0/1596771280 shutdown_connections 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 -- 172.21.15.29:0/1596771280 wait complete. 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 -- start start 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 Processor -- start 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa372d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.585+0000 7fa0dc269740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa373f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0dc269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01aa1c0 con 0x55c0f9f902f0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0dc269740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a9af0 con 0x55c0f9f984d0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0dc269740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a6d30 con 0x55c0f9fb91b0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0dc269740 10 client.0 _ll_register_callbacks cb 0x7fa0c8005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52314/0 (socket says 172.21.15.29:52314) 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 -- 172.21.15.29:0/2495581700 learned_addr learned my addr 172.21.15.29:0/2495581700 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa372d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0cf59e640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa373f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 -- 172.21.15.29:0/2495581700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 msgr2=0x55c0fa373f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa373f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 -- 172.21.15.29:0/2495581700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 msgr2=0x55c0fa372d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa372d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 -- 172.21.15.29:0/2495581700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fa374690 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0ce59c640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa0c402a380 tx=0x7fa0c4005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a965884c6c1b4690 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.586+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4037070 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa0c40037f0 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa0c4069050 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 10 client.27731 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0fa375980 con 0x55c0f9f984d0 2024-09-21T02:00:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4042070 con 0x55c0f9f984d0 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa0c40324d0 con 0x55c0f9f984d0 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 10 client.27731 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 10 client.27731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 20 client.27731 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f4d1c0 0x55c0fa376320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 4 client.27731 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0fa2d10f0 con 0x55c0f9f4d1c0 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.587+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f4d1c0 0x55c0fa376320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.588+0000 7fa0b2ff5640 10 client.27731 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.588+0000 7fa0ced9d640 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f4d1c0 0x55c0fa376320 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffdf21b924d6719 server_cookie=bf68bdac6d58eae4 in_seq=0 out_seq=0 2024-09-21T02:00:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0b2ff5640 1 -- 172.21.15.29:0/2495581700 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0fa2d10f0 con 0x55c0f9f4d1c0 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0b2ff5640 10 client.27731 handle_command_reply: tid=0 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0dc269740 1 client.27731 shutdown 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 msgr2=0x55c0fa373410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa0c402a380 tx=0x7fa0c4005970 comp rx=0 tx=0).stop 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.596+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 shutdown_connections 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55c0f9f4d1c0 0x55c0fa376320 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0f9fb91b0 0x55c0fa373f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0f9f984d0 0x55c0fa373410 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0b2ff5640 0 client.27731 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0dc269740 1 --2- 172.21.15.29:0/2495581700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c0f9f902f0 0x55c0fa372d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.597+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 >> 172.21.15.29:0/2495581700 conn(0x55c0f9fad300 msgr2=0x55c0fa36e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.598+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 shutdown_connections 2024-09-21T02:00:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.598+0000 7fa0dc269740 1 -- 172.21.15.29:0/2495581700 wait complete. 2024-09-21T02:00:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.598+0000 7fa0dc269740 20 client.27731 trim_cache size 0 max 16384 2024-09-21T02:00:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:00:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:00:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:00:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:00:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 msgr2=0x7fa0b80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 0x7fa0b80b8ae0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005e20 tx=0x7fa0c8011040 comp rx=0 tx=0).stop 2024-09-21T02:00:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 msgr2=0x7fa0d006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000d950 tx=0x7fa0c000de10 comp rx=0 tx=0).stop 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 shutdown_connections 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa0b80b65d0 0x7fa0b80b8ae0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0d0103fc0 0x7fa0d01a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0103620 0x7fa0d019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 --2- 172.21.15.29:0/2316843828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d01025a0 0x7fa0d006b430 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 >> 172.21.15.29:0/2316843828 conn(0x7fa0d00fdee0 msgr2=0x7fa0d0106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 shutdown_connections 2024-09-21T02:00:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:08.599+0000 7fa0d5227640 1 -- 172.21.15.29:0/2316843828 wait complete. 2024-09-21T02:00:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:09 smithi066 ceph-mon[35001]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-21T02:00:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:09 smithi029 ceph-mon[31228]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-21T02:00:10.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:09 smithi190 ceph-mon[34368]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-21T02:00:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:12 smithi066 ceph-mon[35001]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-09-21T02:00:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:12 smithi029 ceph-mon[31228]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-09-21T02:00:12.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:12 smithi190 ceph-mon[34368]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-09-21T02:00:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 695.0 s, 2598.2 tps, lat 38.576 ms stddev 54.798 2024-09-21T02:00:14.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:14 smithi066 ceph-mon[35001]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T02:00:14.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:14 smithi029 ceph-mon[31228]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T02:00:14.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:14 smithi190 ceph-mon[34368]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T02:00:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:16 smithi066 ceph-mon[35001]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-09-21T02:00:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:16 smithi029 ceph-mon[31228]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-09-21T02:00:16.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:16 smithi190 ceph-mon[34368]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-09-21T02:00:17.230 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:00:17.660 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 700.0 s, 1961.6 tps, lat 51.255 ms stddev 58.806 2024-09-21T02:00:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:18 smithi029 ceph-mon[31228]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 MiB/s wr, 732 op/s 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.279+0000 7f1d2783e640 1 -- 172.21.15.29:0/3118430127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d201081e0 msgr2=0x7f1d201085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.279+0000 7f1d2783e640 1 --2- 172.21.15.29:0/3118430127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d201081e0 0x7f1d201085f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00b880 tx=0x7f1d1c005f90 comp rx=0 tx=0).stop 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 -- 172.21.15.29:0/3118430127 shutdown_connections 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 --2- 172.21.15.29:0/3118430127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d20108b30 0x7f1d2010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 --2- 172.21.15.29:0/3118430127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d201081e0 0x7f1d201085f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 --2- 172.21.15.29:0/3118430127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201028a0 0x7f1d20102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 -- 172.21.15.29:0/3118430127 >> 172.21.15.29:0/3118430127 conn(0x7f1d200fe2f0 msgr2=0x7f1d20100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 -- 172.21.15.29:0/3118430127 shutdown_connections 2024-09-21T02:00:18.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.280+0000 7f1d2783e640 1 -- 172.21.15.29:0/3118430127 wait complete. 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.281+0000 7f1d2783e640 1 Processor -- start 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.281+0000 7f1d2783e640 1 -- start start 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.281+0000 7f1d2783e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 0x7f1d201980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d2783e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 0x7f1d20198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d2783e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d2783e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20111730 con 0x7f1d201028a0 2024-09-21T02:00:18.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d2783e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d201115b0 con 0x7f1d20108b30 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d2783e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d201118b0 con 0x7f1d201081e0 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d255da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 0x7f1d201980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58236/0 (socket says 172.21.15.29:58236) 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d255da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 0x7f1d201980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52040/0 (socket says 172.21.15.29:52040) 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 -- 172.21.15.29:0/1117665503 learned_addr learned my addr 172.21.15.29:0/1117665503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 0x7f1d20198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 -- 172.21.15.29:0/1117665503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 msgr2=0x7f1d20198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 0x7f1d20198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 -- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 msgr2=0x7f1d201980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 0x7f1d201980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d25ddb640 1 -- 172.21.15.29:0/1117665503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2019e030 con 0x7f1d20108b30 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.282+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 0x7f1d20198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:18.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d10012fc0 tx=0x7f1d10010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d167fc640 1 -- 172.21.15.29:0/1117665503 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000d2e0 con 0x7f1d20108b30 2024-09-21T02:00:18.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2019e2c0 con 0x7f1d20108b30 2024-09-21T02:00:18.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d201a4a80 con 0x7f1d20108b30 2024-09-21T02:00:18.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d167fc640 1 -- 172.21.15.29:0/1117665503 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d10005020 con 0x7f1d20108b30 2024-09-21T02:00:18.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.283+0000 7f1d167fc640 1 -- 172.21.15.29:0/1117665503 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10004a00 con 0x7f1d20108b30 2024-09-21T02:00:18.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.287+0000 7f1d2ddb2740 1 Processor -- start 2024-09-21T02:00:18.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d167fc640 1 -- 172.21.15.29:0/1117665503 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1d1001f020 con 0x7f1d20108b30 2024-09-21T02:00:18.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 -- start start 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b22993020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b22993aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562b22900e50 con 0x562b225e3070 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562b228f0280 con 0x562b225ba1b0 2024-09-21T02:00:18.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562b228f4090 con 0x562b225c2390 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d24dd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b22993020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d255da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d255da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58240/0 (socket says 172.21.15.29:58240) 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.288+0000 7f1d255da640 1 -- 172.21.15.29:0/2164320224 learned_addr learned my addr 172.21.15.29:0/2164320224 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b22993aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 -- 172.21.15.29:0/2164320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 msgr2=0x562b22993aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b22993aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 -- 172.21.15.29:0/2164320224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 msgr2=0x562b22993020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b22993020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 -- 172.21.15.29:0/2164320224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b2264e650 con 0x562b225ba1b0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b22993aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d255da640 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800f450 tx=0x7f1d1800f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8408180d4ea7edbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d167fc640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 0x7f1d0c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d0b7fe640 1 -- 172.21.15.29:0/2164320224 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801d070 con 0x562b225ba1b0 2024-09-21T02:00:18.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 0x7f1d0c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d0b7fe640 1 -- 172.21.15.29:0/2164320224 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1800cbb0 con 0x562b225ba1b0 2024-09-21T02:00:18.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d0b7fe640 1 -- 172.21.15.29:0/2164320224 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18018c30 con 0x562b225ba1b0 2024-09-21T02:00:18.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.289+0000 7f1d167fc640 1 -- 172.21.15.29:0/1117665503 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d100e6940 con 0x7f1d20108b30 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/2164320224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 msgr2=0x562b22993560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800f450 tx=0x7f1d1800f910 comp rx=0 tx=0).stop 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/2164320224 shutdown_connections 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b22993aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b22993560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/2164320224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b22993020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.291+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/2164320224 >> 172.21.15.29:0/2164320224 conn(0x562b225d7460 msgr2=0x562b22990f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.292+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/2164320224 shutdown_connections 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.292+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 0x7f1d0c0b8b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d20105d50 tx=0x7f1d1c005710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.292+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/2164320224 wait complete. 2024-09-21T02:00:18.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.293+0000 7f1d2ddb2740 1 -- start start 2024-09-21T02:00:18.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.293+0000 7f1d2ddb2740 1 Processor -- start 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b2299ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b229aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562b22751700 con 0x562b225e3070 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562b2288b110 con 0x562b225ba1b0 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562b229a1800 con 0x562b225c2390 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d255da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d255da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58248/0 (socket says 172.21.15.29:58248) 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d2ddb2740 10 client.0 _ll_register_callbacks cb 0x562b228a4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d24dd9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b2299ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d255da640 1 -- 172.21.15.29:0/1979830720 learned_addr learned my addr 172.21.15.29:0/1979830720 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.294+0000 7f1d255da640 1 -- 172.21.15.29:0/1979830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 msgr2=0x562b2299ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b229aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d255da640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b2299ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d255da640 1 -- 172.21.15.29:0/1979830720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 msgr2=0x562b229aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d255da640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b229aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d255da640 1 -- 172.21.15.29:0/1979830720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b228b8c40 con 0x562b225ba1b0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b229aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d255da640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a010 tx=0x7f1d18018aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461cb13272f95e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18027060 con 0x562b225ba1b0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d18004a40 con 0x562b225ba1b0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1800cbb0 con 0x562b225ba1b0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1d1801d070 con 0x562b225ba1b0 2024-09-21T02:00:18.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d2ddb2740 10 client.27743 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:18.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.295+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562b229ad5a0 con 0x562b225ba1b0 2024-09-21T02:00:18.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.296+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1801e040 con 0x562b225ba1b0 2024-09-21T02:00:18.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1d1800b040 con 0x562b225ba1b0 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 10 client.27743 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 10 client.27743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 20 client.27743 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b225a19f0 0x562b229b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 4 client.27743 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1d1800cbb0 con 0x562b225a19f0 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.297+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b225a19f0 0x562b229b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.298+0000 7f1d08ff9640 10 client.27743 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:18.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.298+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b225a19f0 0x562b229b0960 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be10c0760356d442 server_cookie=b9081914a12fca0f in_seq=0 out_seq=0 2024-09-21T02:00:18.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:18 smithi066 ceph-mon[35001]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 MiB/s wr, 732 op/s 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.298+0000 7f1d08ff9640 1 -- 172.21.15.29:0/1979830720 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1d1800cbb0 con 0x562b225a19f0 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.298+0000 7f1d08ff9640 10 client.27743 handle_command_reply: tid=0 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 client.27743 shutdown 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 msgr2=0x562b2299e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a010 tx=0x7f1d18018aa0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 shutdown_connections 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b225a19f0 0x562b229b0960 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225e3070 0x562b229aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b2299ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d08ff9640 0 client.27743 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/1979830720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225ba1b0 0x562b2299e710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 >> 172.21.15.29:0/1979830720 conn(0x562b225d7460 msgr2=0x562b229961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 shutdown_connections 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/1979830720 wait complete. 2024-09-21T02:00:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.299+0000 7f1d2ddb2740 20 client.27743 trim_cache size 0 max 16384 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 Processor -- start 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 -- start start 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b2299b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0400e470 con 0x562b225ba1b0 2024-09-21T02:00:18.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04010070 con 0x562b225e3070 2024-09-21T02:00:18.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.328+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0400eb40 con 0x562b225c2390 2024-09-21T02:00:18.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.329+0000 7f1d24dd9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.329+0000 7f1d24dd9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34954/0 (socket says 172.21.15.29:34954) 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.329+0000 7f1d24dd9640 1 -- 172.21.15.29:0/89211842 learned_addr learned my addr 172.21.15.29:0/89211842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.329+0000 7f1d255da640 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.330+0000 7f1d24dd9640 1 -- 172.21.15.29:0/89211842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 msgr2=0x562b2299b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.330+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b2299b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.330+0000 7f1d24dd9640 1 -- 172.21.15.29:0/89211842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 msgr2=0x562b229a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.330+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.330+0000 7f1d24dd9640 1 -- 172.21.15.29:0/89211842 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b22991ca0 con 0x562b225c2390 2024-09-21T02:00:18.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.331+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d2006aa90 tx=0x7f1d1c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85781b0d0a91cd35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.331+0000 7f1d0affd640 1 -- 172.21.15.29:0/89211842 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00ab50 con 0x562b225c2390 2024-09-21T02:00:18.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.331+0000 7f1d0affd640 1 -- 172.21.15.29:0/89211842 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c02bca0 con 0x562b225c2390 2024-09-21T02:00:18.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.331+0000 7f1d0affd640 1 -- 172.21.15.29:0/89211842 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c032810 con 0x562b225c2390 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/89211842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 msgr2=0x562b229aaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d2006aa90 tx=0x7f1d1c00d040 comp rx=0 tx=0).stop 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/89211842 shutdown_connections 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b2299b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229aaa80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/89211842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/89211842 >> 172.21.15.29:0/89211842 conn(0x562b225d7460 msgr2=0x562b2299fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/89211842 shutdown_connections 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/89211842 wait complete. 2024-09-21T02:00:18.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.332+0000 7f1d2ddb2740 1 -- start start 2024-09-21T02:00:18.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 Processor -- start 2024-09-21T02:00:18.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0400f8e0 con 0x562b225ba1b0 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0400f210 con 0x562b225e3070 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0400c450 con 0x562b225c2390 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.333+0000 7f1d2ddb2740 10 client.0 _ll_register_callbacks cb 0x7f1d1801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:58274/0 (socket says 172.21.15.29:58274) 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 -- 172.21.15.29:0/229376499 learned_addr learned my addr 172.21.15.29:0/229376499 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d255da640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 -- 172.21.15.29:0/229376499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 msgr2=0x562b229a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 -- 172.21.15.29:0/229376499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 msgr2=0x562b229a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 -- 172.21.15.29:0/229376499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b229a3270 con 0x562b225e3070 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d24dd9640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d255da640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d25ddb640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1d100aca10 tx=0x7f1d10004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b64550ea29de02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d100d0110 con 0x562b225e3070 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1c02bca0 con 0x562b225e3070 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.334+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1d100abce0 con 0x562b225e3070 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1d100ad180 con 0x562b225e3070 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 10 client.27746 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:18.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562b22a1b040 con 0x562b225e3070 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c02bca0 con 0x562b225e3070 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1d10061150 con 0x562b225e3070 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 10 client.27746 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 10 client.27746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 20 client.27746 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b22576da0 0x562b22a1d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 4 client.27746 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b22902da0 con 0x562b22576da0 2024-09-21T02:00:18.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.335+0000 7f1d255da640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b22576da0 0x562b22a1d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d08ff9640 10 client.27746 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d255da640 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b22576da0 0x562b22a1d840 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2941e673a5fa6581 server_cookie=e568a86e5e74b73f in_seq=0 out_seq=0 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d08ff9640 1 -- 172.21.15.29:0/229376499 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x562b22902da0 con 0x562b22576da0 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d08ff9640 10 client.27746 handle_command_reply: tid=0 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 client.27746 shutdown 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 msgr2=0x562b229a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1d100aca10 tx=0x7f1d10004f70 comp rx=0 tx=0).stop 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 shutdown_connections 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x562b22576da0 0x562b22a1d840 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b225e3070 0x562b229a2bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562b225c2390 0x562b229a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d08ff9640 0 client.27746 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:18.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 --2- 172.21.15.29:0/229376499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562b225ba1b0 0x562b229a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.337+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 >> 172.21.15.29:0/229376499 conn(0x562b225d7460 msgr2=0x562b2299fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 shutdown_connections 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2ddb2740 1 -- 172.21.15.29:0/229376499 wait complete. 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2ddb2740 20 client.27746 trim_cache size 0 max 16384 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:00:18.342 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.343 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.344 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.345 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.346 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.347 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:18.348 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:00:18.349 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 msgr2=0x7f1d0c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 0x7f1d0c0b8b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d20105d50 tx=0x7f1d1c005710 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 msgr2=0x7f1d2019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d10012fc0 tx=0x7f1d10010490 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.338+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 shutdown_connections 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1d0c0b6650 0x7f1d0c0b8b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d20108b30 0x7f1d2019d900 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d201081e0 0x7f1d20198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 --2- 172.21.15.29:0/1117665503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d201028a0 0x7f1d201980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 >> 172.21.15.29:0/1117665503 conn(0x7f1d200fe2f0 msgr2=0x7f1d20100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:18.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 shutdown_connections 2024-09-21T02:00:18.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:18.339+0000 7f1d2783e640 1 -- 172.21.15.29:0/1117665503 wait complete. 2024-09-21T02:00:18.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:18 smithi190 ceph-mon[34368]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 MiB/s wr, 732 op/s 2024-09-21T02:00:19.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:00:19.292 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:00:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:20.158 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:20 smithi029 ceph-mon[31228]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-21T02:00:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:20 smithi066 ceph-mon[35001]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f44feffd640 1 -- 172.21.15.29:0/808907757 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f802f320 con 0x7f4500102630 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 -- 172.21.15.29:0/808907757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 msgr2=0x7f4500106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 --2- 172.21.15.29:0/808907757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f4500106190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009ca0 tx=0x7f44f802a8d0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 -- 172.21.15.29:0/808907757 shutdown_connections 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 --2- 172.21.15.29:0/808907757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f4500106190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 --2- 172.21.15.29:0/808907757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4500101ce0 0x7f45001020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 --2- 172.21.15.29:0/808907757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45001053f0 0x7f4500105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.423+0000 7f4506810640 1 -- 172.21.15.29:0/808907757 >> 172.21.15.29:0/808907757 conn(0x7f4500077c60 msgr2=0x7f45000780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.424+0000 7f4506810640 1 -- 172.21.15.29:0/808907757 shutdown_connections 2024-09-21T02:00:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.424+0000 7f4506810640 1 -- 172.21.15.29:0/808907757 wait complete. 2024-09-21T02:00:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.425+0000 7f4506810640 1 Processor -- start 2024-09-21T02:00:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 -- start start 2024-09-21T02:00:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f450006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 0x7f450006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:20.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f450011a140 con 0x7f45001053f0 2024-09-21T02:00:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4500119fc0 con 0x7f4500102630 2024-09-21T02:00:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.426+0000 7f4506810640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f450011a2c0 con 0x7f4500101ce0 2024-09-21T02:00:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34968/0 (socket says 172.21.15.29:34968) 2024-09-21T02:00:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 -- 172.21.15.29:0/2238511402 learned_addr learned my addr 172.21.15.29:0/2238511402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44ff7fe640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f450006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f4504dad640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 0x7f450006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 -- 172.21.15.29:0/2238511402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 msgr2=0x7f450006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f450006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 -- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 msgr2=0x7f450006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:20.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 0x7f450006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 -- 172.21.15.29:0/2238511402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450006edc0 con 0x7f4500101ce0 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44ff7fe640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f450006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f4504dad640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 0x7f450006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.427+0000 7f44fffff640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44f000e9f0 tx=0x7f44f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.428+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f000cd80 con 0x7f4500101ce0 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.428+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4500073a50 con 0x7f4500101ce0 2024-09-21T02:00:20.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.428+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4500073fe0 con 0x7f4500101ce0 2024-09-21T02:00:20.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.432+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44f00108e0 con 0x7f4500101ce0 2024-09-21T02:00:20.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.432+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0018980 con 0x7f4500101ce0 2024-09-21T02:00:20.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.432+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500102cf0 con 0x7f4500101ce0 2024-09-21T02:00:20.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.437+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f44f0018b20 con 0x7f4500101ce0 2024-09-21T02:00:20.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.437+0000 7f44fd7fa640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 0x7f44e80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:20.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.438+0000 7f44ff7fe640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 0x7f44e80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:20.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.438+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f44f00e7f10 con 0x7f4500101ce0 2024-09-21T02:00:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.438+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44f00af6d0 con 0x7f4500101ce0 2024-09-21T02:00:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.439+0000 7f44ff7fe640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 0x7f44e80b8aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:20.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:20 smithi190 ceph-mon[34368]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-21T02:00:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.787+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4500106190 con 0x7f4500101ce0 2024-09-21T02:00:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.791+0000 7f44fd7fa640 1 -- 172.21.15.29:0/2238511402 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11977 (secure 0 0 0) 0x7f44f00b3ab0 con 0x7f4500101ce0 2024-09-21T02:00:20.794 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:00:20.794 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":47,"btime":"2024-09-21T01:59:00:830691+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T01:59:00.246820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T02:00:20.794 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 47 2024-09-21T02:00:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.797+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 msgr2=0x7f44e80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.797+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 0x7f44e80b8aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2024-09-21T02:00:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.797+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 msgr2=0x7f4500074dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.797+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44f000e9f0 tx=0x7f44f000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 shutdown_connections 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f44e80b6590 0x7f44e80b8aa0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45001053f0 0x7f450006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4500102630 0x7f450006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 --2- 172.21.15.29:0/2238511402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4500101ce0 0x7f4500074dd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 >> 172.21.15.29:0/2238511402 conn(0x7f4500077c60 msgr2=0x7f4500100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 shutdown_connections 2024-09-21T02:00:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:20.798+0000 7f4506810640 1 -- 172.21.15.29:0/2238511402 wait complete. 2024-09-21T02:00:21.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:21 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2238511402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:00:21.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:21 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2238511402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:00:21.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:21 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2238511402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.733+0000 7f1f9d8de640 1 -- 172.21.15.29:0/3940555523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981072f0 msgr2=0x7f1f9810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.733+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/3940555523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981072f0 0x7f1f9810dc90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009be0 tx=0x7f1f9002a6c0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 -- 172.21.15.29:0/3940555523 shutdown_connections 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/3940555523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981072f0 0x7f1f9810dc90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/3940555523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 0x7f1f98106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/3940555523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981058d0 0x7f1f98105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 -- 172.21.15.29:0/3940555523 >> 172.21.15.29:0/3940555523 conn(0x7f1f981011f0 msgr2=0x7f1f981036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 -- 172.21.15.29:0/3940555523 shutdown_connections 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 -- 172.21.15.29:0/3940555523 wait complete. 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 Processor -- start 2024-09-21T02:00:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.734+0000 7f1f9d8de640 1 -- start start 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 0x7f1f9819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 0x7f1f9819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111500 con 0x7f1f981072f0 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111380 con 0x7f1f98106950 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f9d8de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111680 con 0x7f1f981058d0 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 0x7f1f9819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:34984/0 (socket says 172.21.15.29:34984) 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f96ffd640 1 -- 172.21.15.29:0/2683823776 learned_addr learned my addr 172.21.15.29:0/2683823776 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f967fc640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 0x7f1f9819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 -- 172.21.15.29:0/2683823776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 msgr2=0x7f1f9819c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 -- 172.21.15.29:0/2683823776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 msgr2=0x7f1f9819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 0x7f1f9819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f977fe640 1 -- 172.21.15.29:0/2683823776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9819db60 con 0x7f1f981072f0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.735+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.736+0000 7f1f977fe640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 0x7f1f9819d340 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1f9002abe0 tx=0x7f1f90030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.736+0000 7f1f9c8dc640 1 -- 172.21.15.29:0/2683823776 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90041020 con 0x7f1f981072f0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.736+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9819ddf0 con 0x7f1f981072f0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.736+0000 7f1f9c8dc640 1 -- 172.21.15.29:0/2683823776 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900318f0 con 0x7f1f981072f0 2024-09-21T02:00:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.736+0000 7f1f9c8dc640 1 -- 172.21.15.29:0/2683823776 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90036040 con 0x7f1f981072f0 2024-09-21T02:00:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.738+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981a4540 con 0x7f1f981072f0 2024-09-21T02:00:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.741+0000 7f1f9c8dc640 1 -- 172.21.15.29:0/2683823776 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1f90044020 con 0x7f1f981072f0 2024-09-21T02:00:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.742+0000 7f1f9c8dc640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 0x7f1f800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.742+0000 7f1f9c8dc640 1 -- 172.21.15.29:0/2683823776 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f9010a2d0 con 0x7f1f981072f0 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.742+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 0x7f1f800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.742+0000 7f1fa4852740 1 Processor -- start 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.742+0000 7f1fa4852740 1 -- start start 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f892f0 0x55af22362c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f914d0 0x55af223637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55af222cc6f0 con 0x55af21fb21b0 2024-09-21T02:00:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55af222bbb20 con 0x55af21f914d0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1fa4852740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55af222bf930 con 0x55af21f892f0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 0x7f1f800b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f880096c0 tx=0x7f1f88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52096/0 (socket says 172.21.15.29:52096) 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3219158451 learned_addr learned my addr 172.21.15.29:0/3219158451 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f967fc640 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f892f0 0x55af22362c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3219158451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f892f0 msgr2=0x55af22362c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f977fe640 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f914d0 0x55af223637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f892f0 0x55af22362c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3219158451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f914d0 msgr2=0x55af223637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f914d0 0x55af223637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3219158451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af22281530 con 0x55af21fb21b0 2024-09-21T02:00:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.743+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800f750 tx=0x7f1f88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df2d21da9e3ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.744+0000 7f1f7dffb640 1 -- 172.21.15.29:0/3219158451 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88005220 con 0x55af21fb21b0 2024-09-21T02:00:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.744+0000 7f1f7dffb640 1 -- 172.21.15.29:0/3219158451 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f88004c50 con 0x55af21fb21b0 2024-09-21T02:00:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.744+0000 7f1f7dffb640 1 -- 172.21.15.29:0/3219158451 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8801cbb0 con 0x55af21fb21b0 2024-09-21T02:00:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- 172.21.15.29:0/3219158451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 msgr2=0x55af22362620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800f750 tx=0x7f1f88014000 comp rx=0 tx=0).stop 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- 172.21.15.29:0/3219158451 shutdown_connections 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f914d0 0x55af223637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f892f0 0x55af22362c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3219158451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22362620 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- 172.21.15.29:0/3219158451 >> 172.21.15.29:0/3219158451 conn(0x55af21fa6300 msgr2=0x55af2235e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- 172.21.15.29:0/3219158451 shutdown_connections 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- 172.21.15.29:0/3219158451 wait complete. 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.745+0000 7f1fa4852740 1 -- start start 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 Processor -- start 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af2236c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55af221f3ab0 con 0x55af21f892f0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55af22261730 con 0x55af21fb21b0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55af22364300 con 0x55af21f914d0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1fa4852740 10 client.0 _ll_register_callbacks cb 0x55af222b56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52112/0 (socket says 172.21.15.29:52112) 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 -- 172.21.15.29:0/2214242808 learned_addr learned my addr 172.21.15.29:0/2214242808 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f967fc640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af2236c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f977fe640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 -- 172.21.15.29:0/2214242808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 msgr2=0x55af2236c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af2236c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 -- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 msgr2=0x55af2236d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f96ffd640 1 -- 172.21.15.29:0/2214242808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2201d790 con 0x55af21f892f0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.746+0000 7f1f967fc640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af2236c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801cfc0 tx=0x7f1f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f44c4db3aa0af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8802d070 con 0x55af21f892f0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f88005000 con 0x55af21f892f0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f88004c50 con 0x55af21f892f0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1f88039050 con 0x55af21f892f0 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1fa4852740 10 client.17931 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.747+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55af22096620 con 0x55af21f892f0 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.749+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f88004c50 con 0x55af21f892f0 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.749+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1f88026be0 con 0x55af21f892f0 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 10 client.17931 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 20 client.17931 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af21f70d80 0x55af22379260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 4 client.17931 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.750+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af222ce640 con 0x55af21f70d80 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.751+0000 7f1f967fc640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af21f70d80 0x55af22379260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.751+0000 7f1f967fc640 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af21f70d80 0x55af22379260 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3351609c79f6baad server_cookie=6ecc93a6b6b29b86 in_seq=0 out_seq=0 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.751+0000 7f1f76ffd640 10 client.17931 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1f76ffd640 1 -- 172.21.15.29:0/2214242808 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af222ce640 con 0x55af21f70d80 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1f76ffd640 10 client.17931 handle_command_reply: tid=0 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 client.17931 shutdown 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 msgr2=0x55af2236bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801cfc0 tx=0x7f1f880046c0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 shutdown_connections 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af21f70d80 0x55af22379260 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af2236c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1f76ffd640 0 client.17931 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 --2- 172.21.15.29:0/2214242808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f892f0 0x55af2236bde0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 >> 172.21.15.29:0/2214242808 conn(0x55af21fa6300 msgr2=0x55af22353ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 shutdown_connections 2024-09-21T02:00:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 1 -- 172.21.15.29:0/2214242808 wait complete. 2024-09-21T02:00:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.752+0000 7f1fa4852740 20 client.17931 trim_cache size 0 max 16384 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 Processor -- start 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 -- start start 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f461c0 0x55af2236a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981a8e00 con 0x55af21f914d0 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981aaa00 con 0x55af21fb21b0 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1fa4852740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981a94d0 con 0x55af21f461c0 2024-09-21T02:00:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f461c0 0x55af2236a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52128/0 (socket says 172.21.15.29:52128) 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.774+0000 7f1f96ffd640 1 -- 172.21.15.29:0/1671602542 learned_addr learned my addr 172.21.15.29:0/1671602542 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f967fc640 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 -- 172.21.15.29:0/1671602542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f461c0 msgr2=0x55af2236a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f461c0 0x55af2236a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 -- 172.21.15.29:0/1671602542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 msgr2=0x55af2236ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 -- 172.21.15.29:0/1671602542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af223641e0 con 0x55af21f914d0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1f88016040 tx=0x7f1f88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5986fbfdf4202e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f7d7fa640 1 -- 172.21.15.29:0/1671602542 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8802d040 con 0x55af21f914d0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f7d7fa640 1 -- 172.21.15.29:0/1671602542 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8800e5a0 con 0x55af21f914d0 2024-09-21T02:00:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.775+0000 7f1f7d7fa640 1 -- 172.21.15.29:0/1671602542 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8801c2f0 con 0x55af21f914d0 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 -- 172.21.15.29:0/1671602542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 msgr2=0x55af223634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1f88016040 tx=0x7f1f88009840 comp rx=0 tx=0).stop 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 -- 172.21.15.29:0/1671602542 shutdown_connections 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f461c0 0x55af2236a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21fb21b0 0x55af2236ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 --2- 172.21.15.29:0/1671602542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21f914d0 0x55af223634b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 -- 172.21.15.29:0/1671602542 >> 172.21.15.29:0/1671602542 conn(0x55af21fa6300 msgr2=0x55af2236b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 -- 172.21.15.29:0/1671602542 shutdown_connections 2024-09-21T02:00:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.776+0000 7f1fa4852740 1 -- 172.21.15.29:0/1671602542 wait complete. 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 -- start start 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 Processor -- start 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 0x55af22371020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af223716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22372250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981aa270 con 0x55af21fb21b0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981a9ba0 con 0x55af21f461c0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981a6de0 con 0x55af21f914d0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1fa4852740 10 client.0 _ll_register_callbacks cb 0x7f1f8c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22372250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1f96ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 0x55af22371020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22372250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52134/0 (socket says 172.21.15.29:52134) 2024-09-21T02:00:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1f977fe640 1 -- 172.21.15.29:0/3236424026 learned_addr learned my addr 172.21.15.29:0/3236424026 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.777+0000 7f1f967fc640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af223716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3236424026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 msgr2=0x55af223716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af223716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3236424026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 msgr2=0x55af22372250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22372250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 -- 172.21.15.29:0/3236424026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af223729b0 con 0x55af21f461c0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f96ffd640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 0x55af22371020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f88016040 tx=0x7f1f880095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4078fe340d0f5fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8802d070 con 0x55af21f461c0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f88004020 con 0x55af21f461c0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8800e5a0 con 0x55af21f461c0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1f88004220 con 0x55af21f461c0 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1fa4852740 10 client.27764 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.778+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55af22373ca0 con 0x55af21f461c0 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8800e5a0 con 0x55af21f461c0 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1f88003680 con 0x55af21f461c0 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 10 client.27764 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 10 client.27764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 20 client.27764 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af223e4090 0x55af223e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 4 client.27764 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af222ca0f0 con 0x55af223e4090 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1f967fc640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af223e4090 0x55af223e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1f76ffd640 10 client.27764 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.779+0000 7f1f967fc640 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af223e4090 0x55af223e64b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=106617a9fb9f279d server_cookie=e3d18266904de0f3 in_seq=0 out_seq=0 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.787+0000 7f1f76ffd640 1 -- 172.21.15.29:0/3236424026 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af222ca0f0 con 0x55af223e4090 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1f76ffd640 10 client.27764 handle_command_reply: tid=0 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 client.27764 shutdown 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 msgr2=0x55af22371020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 0x55af22371020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f88016040 tx=0x7f1f880095e0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 shutdown_connections 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55af223e4090 0x55af223e64b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55af21fb21b0 0x55af22372250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1f76ffd640 0 client.27764 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af21f914d0 0x55af223716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 --2- 172.21.15.29:0/3236424026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af21f461c0 0x55af22371020 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 >> 172.21.15.29:0/3236424026 conn(0x55af21fa6300 msgr2=0x55af2235eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 shutdown_connections 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 1 -- 172.21.15.29:0/3236424026 wait complete. 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.788+0000 7f1fa4852740 20 client.27764 trim_cache size 0 max 16384 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:00:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:00:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:00:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 msgr2=0x7f1f800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 0x7f1f800b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f880096c0 tx=0x7f1f88009260 comp rx=0 tx=0).stop 2024-09-21T02:00:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 msgr2=0x7f1f9819d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 0x7f1f9819d340 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1f9002abe0 tx=0x7f1f90030d20 comp rx=0 tx=0).stop 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 shutdown_connections 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f800b6510 0x7f1f800b8a20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f981072f0 0x7f1f9819d340 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f98106950 0x7f1f9819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 --2- 172.21.15.29:0/2683823776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f981058d0 0x7f1f9819c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 >> 172.21.15.29:0/2683823776 conn(0x7f1f981011f0 msgr2=0x7f1f98109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 shutdown_connections 2024-09-21T02:00:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:21.789+0000 7f1f9d8de640 1 -- 172.21.15.29:0/2683823776 wait complete. 2024-09-21T02:00:22.342 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:22 smithi029 ceph-mon[31228]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-09-21T02:00:22.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:00:22.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:22 smithi190 ceph-mon[34368]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-09-21T02:00:22.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:22 smithi066 ceph-mon[35001]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-09-21T02:00:22.773 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 705.0 s, 1549.6 tps, lat 64.178 ms stddev 70.939 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 -- 172.21.15.29:0/2076126925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c109e00 msgr2=0x7fbb0c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/2076126925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c109e00 0x7fbb0c110630 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ca0 tx=0x7fbb0402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 -- 172.21.15.29:0/2076126925 shutdown_connections 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/2076126925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c109e00 0x7fbb0c110630 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/2076126925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.477+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/2076126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.478+0000 7fbb10ec7640 1 -- 172.21.15.29:0/2076126925 >> 172.21.15.29:0/2076126925 conn(0x7fbb0c07b910 msgr2=0x7fbb0c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.478+0000 7fbb10ec7640 1 -- 172.21.15.29:0/2076126925 shutdown_connections 2024-09-21T02:00:23.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.478+0000 7fbb10ec7640 1 -- 172.21.15.29:0/2076126925 wait complete. 2024-09-21T02:00:23.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.478+0000 7fbb10ec7640 1 Processor -- start 2024-09-21T02:00:23.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 -- start start 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c109e00 0x7fbb0c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113ac0 con 0x7fbb0c1094b0 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113940 con 0x7fbb0c109e00 2024-09-21T02:00:23.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb10ec7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113c40 con 0x7fbb0c103bc0 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51848/0 (socket says 172.21.15.29:51848) 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb03fff640 1 -- 172.21.15.29:0/3850806112 learned_addr learned my addr 172.21.15.29:0/3850806112 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48404/0 (socket says 172.21.15.29:48404) 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.479+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c109e00 0x7fbb0c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 -- 172.21.15.29:0/3850806112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 msgr2=0x7fbb0c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 -- 172.21.15.29:0/3850806112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c109e00 msgr2=0x7fbb0c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c109e00 0x7fbb0c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 -- 172.21.15.29:0/3850806112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c1a03c0 con 0x7fbb0c1094b0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb03fff640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400d740 tx=0x7fbaf400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb037fe640 1 -- 172.21.15.29:0/3850806112 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017070 con 0x7fbb0c1094b0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c1a06b0 con 0x7fbb0c1094b0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb037fe640 1 -- 172.21.15.29:0/3850806112 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf400de50 con 0x7fbb0c1094b0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb037fe640 1 -- 172.21.15.29:0/3850806112 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400f620 con 0x7fbb0c1094b0 2024-09-21T02:00:23.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.480+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a6e10 con 0x7fbb0c1094b0 2024-09-21T02:00:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.484+0000 7fbb1743b740 1 Processor -- start 2024-09-21T02:00:23.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.484+0000 7fbb1743b740 1 -- start start 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a370070 0x559d0a7176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a718120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559d0a68dea0 con 0x559d0a3471b0 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559d0a67d2d0 con 0x559d0a370070 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb1743b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559d0a6810e0 con 0x559d0a34f390 2024-09-21T02:00:23.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51860/0 (socket says 172.21.15.29:51860) 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb0a59c640 1 -- 172.21.15.29:0/3425034258 learned_addr learned my addr 172.21.15.29:0/3425034258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a370070 0x559d0a7176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.485+0000 7fbb037fe640 1 -- 172.21.15.29:0/3850806112 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbaf401a070 con 0x7fbb0c1094b0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 -- 172.21.15.29:0/3425034258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 msgr2=0x559d0a718120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a718120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 -- 172.21.15.29:0/3425034258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a370070 msgr2=0x559d0a7176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a370070 0x559d0a7176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 -- 172.21.15.29:0/3425034258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0a3db650 con 0x559d0a3471b0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0115c0 tx=0x7fbafc011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edfea7b41b7bdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb00ff9640 1 -- 172.21.15.29:0/3425034258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00f5e0 con 0x559d0a3471b0 2024-09-21T02:00:23.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb00ff9640 1 -- 172.21.15.29:0/3425034258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc00fc30 con 0x559d0a3471b0 2024-09-21T02:00:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.486+0000 7fbb00ff9640 1 -- 172.21.15.29:0/3425034258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc019c80 con 0x559d0a3471b0 2024-09-21T02:00:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb037fe640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 0x7fbaf80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb037fe640 1 -- 172.21.15.29:0/3850806112 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbaf40e6c50 con 0x7fbb0c1094b0 2024-09-21T02:00:23.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 0x7fbaf80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 -- 172.21.15.29:0/3425034258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 msgr2=0x559d0a717be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0115c0 tx=0x7fbafc011a80 comp rx=0 tx=0).stop 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 0x7fbaf80b8a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402f500 tx=0x7fbb0402f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 -- 172.21.15.29:0/3425034258 shutdown_connections 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a718120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a3471b0 0x559d0a717be0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 --2- 172.21.15.29:0/3425034258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a370070 0x559d0a7176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.487+0000 7fbb1743b740 1 -- 172.21.15.29:0/3425034258 >> 172.21.15.29:0/3425034258 conn(0x559d0a364460 msgr2=0x559d0a715590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.488+0000 7fbb1743b740 1 -- 172.21.15.29:0/3425034258 shutdown_connections 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.488+0000 7fbb1743b740 1 -- 172.21.15.29:0/3425034258 wait complete. 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.488+0000 7fbb1743b740 1 -- start start 2024-09-21T02:00:23.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.488+0000 7fbb1743b740 1 Processor -- start 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a7276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a733540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559d0a4de700 con 0x559d0a370070 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559d0a618160 con 0x559d0a3471b0 2024-09-21T02:00:23.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559d0a72a1e0 con 0x559d0a34f390 2024-09-21T02:00:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb1743b740 10 client.0 _ll_register_callbacks cb 0x559d0a6319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36904/0 (socket says 172.21.15.29:36904) 2024-09-21T02:00:23.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.490+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1623238461 learned_addr learned my addr 172.21.15.29:0/1623238461 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a7276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1623238461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 msgr2=0x559d0a7276b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a7276b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1623238461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 msgr2=0x559d0a733540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a733540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.491+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1623238461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0a645c90 con 0x559d0a3471b0 2024-09-21T02:00:23.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.499+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ce0 tx=0x7fbafc009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774c8f82299305d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.499+0000 7fbaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:23.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.499+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc026020 con 0x559d0a3471b0 2024-09-21T02:00:23.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.500+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc00fc30 con 0x559d0a3471b0 2024-09-21T02:00:23.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.500+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbafc004350 con 0x559d0a3471b0 2024-09-21T02:00:23.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.500+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbafc00f5e0 con 0x559d0a3471b0 2024-09-21T02:00:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.504+0000 7fbb1743b740 10 client.27776 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.504+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559d0a735f30 con 0x559d0a3471b0 2024-09-21T02:00:23.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.509+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc00fc30 con 0x559d0a3471b0 2024-09-21T02:00:23.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.509+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbafc003870 con 0x559d0a3471b0 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 10 client.27776 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 10 client.27776 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 20 client.27776 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a32e9f0 0x559d0a7392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 4 client.27776 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:23.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.512+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0a68fdf0 con 0x559d0a32e9f0 2024-09-21T02:00:23.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.516+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a32e9f0 0x559d0a7392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.523+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a32e9f0 0x559d0a7392f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a31038ae6fcecc3 server_cookie=58a663a34516d94b in_seq=0 out_seq=0 2024-09-21T02:00:23.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.524+0000 7fbaf1ffb640 10 client.27776 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:00:23.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.525+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1623238461 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d0a68fdf0 con 0x559d0a32e9f0 2024-09-21T02:00:23.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.525+0000 7fbaf1ffb640 10 client.27776 handle_command_reply: tid=0 2024-09-21T02:00:23.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.525+0000 7fbb1743b740 1 client.27776 shutdown 2024-09-21T02:00:23.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 msgr2=0x559d0a727170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ce0 tx=0x7fbafc009db0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 shutdown_connections 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a32e9f0 0x559d0a7392f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a733540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a34f390 0x559d0a7276b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1623238461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a3471b0 0x559d0a727170 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 >> 172.21.15.29:0/1623238461 conn(0x559d0a364460 msgr2=0x559d0a719090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbaf1ffb640 0 client.27776 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:00:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 shutdown_connections 2024-09-21T02:00:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 1 -- 172.21.15.29:0/1623238461 wait complete. 2024-09-21T02:00:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.529+0000 7fbb1743b740 20 client.27776 trim_cache size 0 max 16384 2024-09-21T02:00:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.550+0000 7fbb1743b740 1 Processor -- start 2024-09-21T02:00:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.550+0000 7fbb1743b740 1 -- start start 2024-09-21T02:00:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a733440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a7218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ad240 con 0x559d0a370070 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1aee40 con 0x559d0a34f390 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48444/0 (socket says 172.21.15.29:48444) 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1599446617 learned_addr learned my addr 172.21.15.29:0/1599446617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb1743b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ad910 con 0x559d0a3471b0 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a733440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.551+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a7218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.552+0000 7fbb03fff640 1 -- 172.21.15.29:0/1599446617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 msgr2=0x559d0a7155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.552+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7155c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.552+0000 7fbb03fff640 1 -- 172.21.15.29:0/1599446617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 msgr2=0x559d0a7218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.552+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a7218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.552+0000 7fbb03fff640 1 -- 172.21.15.29:0/1599446617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0a725670 con 0x559d0a34f390 2024-09-21T02:00:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.553+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a733440 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005d00 tx=0x7fbaf4010c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d638974ee1f07ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.553+0000 7fbaf3fff640 1 -- 172.21.15.29:0/1599446617 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4098070 con 0x559d0a34f390 2024-09-21T02:00:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.553+0000 7fbaf3fff640 1 -- 172.21.15.29:0/1599446617 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf40b6810 con 0x559d0a34f390 2024-09-21T02:00:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.553+0000 7fbaf3fff640 1 -- 172.21.15.29:0/1599446617 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4086e60 con 0x559d0a34f390 2024-09-21T02:00:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 -- 172.21.15.29:0/1599446617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 msgr2=0x559d0a733440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a733440 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005d00 tx=0x7fbaf4010c70 comp rx=0 tx=0).stop 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 -- 172.21.15.29:0/1599446617 shutdown_connections 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a7218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a733440 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1599446617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 -- 172.21.15.29:0/1599446617 >> 172.21.15.29:0/1599446617 conn(0x559d0a364460 msgr2=0x559d0a714eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 -- 172.21.15.29:0/1599446617 shutdown_connections 2024-09-21T02:00:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.554+0000 7fbb1743b740 1 -- 172.21.15.29:0/1599446617 wait complete. 2024-09-21T02:00:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.555+0000 7fbb1743b740 1 -- start start 2024-09-21T02:00:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.555+0000 7fbb1743b740 1 Processor -- start 2024-09-21T02:00:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.555+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a72f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a72f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ae6b0 con 0x559d0a370070 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1adfe0 con 0x559d0a34f390 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ab220 con 0x559d0a3471b0 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48452/0 (socket says 172.21.15.29:48452) 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1227486655 learned_addr learned my addr 172.21.15.29:0/1227486655 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb1743b740 10 client.0 _ll_register_callbacks cb 0x7fbb04009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0ad9d640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a72f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a72f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 msgr2=0x559d0a72f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a72f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1227486655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 msgr2=0x559d0a72f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a72f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 -- 172.21.15.29:0/1227486655 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0a72ff20 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbb0a59c640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009250 tx=0x7fbafc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94484c77dc332fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.556+0000 7fbaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc026030 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbafc0035f0 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf40b6810 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbafc003e60 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 10 client.27739 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559d0a71f600 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf40b6810 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbafc034020 con 0x559d0a3471b0 2024-09-21T02:00:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 10 client.27739 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 10 client.27739 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 20 client.27739 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a303da0 0x559d0a7aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 4 client.27739 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.557+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d0a68b8a0 con 0x559d0a303da0 2024-09-21T02:00:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.558+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a303da0 0x559d0a7aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbaf1ffb640 10 client.27739 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:00:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb03fff640 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a303da0 0x559d0a7aa790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39bb75769d4c97eb server_cookie=7e48299aee282195 in_seq=0 out_seq=0 2024-09-21T02:00:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbaf1ffb640 1 -- 172.21.15.29:0/1227486655 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d0a68b8a0 con 0x559d0a303da0 2024-09-21T02:00:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbaf1ffb640 10 client.27739 handle_command_reply: tid=0 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 client.27739 shutdown 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 msgr2=0x559d0a7213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009250 tx=0x7fbafc009ca0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 shutdown_connections 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x559d0a303da0 0x559d0a7aa790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d0a370070 0x559d0a72f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d0a34f390 0x559d0a72f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 --2- 172.21.15.29:0/1227486655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d0a3471b0 0x559d0a7213e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbaf1ffb640 0 client.27739 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.561+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 >> 172.21.15.29:0/1227486655 conn(0x559d0a364460 msgr2=0x559d0a7186d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 shutdown_connections 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb1743b740 1 -- 172.21.15.29:0/1227486655 wait complete. 2024-09-21T02:00:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb1743b740 20 client.27739 trim_cache size 0 max 16384 2024-09-21T02:00:23.565 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 msgr2=0x7fbaf80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 0x7fbaf80b8a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402f500 tx=0x7fbb0402f310 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 msgr2=0x7fbb0c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.562+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400d740 tx=0x7fbaf400dc00 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 shutdown_connections 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbaf80b6510 0x7fbaf80b8a20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb0c109e00 0x7fbb0c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb0c1094b0 0x7fbb0c19a9f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 --2- 172.21.15.29:0/3850806112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c103bc0 0x7fbb0c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.563+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 >> 172.21.15.29:0/3850806112 conn(0x7fbb0c07b910 msgr2=0x7fbb0c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.564+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 shutdown_connections 2024-09-21T02:00:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:23.564+0000 7fbb10ec7640 1 -- 172.21.15.29:0/3850806112 wait complete. 2024-09-21T02:00:24.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:24 smithi029 ceph-mon[31228]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-09-21T02:00:24.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:24 smithi066 ceph-mon[35001]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-09-21T02:00:24.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:24 smithi190 ceph-mon[34368]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-09-21T02:00:24.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:00:25.160 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.753+0000 7f8711e6b640 1 -- 172.21.15.29:0/2276868277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102630 msgr2=0x7f870c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.753+0000 7f8711e6b640 1 --2- 172.21.15.29:0/2276868277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102630 0x7f870c102a40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a480 comp rx=0 tx=0).stop 2024-09-21T02:00:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.754+0000 7f8711e6b640 1 -- 172.21.15.29:0/2276868277 shutdown_connections 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.754+0000 7f8711e6b640 1 --2- 172.21.15.29:0/2276868277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.754+0000 7f8711e6b640 1 --2- 172.21.15.29:0/2276868277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c102f80 0x7f870c10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.754+0000 7f8711e6b640 1 --2- 172.21.15.29:0/2276868277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102630 0x7f870c102a40 secure :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a480 comp rx=0 tx=0).stop 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.754+0000 7f8711e6b640 1 -- 172.21.15.29:0/2276868277 >> 172.21.15.29:0/2276868277 conn(0x7f870c079730 msgr2=0x7f870c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.755+0000 7f8711e6b640 1 -- 172.21.15.29:0/2276868277 shutdown_connections 2024-09-21T02:00:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.755+0000 7f8711e6b640 1 -- 172.21.15.29:0/2276868277 wait complete. 2024-09-21T02:00:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.755+0000 7f8711e6b640 1 Processor -- start 2024-09-21T02:00:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 -- start start 2024-09-21T02:00:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 0x7f870c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 0x7f870c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f870c115c50 con 0x7f870c102f80 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f870c115ad0 con 0x7f870c1a1f20 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f8711e6b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f870c115dd0 con 0x7f870c1101e0 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f870affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 0x7f870c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f870bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 0x7f870c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 0x7f870c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51906/0 (socket says 172.21.15.29:51906) 2024-09-21T02:00:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.756+0000 7f870b7fe640 1 -- 172.21.15.29:0/3755944418 learned_addr learned my addr 172.21.15.29:0/3755944418 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 -- 172.21.15.29:0/3755944418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 msgr2=0x7f870c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 -- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 msgr2=0x7f870c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 0x7f870c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 -- 172.21.15.29:0/3755944418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1a2b00 con 0x7f870c1a1f20 2024-09-21T02:00:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870affd640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.757+0000 7f870bfff640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 0x7f870c1a2330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8704009fd0 tx=0x7f870400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.758+0000 7f8708ff9640 1 -- 172.21.15.29:0/3755944418 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400ce20 con 0x7f870c1a1f20 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.758+0000 7f8708ff9640 1 -- 172.21.15.29:0/3755944418 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704005020 con 0x7f870c1a1f20 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.758+0000 7f8708ff9640 1 -- 172.21.15.29:0/3755944418 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704010690 con 0x7f870c1a1f20 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.758+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1a90a0 con 0x7f870c1a1f20 2024-09-21T02:00:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.758+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a9440 con 0x7f870c1a1f20 2024-09-21T02:00:25.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.761+0000 7f87183df740 1 Processor -- start 2024-09-21T02:00:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.761+0000 7f8708ff9640 1 -- 172.21.15.29:0/3755944418 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f87040054d0 con 0x7f870c1a1f20 2024-09-21T02:00:25.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.762+0000 7f8708ff9640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 0x7f86f80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 -- start start 2024-09-21T02:00:25.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f8708ff9640 1 -- 172.21.15.29:0/3755944418 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f87040e7900 con 0x7f870c1a1f20 2024-09-21T02:00:25.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f870affd640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 0x7f86f80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab522930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5599ab48fea0 con 0x5599ab1491b0 2024-09-21T02:00:25.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ab47f2d0 con 0x5599ab172070 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.763+0000 7f87183df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599ab4830e0 con 0x5599ab151390 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36950/0 (socket says 172.21.15.29:36950) 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 -- 172.21.15.29:0/747809105 learned_addr learned my addr 172.21.15.29:0/747809105 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870bfff640 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab522930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 -- 172.21.15.29:0/747809105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 msgr2=0x5599ab522930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab522930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 -- 172.21.15.29:0/747809105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 msgr2=0x5599ab5223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.764+0000 7f870b7fe640 1 -- 172.21.15.29:0/747809105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ab1dd650 con 0x5599ab172070 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.765+0000 7f870b7fe640 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fdd90529725ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.765+0000 7f8701ffb640 1 -- 172.21.15.29:0/747809105 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4040070 con 0x5599ab172070 2024-09-21T02:00:25.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.765+0000 7f8701ffb640 1 -- 172.21.15.29:0/747809105 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4004030 con 0x5599ab172070 2024-09-21T02:00:25.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.765+0000 7f8701ffb640 1 -- 172.21.15.29:0/747809105 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035040 con 0x5599ab172070 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 -- 172.21.15.29:0/747809105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 msgr2=0x5599ab521eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4031b10 comp rx=0 tx=0).stop 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f870affd640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 0x7f86f80b8bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f870c102310 tx=0x7f86fc009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 -- 172.21.15.29:0/747809105 shutdown_connections 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab522930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 --2- 172.21.15.29:0/747809105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab521eb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 -- 172.21.15.29:0/747809105 >> 172.21.15.29:0/747809105 conn(0x5599ab166460 msgr2=0x5599ab51fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 -- 172.21.15.29:0/747809105 shutdown_connections 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.766+0000 7f87183df740 1 -- 172.21.15.29:0/747809105 wait complete. 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 -- start start 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 Processor -- start 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5599ab2e0700 con 0x5599ab172070 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ab41a160 con 0x5599ab1491b0 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599ab530630 con 0x5599ab151390 2024-09-21T02:00:25.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f87183df740 10 client.0 _ll_register_callbacks cb 0x5599ab4339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.767+0000 7f870bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36960/0 (socket says 172.21.15.29:36960) 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51924/0 (socket says 172.21.15.29:51924) 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 -- 172.21.15.29:0/933440657 learned_addr learned my addr 172.21.15.29:0/933440657 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:25.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870affd640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 -- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 msgr2=0x5599ab52da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 -- 172.21.15.29:0/933440657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 msgr2=0x5599ab52dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.768+0000 7f870b7fe640 1 -- 172.21.15.29:0/933440657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ab447c90 con 0x5599ab1491b0 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f870b7fe640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004440 tx=0x7f86f402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba23d93e268a93b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f86eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:25.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f404c020 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4004030 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f870bfff640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.769+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f86f4005170 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.770+0000 7f870affd640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.770+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f86f4040070 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.770+0000 7f87183df740 10 client.27791 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.770+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5599ab53c2d0 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.770+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4004030 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f86f4004d20 con 0x5599ab1491b0 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 10 client.27791 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 10 client.27791 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 20 client.27791 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab1309f0 0x5599ab53f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 4 client.27791 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:25.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599ab491df0 con 0x5599ab1309f0 2024-09-21T02:00:25.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.771+0000 7f870affd640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab1309f0 0x5599ab53f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.772+0000 7f86eaffd640 10 client.27791 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.772+0000 7f870affd640 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab1309f0 0x5599ab53f630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63c5728960b0e52f server_cookie=265b593e55133090 in_seq=0 out_seq=0 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.773+0000 7f86eaffd640 1 -- 172.21.15.29:0/933440657 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599ab491df0 con 0x5599ab1309f0 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.773+0000 7f86eaffd640 10 client.27791 handle_command_reply: tid=0 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.773+0000 7f87183df740 1 client.27791 shutdown 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 msgr2=0x5599ab52d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004440 tx=0x7f86f402a480 comp rx=0 tx=0).stop 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 shutdown_connections 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab172070 0x5599ab52dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab1309f0 0x5599ab53f630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 --2- 172.21.15.29:0/933440657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab1491b0 0x5599ab52d550 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f86eaffd640 0 client.27791 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 >> 172.21.15.29:0/933440657 conn(0x5599ab166460 msgr2=0x5599ab525040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 shutdown_connections 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 1 -- 172.21.15.29:0/933440657 wait complete. 2024-09-21T02:00:25.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.774+0000 7f87183df740 20 client.27791 trim_cache size 0 max 16384 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 Processor -- start 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 -- start start 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab539020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ad600 con 0x5599ab1491b0 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1af200 con 0x5599ab172070 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f87183df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1adcd0 con 0x5599ab151390 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.793+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51938/0 (socket says 172.21.15.29:51938) 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 -- 172.21.15.29:0/1346092529 learned_addr learned my addr 172.21.15.29:0/1346092529 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870affd640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870bfff640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab539020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 -- 172.21.15.29:0/1346092529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 msgr2=0x5599ab52f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 -- 172.21.15.29:0/1346092529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 msgr2=0x5599ab539020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab539020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 -- 172.21.15.29:0/1346092529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ab5243d0 con 0x5599ab1491b0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870affd640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005850 tx=0x7f86f4031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff1cfaf1959a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f87017fa640 1 -- 172.21.15.29:0/1346092529 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f404c030 con 0x5599ab1491b0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f87017fa640 1 -- 172.21.15.29:0/1346092529 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f40038b0 con 0x5599ab1491b0 2024-09-21T02:00:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.794+0000 7f87017fa640 1 -- 172.21.15.29:0/1346092529 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035040 con 0x5599ab1491b0 2024-09-21T02:00:25.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 -- 172.21.15.29:0/1346092529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 msgr2=0x5599ab5239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005850 tx=0x7f86f4031b10 comp rx=0 tx=0).stop 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 -- 172.21.15.29:0/1346092529 shutdown_connections 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab172070 0x5599ab539020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab151390 0x5599ab52f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 --2- 172.21.15.29:0/1346092529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5239b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.795+0000 7f87183df740 1 -- 172.21.15.29:0/1346092529 >> 172.21.15.29:0/1346092529 conn(0x5599ab166460 msgr2=0x5599ab538910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.797+0000 7f87183df740 1 -- 172.21.15.29:0/1346092529 shutdown_connections 2024-09-21T02:00:25.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.797+0000 7f87183df740 1 -- 172.21.15.29:0/1346092529 wait complete. 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 -- start start 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 Processor -- start 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 0x5599ab540be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1aea70 con 0x5599ab1491b0 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ae3a0 con 0x5599ab151390 2024-09-21T02:00:25.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ab5e0 con 0x5599ab172070 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.798+0000 7f87183df740 10 client.0 _ll_register_callbacks cb 0x7f86fc009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 0x5599ab540be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 0x5599ab540be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:36984/0 (socket says 172.21.15.29:36984) 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51944/0 (socket says 172.21.15.29:51944) 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48500/0 (socket says 172.21.15.29:48500) 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870affd640 1 -- 172.21.15.29:0/1123818753 learned_addr learned my addr 172.21.15.29:0/1123818753 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 -- 172.21.15.29:0/1123818753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 msgr2=0x5599ab540be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 0x5599ab540be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 -- 172.21.15.29:0/1123818753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 msgr2=0x5599ab5406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870bfff640 1 -- 172.21.15.29:0/1123818753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ab5a8e00 con 0x5599ab172070 2024-09-21T02:00:25.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.799+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f870bfff640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f870400c760 tx=0x7f87040ad390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31813c247122c621 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f86eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040ae140 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f87040aceb0 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f40038b0 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f87040b2520 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f87183df740 10 client.27757 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.800+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5599ab5aa080 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f40038b0 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f87040b36b0 con 0x5599ab172070 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 10 client.27757 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 10 client.27757 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 20 client.27757 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:25.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab105da0 0x5599ab5ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 4 client.27757 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:00:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599ab48d8a0 con 0x5599ab105da0 2024-09-21T02:00:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.801+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab105da0 0x5599ab5ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f86eaffd640 10 client.27757 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:00:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f870b7fe640 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab105da0 0x5599ab5ac870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce975ea62d33b262 server_cookie=ac6987b656f65c4c in_seq=0 out_seq=0 2024-09-21T02:00:25.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f86eaffd640 1 -- 172.21.15.29:0/1123818753 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599ab48d8a0 con 0x5599ab105da0 2024-09-21T02:00:25.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f86eaffd640 10 client.27757 handle_command_reply: tid=0 2024-09-21T02:00:25.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 client.27757 shutdown 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 msgr2=0x5599ab541120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f870400c760 tx=0x7f87040ad390 comp rx=0 tx=0).stop 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 shutdown_connections 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x5599ab105da0 0x5599ab5ac870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599ab172070 0x5599ab541120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ab151390 0x5599ab540be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 --2- 172.21.15.29:0/1123818753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5599ab1491b0 0x5599ab5406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f86eaffd640 0 client.27757 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 >> 172.21.15.29:0/1123818753 conn(0x5599ab166460 msgr2=0x5599ab52d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 shutdown_connections 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.802+0000 7f87183df740 1 -- 172.21.15.29:0/1123818753 wait complete. 2024-09-21T02:00:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.803+0000 7f87183df740 20 client.27757 trim_cache size 0 max 16384 2024-09-21T02:00:25.809 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.804+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 msgr2=0x7f86f80b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.804+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 0x7f86f80b8bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f870c102310 tx=0x7f86fc009f90 comp rx=0 tx=0).stop 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.804+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 msgr2=0x7f870c1a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.804+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 0x7f870c1a2330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8704009fd0 tx=0x7f870400ef10 comp rx=0 tx=0).stop 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 shutdown_connections 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f86f80b66e0 0x7f86f80b8bf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f870c1a1f20 0x7f870c1a2330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f870c1101e0 0x7f870c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 --2- 172.21.15.29:0/3755944418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f870c102f80 0x7f870c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 >> 172.21.15.29:0/3755944418 conn(0x7f870c079730 msgr2=0x7f870c1116f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 shutdown_connections 2024-09-21T02:00:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:25.805+0000 7f8711e6b640 1 -- 172.21.15.29:0/3755944418 wait complete. 2024-09-21T02:00:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:26 smithi066 ceph-mon[35001]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-09-21T02:00:26.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:26 smithi029 ceph-mon[31228]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-09-21T02:00:26.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:26 smithi190 ceph-mon[34368]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-09-21T02:00:27.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:00:27.465 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 710.0 s, 1822.4 tps, lat 55.056 ms stddev 69.555 2024-09-21T02:00:28.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:28 smithi029 ceph-mon[31228]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-09-21T02:00:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:28 smithi190 ceph-mon[34368]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-09-21T02:00:28.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:28 smithi066 ceph-mon[35001]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-09-21T02:00:29.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 -- 172.21.15.29:0/3252888850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c111f40 msgr2=0x7f5f7c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/3252888850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c111f40 0x7f5f7c10a3c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009ca0 tx=0x7f5f7402a8e0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 -- 172.21.15.29:0/3252888850 shutdown_connections 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/3252888850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f7c11cc30 0x7f5f7c11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/3252888850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f7c10a9e0 0x7f5f7c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/3252888850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c111f40 0x7f5f7c10a3c0 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009ca0 tx=0x7f5f7402a8e0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.076+0000 7f5f82ec7640 1 -- 172.21.15.29:0/3252888850 >> 172.21.15.29:0/3252888850 conn(0x7f5f7c070ec0 msgr2=0x7f5f7c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.082+0000 7f5f82ec7640 1 -- 172.21.15.29:0/3252888850 shutdown_connections 2024-09-21T02:00:29.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.083+0000 7f5f82ec7640 1 -- 172.21.15.29:0/3252888850 wait complete. 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.084+0000 7f5f82ec7640 1 Processor -- start 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 -- start start 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f7c10a9e0 0x7f5f7c112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f7c11cc30 0x7f5f7c112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c122910 con 0x7f5f7c11cc30 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c122790 con 0x7f5f7c117eb0 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.085+0000 7f5f82ec7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c122a90 con 0x7f5f7c10a9e0 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37004/0 (socket says 172.21.15.29:37004) 2024-09-21T02:00:29.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 -- 172.21.15.29:0/2796163613 learned_addr learned my addr 172.21.15.29:0/2796163613 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 -- 172.21.15.29:0/2796163613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f7c10a9e0 msgr2=0x7f5f7c112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f7c10a9e0 0x7f5f7c112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 -- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f7c11cc30 msgr2=0x7f5f7c112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f7c11cc30 0x7f5f7c112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.088+0000 7f5f81464640 1 -- 172.21.15.29:0/2796163613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c118a00 con 0x7f5f7c117eb0 2024-09-21T02:00:29.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.089+0000 7f5f81464640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c02e980 tx=0x7f5f6c02ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.091+0000 7f5f79ffb640 1 -- 172.21.15.29:0/2796163613 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c007bb0 con 0x7f5f7c117eb0 2024-09-21T02:00:29.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.091+0000 7f5f79ffb640 1 -- 172.21.15.29:0/2796163613 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c007d50 con 0x7f5f7c117eb0 2024-09-21T02:00:29.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.091+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c118cf0 con 0x7f5f7c117eb0 2024-09-21T02:00:29.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.091+0000 7f5f79ffb640 1 -- 172.21.15.29:0/2796163613 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c03a800 con 0x7f5f7c117eb0 2024-09-21T02:00:29.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.091+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c119410 con 0x7f5f7c117eb0 2024-09-21T02:00:29.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.098+0000 7f5f79ffb640 1 -- 172.21.15.29:0/2796163613 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5f6c0433f0 con 0x7f5f7c117eb0 2024-09-21T02:00:29.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 Processor -- start 2024-09-21T02:00:29.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 -- start start 2024-09-21T02:00:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5608c7438ea0 con 0x5608c70f21b0 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5608c74282d0 con 0x5608c70fa390 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.099+0000 7f5f8943b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608c742c0e0 con 0x5608c711b070 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48518/0 (socket says 172.21.15.29:48518) 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 -- 172.21.15.29:0/4001844896 learned_addr learned my addr 172.21.15.29:0/4001844896 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 -- 172.21.15.29:0/4001844896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 msgr2=0x5608c74c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 -- 172.21.15.29:0/4001844896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 msgr2=0x5608c74c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 -- 172.21.15.29:0/4001844896 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c7186650 con 0x5608c711b070 2024-09-21T02:00:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.100+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5f700153a0 tx=0x7f5f70009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577fd366b4223120 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.103+0000 7f5f79ffb640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 0x7f5f640b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.103+0000 7f5f79ffb640 1 -- 172.21.15.29:0/2796163613 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5f6c109a00 con 0x7f5f7c117eb0 2024-09-21T02:00:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.104+0000 7f5f62ffd640 1 -- 172.21.15.29:0/4001844896 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70012040 con 0x5608c711b070 2024-09-21T02:00:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.104+0000 7f5f62ffd640 1 -- 172.21.15.29:0/4001844896 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f700075a0 con 0x5608c711b070 2024-09-21T02:00:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.105+0000 7f5f80c63640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 0x7f5f640b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.106+0000 7f5f8943b740 1 -- 172.21.15.29:0/4001844896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 msgr2=0x5608c74c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.106+0000 7f5f8943b740 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5f700153a0 tx=0x7f5f70009dd0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.107+0000 7f5f8943b740 1 -- 172.21.15.29:0/4001844896 shutdown_connections 2024-09-21T02:00:29.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.107+0000 7f5f8943b740 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.107+0000 7f5f8943b740 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.107+0000 7f5f8943b740 1 --2- 172.21.15.29:0/4001844896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c711b070 0x5608c74c26a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.107+0000 7f5f8943b740 1 -- 172.21.15.29:0/4001844896 >> 172.21.15.29:0/4001844896 conn(0x5608c710f460 msgr2=0x5608c74c0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.108+0000 7f5f8943b740 1 -- 172.21.15.29:0/4001844896 shutdown_connections 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.109+0000 7f5f8943b740 1 -- 172.21.15.29:0/4001844896 wait complete. 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.109+0000 7f5f8943b740 1 -- start start 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.109+0000 7f5f8943b740 1 Processor -- start 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f80c63640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 0x7f5f640b8bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5608c74c1320 tx=0x7f5f74009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5608c7289700 con 0x5608c70f21b0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5608c73c3160 con 0x5608c70d99f0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608c74d0ed0 con 0x5608c70fa390 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.110+0000 7f5f8943b740 10 client.0 _ll_register_callbacks cb 0x5608c73dc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.111+0000 7f5f81464640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.111+0000 7f5f81464640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37024/0 (socket says 172.21.15.29:37024) 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.111+0000 7f5f81464640 1 -- 172.21.15.29:0/2086895973 learned_addr learned my addr 172.21.15.29:0/2086895973 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.111+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.111+0000 7f5f80c63640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.112+0000 7f5f7bfff640 1 -- 172.21.15.29:0/2086895973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 msgr2=0x5608c74ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.112+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.112+0000 7f5f7bfff640 1 -- 172.21.15.29:0/2086895973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 msgr2=0x5608c74d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.112+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.112+0000 7f5f7bfff640 1 -- 172.21.15.29:0/2086895973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c73f0c90 con 0x5608c70fa390 2024-09-21T02:00:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.114+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74cda50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f70015cf0 tx=0x7f5f700016f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798339cfc685eb03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.114+0000 7f5f5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.116+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70000f10 con 0x5608c70fa390 2024-09-21T02:00:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.116+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5f70008b90 con 0x5608c70fa390 2024-09-21T02:00:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.119+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608c743adf0 con 0x5608c70fa390 2024-09-21T02:00:29.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.119+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f70008e10 con 0x5608c70fa390 2024-09-21T02:00:29.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.121+0000 7f5f8943b740 10 client.27763 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:29.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.121+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608c74dc910 con 0x5608c70fa390 2024-09-21T02:00:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.121+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5608c743adf0 con 0x5608c70fa390 2024-09-21T02:00:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f5f700035f0 con 0x5608c70fa390 2024-09-21T02:00:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 10 client.27763 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 10 client.27763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:00:29.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 20 client.27763 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:29.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c70aeda0 0x5608c74dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 4 client.27763 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:29.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608c74368a0 con 0x5608c70aeda0 2024-09-21T02:00:29.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.122+0000 7f5f80c63640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c70aeda0 0x5608c74dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.124+0000 7f5f80c63640 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c70aeda0 0x5608c74dffd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea4e46f948f5b173 server_cookie=582c5d1730775056 in_seq=0 out_seq=0 2024-09-21T02:00:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.124+0000 7f5f5bfff640 10 client.27763 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:00:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.125+0000 7f5f5bfff640 1 -- 172.21.15.29:0/2086895973 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608c74368a0 con 0x5608c70aeda0 2024-09-21T02:00:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.125+0000 7f5f5bfff640 10 client.27763 handle_command_reply: tid=0 2024-09-21T02:00:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.125+0000 7f5f8943b740 1 client.27763 shutdown 2024-09-21T02:00:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.125+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 msgr2=0x5608c74cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.125+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74cda50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f70015cf0 tx=0x7f5f700016f0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 shutdown_connections 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c70aeda0 0x5608c74dffd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74cda50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 --2- 172.21.15.29:0/2086895973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 >> 172.21.15.29:0/2086895973 conn(0x5608c710f460 msgr2=0x5608c74c5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f5bfff640 0 client.27763 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 shutdown_connections 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 1 -- 172.21.15.29:0/2086895973 wait complete. 2024-09-21T02:00:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.126+0000 7f5f8943b740 20 client.27763 trim_cache size 0 max 16384 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.152+0000 7f5f8943b740 1 Processor -- start 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.152+0000 7f5f8943b740 1 -- start start 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1b0170 con 0x5608c70f21b0 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51994/0 (socket says 172.21.15.29:51994) 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f7bfff640 1 -- 172.21.15.29:0/1810435166 learned_addr learned my addr 172.21.15.29:0/1810435166 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1b1d70 con 0x5608c70d99f0 2024-09-21T02:00:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1b0840 con 0x5608c70fa390 2024-09-21T02:00:29.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.153+0000 7f5f80c63640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 -- 172.21.15.29:0/1810435166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 msgr2=0x5608c74c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f81464640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74c0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 -- 172.21.15.29:0/1810435166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 msgr2=0x5608c74bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 -- 172.21.15.29:0/1810435166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c74c4bc0 con 0x5608c70d99f0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f80c63640 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74da110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5f74008000 tx=0x7f5f7403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e0db7d61536532 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f627fc640 1 -- 172.21.15.29:0/1810435166 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74056070 con 0x5608c70d99f0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f627fc640 1 -- 172.21.15.29:0/1810435166 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f74036050 con 0x5608c70d99f0 2024-09-21T02:00:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.154+0000 7f5f627fc640 1 -- 172.21.15.29:0/1810435166 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402d070 con 0x5608c70d99f0 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 msgr2=0x5608c74da110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74da110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5f74008000 tx=0x7f5f7403d000 comp rx=0 tx=0).stop 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 shutdown_connections 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70fa390 0x5608c74c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70f21b0 0x5608c74bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 --2- 172.21.15.29:0/1810435166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70d99f0 0x5608c74da110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 >> 172.21.15.29:0/1810435166 conn(0x5608c710f460 msgr2=0x5608c74cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 shutdown_connections 2024-09-21T02:00:29.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.155+0000 7f5f8943b740 1 -- 172.21.15.29:0/1810435166 wait complete. 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.156+0000 7f5f8943b740 1 -- start start 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.156+0000 7f5f8943b740 1 Processor -- start 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70f21b0 0x5608c74d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f80c63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f80c63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52004/0 (socket says 172.21.15.29:52004) 2024-09-21T02:00:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f80c63640 1 -- 172.21.15.29:0/730518018 learned_addr learned my addr 172.21.15.29:0/730518018 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1b15e0 con 0x5608c70d99f0 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1b0f10 con 0x5608c70fa390 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1ae150 con 0x5608c70f21b0 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.157+0000 7f5f8943b740 10 client.0 _ll_register_callbacks cb 0x7f5f70001920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70f21b0 0x5608c74d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f81464640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f80c63640 1 -- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70f21b0 msgr2=0x5608c74d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f80c63640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70f21b0 0x5608c74d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f80c63640 1 -- 172.21.15.29:0/730518018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 msgr2=0x5608c74d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f80c63640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f80c63640 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c7548b90 con 0x5608c70d99f0 2024-09-21T02:00:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.158+0000 7f5f81464640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.159+0000 7f5f80c63640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5f74030890 tx=0x7f5f74009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9aff60a97cd616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.159+0000 7f5f5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:29.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.159+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74056070 con 0x5608c70d99f0 2024-09-21T02:00:29.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.159+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5f74030c80 con 0x5608c70d99f0 2024-09-21T02:00:29.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.159+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f74036050 con 0x5608c70d99f0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f74030e80 con 0x5608c70d99f0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f8943b740 10 client.17979 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608c7549e10 con 0x5608c70d99f0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f74036050 con 0x5608c70d99f0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f5f74049770 con 0x5608c70d99f0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f8943b740 10 client.17979 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f8943b740 10 client.17979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.160+0000 7f5f8943b740 20 client.17979 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c754a440 0x5608c754c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f8943b740 4 client.17979 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608c7433cd0 con 0x5608c754a440 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c754a440 0x5608c754c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:29.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f7bfff640 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c754a440 0x5608c754c860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1700082baa64421 server_cookie=2c0258edec302f3d in_seq=0 out_seq=0 2024-09-21T02:00:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f5bfff640 10 client.17979 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:00:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f5bfff640 1 -- 172.21.15.29:0/730518018 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608c7433cd0 con 0x5608c754a440 2024-09-21T02:00:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f5bfff640 10 client.17979 handle_command_reply: tid=0 2024-09-21T02:00:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.161+0000 7f5f8943b740 1 client.17979 shutdown 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 msgr2=0x5608c74d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5f74030890 tx=0x7f5f74009a20 comp rx=0 tx=0).stop 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 shutdown_connections 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5608c754a440 0x5608c754c860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f5bfff640 0 client.17979 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:00:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608c70fa390 0x5608c74d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608c70f21b0 0x5608c74d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 --2- 172.21.15.29:0/730518018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5608c70d99f0 0x5608c74d1360 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 >> 172.21.15.29:0/730518018 conn(0x5608c710f460 msgr2=0x5608c74bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.163+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 shutdown_connections 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.164+0000 7f5f8943b740 1 -- 172.21.15.29:0/730518018 wait complete. 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.164+0000 7f5f8943b740 20 client.17979 trim_cache size 0 max 16384 2024-09-21T02:00:29.167 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:00:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 msgr2=0x7f5f640b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 0x7f5f640b8bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5608c74c1320 tx=0x7f5f74009360 comp rx=0 tx=0).stop 2024-09-21T02:00:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 msgr2=0x7f5f7c1182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c02e980 tx=0x7f5f6c02ee40 comp rx=0 tx=0).stop 2024-09-21T02:00:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 shutdown_connections 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f640b66e0 0x7f5f640b8bf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f7c117eb0 0x7f5f7c1182c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f7c11cc30 0x7f5f7c112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 --2- 172.21.15.29:0/2796163613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f7c10a9e0 0x7f5f7c112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 >> 172.21.15.29:0/2796163613 conn(0x7f5f7c070ec0 msgr2=0x7f5f7c11d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 shutdown_connections 2024-09-21T02:00:29.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:29.165+0000 7f5f82ec7640 1 -- 172.21.15.29:0/2796163613 wait complete. 2024-09-21T02:00:30.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:30 smithi029 ceph-mon[31228]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2024-09-21T02:00:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:30 smithi190 ceph-mon[34368]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2024-09-21T02:00:30.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:30 smithi066 ceph-mon[35001]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2024-09-21T02:00:31.021 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:00:31.395 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.961+0000 7fa75713a640 1 -- 172.21.15.29:0/2684703708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 msgr2=0x7fa7501098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.961+0000 7fa75713a640 1 --2- 172.21.15.29:0/2684703708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa7501098c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa744009900 tx=0x7fa74402a410 comp rx=0 tx=0).stop 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 -- 172.21.15.29:0/2684703708 shutdown_connections 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 --2- 172.21.15.29:0/2684703708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750109e00 0x7fa750110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 --2- 172.21.15.29:0/2684703708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa7501098c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 --2- 172.21.15.29:0/2684703708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750103b70 0x7fa750103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 -- 172.21.15.29:0/2684703708 >> 172.21.15.29:0/2684703708 conn(0x7fa7500ff5e0 msgr2=0x7fa750101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 -- 172.21.15.29:0/2684703708 shutdown_connections 2024-09-21T02:00:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.962+0000 7fa75713a640 1 -- 172.21.15.29:0/2684703708 wait complete. 2024-09-21T02:00:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 Processor -- start 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 -- start start 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 0x7fa75019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa75019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 0x7fa75019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113b10 con 0x7fa7501094b0 2024-09-21T02:00:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113990 con 0x7fa750109e00 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa75713a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113c90 con 0x7fa750103b70 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa754ed6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 0x7fa75019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa74ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa75019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 0x7fa75019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 0x7fa75019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51332/0 (socket says 172.21.15.29:51332) 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.963+0000 7fa7556d7640 1 -- 172.21.15.29:0/1606121212 learned_addr learned my addr 172.21.15.29:0/1606121212 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 -- 172.21.15.29:0/1606121212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 msgr2=0x7fa75019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 0x7fa75019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 -- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 msgr2=0x7fa75019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa75019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 -- 172.21.15.29:0/1606121212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7501a0430 con 0x7fa750103b70 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.964+0000 7fa754ed6640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 0x7fa75019a4c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e9f0 tx=0x7fa73c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.965+0000 7fa74dffb640 1 -- 172.21.15.29:0/1606121212 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00ccc0 con 0x7fa750103b70 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.965+0000 7fa74dffb640 1 -- 172.21.15.29:0/1606121212 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa73c0108e0 con 0x7fa750103b70 2024-09-21T02:00:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.965+0000 7fa74dffb640 1 -- 172.21.15.29:0/1606121212 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0188b0 con 0x7fa750103b70 2024-09-21T02:00:31.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.965+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7501a0720 con 0x7fa750103b70 2024-09-21T02:00:31.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.965+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a6e80 con 0x7fa750103b70 2024-09-21T02:00:31.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 Processor -- start 2024-09-21T02:00:31.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 -- start start 2024-09-21T02:00:31.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aeffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bf1ae75ea0 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa74dffb640 1 -- 172.21.15.29:0/1606121212 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa73c010430 con 0x7fa750103b70 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bf1ae652d0 con 0x55bf1ab58070 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.969+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bf1ae690e0 con 0x55bf1ab37390 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51336/0 (socket says 172.21.15.29:51336) 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 -- 172.21.15.29:0/152013049 learned_addr learned my addr 172.21.15.29:0/152013049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa754ed6640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aeffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa74ffff640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 -- 172.21.15.29:0/152013049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 msgr2=0x55bf1af00120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af00120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 -- 172.21.15.29:0/152013049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 msgr2=0x55bf1aeffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:31.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aeffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 -- 172.21.15.29:0/152013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1abc3650 con 0x55bf1ab58070 2024-09-21T02:00:31.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa74ffff640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:00:31.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa7556d7640 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa748011570 tx=0x7fa748011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f4d6ed0ec17bfa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:31.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.970+0000 7fa73affd640 1 -- 172.21.15.29:0/152013049 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748004b20 con 0x55bf1ab58070 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.971+0000 7fa73affd640 1 -- 172.21.15.29:0/152013049 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa748004cc0 con 0x55bf1ab58070 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.971+0000 7fa73affd640 1 -- 172.21.15.29:0/152013049 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748005390 con 0x55bf1ab58070 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.971+0000 7fa74dffb640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 0x7fa7400b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.971+0000 7fa754ed6640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 0x7fa7400b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.971+0000 7fa74dffb640 1 -- 172.21.15.29:0/1606121212 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa73c0e7720 con 0x7fa750103b70 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- 172.21.15.29:0/152013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 msgr2=0x55bf1aeff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa754ed6640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 0x7fa7400b8b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0a9e30 tx=0x7fa73c0ed530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa748011570 tx=0x7fa748011a30 comp rx=0 tx=0).stop 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- 172.21.15.29:0/152013049 shutdown_connections 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aeffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/152013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab58070 0x55bf1aeff6a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- 172.21.15.29:0/152013049 >> 172.21.15.29:0/152013049 conn(0x55bf1ab4c460 msgr2=0x55bf1aefd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- 172.21.15.29:0/152013049 shutdown_connections 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- 172.21.15.29:0/152013049 wait complete. 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 -- start start 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.972+0000 7fa75d6ae740 1 Processor -- start 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab37390 0x55bf1af0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab58070 0x55bf1af0b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bf1acc6700 con 0x55bf1ab37390 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bf1ae00160 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bf1af0ded0 con 0x55bf1ab58070 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa75d6ae740 10 client.0 _ll_register_callbacks cb 0x55bf1ae199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51344/0 (socket says 172.21.15.29:51344) 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 -- 172.21.15.29:0/1079098778 learned_addr learned my addr 172.21.15.29:0/1079098778 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:31.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab58070 0x55bf1af0b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa74ffff640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab37390 0x55bf1af0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 -- 172.21.15.29:0/1079098778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab58070 msgr2=0x55bf1af0b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab58070 0x55bf1af0b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 -- 172.21.15.29:0/1079098778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab37390 msgr2=0x55bf1af0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.973+0000 7fa7556d7640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab37390 0x55bf1af0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.974+0000 7fa7556d7640 1 -- 172.21.15.29:0/1079098778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1ae2dc90 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.975+0000 7fa7556d7640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa748004f90 tx=0x7fa74800fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f2d9c5eba607ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.975+0000 7fa72ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748005340 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa748016040 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa748004cc0 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa748038050 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa75d6ae740 10 client.27827 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:31.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.976+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bf1af19ad0 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa748005990 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa748031070 con 0x55bf1ab2f1b0 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 10 client.27827 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 10 client.27827 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 20 client.27827 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab169f0 0x55bf1af1cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 4 client.27827 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa748004cc0 con 0x55bf1ab169f0 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.977+0000 7fa74ffff640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab169f0 0x55bf1af1cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:31.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.978+0000 7fa72ffff640 10 client.27827 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:00:31.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.978+0000 7fa74ffff640 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab169f0 0x55bf1af1cf40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56dd7b4a721e22bc server_cookie=69c1f67dd3352d80 in_seq=0 out_seq=0 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa72ffff640 1 -- 172.21.15.29:0/1079098778 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa748004cc0 con 0x55bf1ab169f0 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa72ffff640 10 client.27827 handle_command_reply: tid=0 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 client.27827 shutdown 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 msgr2=0x55bf1af0ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa748004f90 tx=0x7fa74800fee0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 shutdown_connections 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab169f0 0x55bf1af1cf40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab58070 0x55bf1af0b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab37390 0x55bf1af0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/1079098778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0ad00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:31.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa72ffff640 0 client.27827 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:00:31.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 >> 172.21.15.29:0/1079098778 conn(0x55bf1ab4c460 msgr2=0x55bf1af02830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:31.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 shutdown_connections 2024-09-21T02:00:31.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 1 -- 172.21.15.29:0/1079098778 wait complete. 2024-09-21T02:00:31.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:31.979+0000 7fa75d6ae740 20 client.27827 trim_cache size 0 max 16384 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 Processor -- start 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 -- start start 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab169f0 0x55bf1af0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aefcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ad210 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501aee10 con 0x55bf1ab169f0 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa75d6ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ad8e0 con 0x55bf1ab37390 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:32.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51762/0 (socket says 172.21.15.29:51762) 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa7556d7640 1 -- 172.21.15.29:0/161815722 learned_addr learned my addr 172.21.15.29:0/161815722 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.004+0000 7fa7556d7640 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab169f0 0x55bf1af0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.005+0000 7fa7556d7640 1 -- 172.21.15.29:0/161815722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab169f0 msgr2=0x55bf1af0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.005+0000 7fa7556d7640 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab169f0 0x55bf1af0c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.005+0000 7fa7556d7640 1 -- 172.21.15.29:0/161815722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 msgr2=0x55bf1aefcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.005+0000 7fa7556d7640 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aefcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.005+0000 7fa7556d7640 1 -- 172.21.15.29:0/161815722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1af0d0d0 con 0x55bf1ab37390 2024-09-21T02:00:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.007+0000 7fa7556d7640 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa748009eb0 tx=0x7fa748011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347c6e90588ca068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.012+0000 7fa73a7fc640 1 -- 172.21.15.29:0/161815722 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748025a60 con 0x55bf1ab37390 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.012+0000 7fa73a7fc640 1 -- 172.21.15.29:0/161815722 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7480057f0 con 0x55bf1ab37390 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.012+0000 7fa73a7fc640 1 -- 172.21.15.29:0/161815722 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748033720 con 0x55bf1ab37390 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 -- 172.21.15.29:0/161815722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 msgr2=0x55bf1aefd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa748009eb0 tx=0x7fa748011a30 comp rx=0 tx=0).stop 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 -- 172.21.15.29:0/161815722 shutdown_connections 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1aefd100 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab2f1b0 0x55bf1aefcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/161815722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab169f0 0x55bf1af0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.014+0000 7fa75d6ae740 1 -- 172.21.15.29:0/161815722 >> 172.21.15.29:0/161815722 conn(0x55bf1ab4c460 msgr2=0x55bf1af01090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:32.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.021+0000 7fa75d6ae740 1 -- 172.21.15.29:0/161815722 shutdown_connections 2024-09-21T02:00:32.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.021+0000 7fa75d6ae740 1 -- 172.21.15.29:0/161815722 wait complete. 2024-09-21T02:00:32.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.022+0000 7fa75d6ae740 1 -- start start 2024-09-21T02:00:32.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.022+0000 7fa75d6ae740 1 Processor -- start 2024-09-21T02:00:32.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.024+0000 7fa75d6ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.024+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:32.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.025+0000 7fa7556d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45020/0 (socket says 172.21.15.29:45020) 2024-09-21T02:00:32.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.025+0000 7fa7556d7640 1 -- 172.21.15.29:0/385596616 learned_addr learned my addr 172.21.15.29:0/385596616 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:32.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.025+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.026+0000 7fa74ffff640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:32.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.026+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af0f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.026+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ae680 con 0x55bf1ab169f0 2024-09-21T02:00:32.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.027+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501adfb0 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.027+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ab1f0 con 0x55bf1ab37390 2024-09-21T02:00:32.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.028+0000 7fa75d6ae740 10 client.0 _ll_register_callbacks cb 0x7fa74800fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:32.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa74ffff640 1 -- 172.21.15.29:0/385596616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 msgr2=0x55bf1af0f0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:32.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa74ffff640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af0f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa74ffff640 1 -- 172.21.15.29:0/385596616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 msgr2=0x55bf1af13680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:32.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa74ffff640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa74ffff640 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1af0f600 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.033+0000 7fa7556d7640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:32.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.036+0000 7fa74ffff640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0eb80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa744009230 tx=0x7fa7440044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd792970653f379 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:32.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.036+0000 7fa72ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:32.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.039+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744048020 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.040+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa744005510 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.041+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7480057f0 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.045+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa744035040 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.045+0000 7fa75d6ae740 10 client.27833 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:32.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.045+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bf1af87440 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.045+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7480057f0 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.046+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa744003960 con 0x55bf1ab2f1b0 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 10 client.27833 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 10 client.27833 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 20 client.27833 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab58070 0x55bf1af89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 4 client.27833 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:00:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.047+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf1ae77df0 con 0x55bf1ab58070 2024-09-21T02:00:32.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.048+0000 7fa7556d7640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab58070 0x55bf1af89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.050+0000 7fa7556d7640 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab58070 0x55bf1af89c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4455066b42df729 server_cookie=bc712d7962ff201b in_seq=0 out_seq=0 2024-09-21T02:00:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.050+0000 7fa72ffff640 10 client.27833 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:00:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.050+0000 7fa72ffff640 1 -- 172.21.15.29:0/385596616 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf1ae77df0 con 0x55bf1ab58070 2024-09-21T02:00:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.050+0000 7fa72ffff640 10 client.27833 handle_command_reply: tid=0 2024-09-21T02:00:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.050+0000 7fa75d6ae740 1 client.27833 shutdown 2024-09-21T02:00:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 msgr2=0x55bf1af0eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0eb80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa744009230 tx=0x7fa7440044b0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 shutdown_connections 2024-09-21T02:00:32.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x55bf1ab58070 0x55bf1af89c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1ab37390 0x55bf1af0f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf1ab2f1b0 0x55bf1af0eb80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 --2- 172.21.15.29:0/385596616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bf1ab169f0 0x55bf1af13680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.051+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 >> 172.21.15.29:0/385596616 conn(0x55bf1ab4c460 msgr2=0x55bf1af00f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.052+0000 7fa72ffff640 0 client.27833 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.052+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 shutdown_connections 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.052+0000 7fa75d6ae740 1 -- 172.21.15.29:0/385596616 wait complete. 2024-09-21T02:00:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.052+0000 7fa75d6ae740 20 client.27833 trim_cache size 0 max 16384 2024-09-21T02:00:32.056 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:00:32.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.056+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 msgr2=0x7fa7400b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:32.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.056+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 0x7fa7400b8b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0a9e30 tx=0x7fa73c0ed530 comp rx=0 tx=0).stop 2024-09-21T02:00:32.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.056+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 msgr2=0x7fa75019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:32.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.056+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 0x7fa75019a4c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e9f0 tx=0x7fa73c00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 shutdown_connections 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa7400b6660 0x7fa7400b8b70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750109e00 0x7fa75019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7501094b0 0x7fa75019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 --2- 172.21.15.29:0/1606121212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa750103b70 0x7fa75019a4c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:32.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.057+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 >> 172.21.15.29:0/1606121212 conn(0x7fa7500ff5e0 msgr2=0x7fa750101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:32.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.058+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 shutdown_connections 2024-09-21T02:00:32.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:32.058+0000 7fa75713a640 1 -- 172.21.15.29:0/1606121212 wait complete. 2024-09-21T02:00:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:32 smithi029 ceph-mon[31228]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-21T02:00:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:32 smithi190 ceph-mon[34368]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-21T02:00:32.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:32 smithi066 ceph-mon[35001]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-21T02:00:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 715.0 s, 2011.4 tps, lat 49.179 ms stddev 55.674 2024-09-21T02:00:33.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:00:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 -- 172.21.15.29:0/3958742476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5810a590 msgr2=0x7fdf5810a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 --2- 172.21.15.29:0/3958742476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5810a590 0x7fdf5810a9a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009be0 tx=0x7fdf5002a6c0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 -- 172.21.15.29:0/3958742476 shutdown_connections 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 --2- 172.21.15.29:0/3958742476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf58116640 0x7fdf58118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 --2- 172.21.15.29:0/3958742476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810aee0 0x7fdf58115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 --2- 172.21.15.29:0/3958742476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5810a590 0x7fdf5810a9a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 -- 172.21.15.29:0/3958742476 >> 172.21.15.29:0/3958742476 conn(0x7fdf58070ec0 msgr2=0x7fdf580712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 -- 172.21.15.29:0/3958742476 shutdown_connections 2024-09-21T02:00:34.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.301+0000 7fdf602d7640 1 -- 172.21.15.29:0/3958742476 wait complete. 2024-09-21T02:00:34.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.302+0000 7fdf602d7640 1 Processor -- start 2024-09-21T02:00:34.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.303+0000 7fdf602d7640 1 -- start start 2024-09-21T02:00:34.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf602d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 0x7fdf581a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 0x7fdf581a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 0x7fdf581a2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45038/0 (socket says 172.21.15.29:45038) 2024-09-21T02:00:34.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf602d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 0x7fdf581a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf5e073640 1 -- 172.21.15.29:0/4204587821 learned_addr learned my addr 172.21.15.29:0/4204587821 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.304+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf58116640 0x7fdf581a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5d872640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 0x7fdf581a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5811c2f0 con 0x7fdf5810a590 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5e874640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf58116640 0x7fdf581a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5811c170 con 0x7fdf5810aee0 2024-09-21T02:00:34.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5811c470 con 0x7fdf58116640 2024-09-21T02:00:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5d872640 1 -- 172.21.15.29:0/4204587821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf58116640 msgr2=0x7fdf581a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5d872640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf58116640 0x7fdf581a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5d872640 1 -- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 msgr2=0x7fdf581a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.305+0000 7fdf5d872640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 0x7fdf581a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.306+0000 7fdf5d872640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf581a8cb0 con 0x7fdf5810aee0 2024-09-21T02:00:34.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.307+0000 7fdf5d872640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 0x7fdf581a32e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf5806c6e0 tx=0x7fdf5400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.308+0000 7fdf4b7fe640 1 -- 172.21.15.29:0/4204587821 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400ccd0 con 0x7fdf5810aee0 2024-09-21T02:00:34.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.308+0000 7fdf4b7fe640 1 -- 172.21.15.29:0/4204587821 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf540035f0 con 0x7fdf5810aee0 2024-09-21T02:00:34.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.309+0000 7fdf4b7fe640 1 -- 172.21.15.29:0/4204587821 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54010690 con 0x7fdf5810aee0 2024-09-21T02:00:34.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.309+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581a8fa0 con 0x7fdf5810aee0 2024-09-21T02:00:34.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.309+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5810bd60 con 0x7fdf5810aee0 2024-09-21T02:00:34.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.315+0000 7fdf4b7fe640 1 -- 172.21.15.29:0/4204587821 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdf54003a80 con 0x7fdf5810aee0 2024-09-21T02:00:34.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.318+0000 7fdf4b7fe640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 0x7fdf440b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.318+0000 7fdf4b7fe640 1 -- 172.21.15.29:0/4204587821 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdf540e7470 con 0x7fdf5810aee0 2024-09-21T02:00:34.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.318+0000 7fdf6684b740 1 Processor -- start 2024-09-21T02:00:34.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.319+0000 7fdf5e073640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 0x7fdf440b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.319+0000 7fdf6684b740 1 -- start start 2024-09-21T02:00:34.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.320+0000 7fdf5e073640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 0x7fdf440b8b20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdf50000f80 tx=0x7fdf50004a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.321+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.321+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e006a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.321+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.321+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51810/0 (socket says 172.21.15.29:51810) 2024-09-21T02:00:34.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf5e073640 1 -- 172.21.15.29:0/2770799048 learned_addr learned my addr 172.21.15.29:0/2770799048 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf6684b740 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfca2390 0x5606e006ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5606dffe0ea0 con 0x5606dfc9a1b0 2024-09-21T02:00:34.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5606dffd02d0 con 0x5606dfca2390 2024-09-21T02:00:34.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5606dffd40e0 con 0x5606dfcc3070 2024-09-21T02:00:34.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.322+0000 7fdf5d872640 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e006a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 -- 172.21.15.29:0/2770799048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfca2390 msgr2=0x5606e006ad40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfca2390 0x5606e006ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 -- 172.21.15.29:0/2770799048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 msgr2=0x5606e006a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e006a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 -- 172.21.15.29:0/2770799048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dfd2e650 con 0x5606dfcc3070 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5d872640 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e006a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:34.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.323+0000 7fdf5e073640 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006010 tx=0x7fdf5003d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eeb06e4b4799bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.329+0000 7fdf48d9a640 1 -- 172.21.15.29:0/2770799048 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50036040 con 0x5606dfcc3070 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.329+0000 7fdf48d9a640 1 -- 172.21.15.29:0/2770799048 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5004e070 con 0x5606dfcc3070 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.329+0000 7fdf48d9a640 1 -- 172.21.15.29:0/2770799048 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50004820 con 0x5606dfcc3070 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 msgr2=0x5606e006a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006010 tx=0x7fdf5003d000 comp rx=0 tx=0).stop 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 shutdown_connections 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfca2390 0x5606e006ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e006a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 --2- 172.21.15.29:0/2770799048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfcc3070 0x5606e006a2c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.330+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 >> 172.21.15.29:0/2770799048 conn(0x5606dfcb7460 msgr2=0x5606e00681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.334 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:34 smithi029 ceph-mon[31228]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-09-21T02:00:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.334+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 shutdown_connections 2024-09-21T02:00:34.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.336+0000 7fdf6684b740 1 -- 172.21.15.29:0/2770799048 wait complete. 2024-09-21T02:00:34.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.337+0000 7fdf6684b740 1 -- start start 2024-09-21T02:00:34.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.338+0000 7fdf6684b740 1 Processor -- start 2024-09-21T02:00:34.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.338+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0075a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e0075f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf6684b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5606dfe31700 con 0x5606dfc9a1b0 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf6684b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5606dff6b160 con 0x5606dfcc3070 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf5e874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf5e874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51394/0 (socket says 172.21.15.29:51394) 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf5e874640 1 -- 172.21.15.29:0/3797797209 learned_addr learned my addr 172.21.15.29:0/3797797209 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.339+0000 7fdf6684b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5606e0078a40 con 0x5606dfca2390 2024-09-21T02:00:34.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.340+0000 7fdf5d872640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e0075f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.340+0000 7fdf6684b740 10 client.0 _ll_register_callbacks cb 0x5606dff849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:34.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.343+0000 7fdf5e073640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.344+0000 7fdf5e874640 1 -- 172.21.15.29:0/3797797209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 msgr2=0x5606e0075f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.344+0000 7fdf5e874640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e0075f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.344+0000 7fdf5e874640 1 -- 172.21.15.29:0/3797797209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 msgr2=0x5606e0075a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.344+0000 7fdf5e874640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0075a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.344+0000 7fdf5e874640 1 -- 172.21.15.29:0/3797797209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dff98c90 con 0x5606dfcc3070 2024-09-21T02:00:34.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.347+0000 7fdf5e874640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0116f0 tx=0x7fdf4c011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7498caa063dc73bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.347+0000 7fdf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c01e060 con 0x5606dfcc3070 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5004e070 con 0x5606dfcc3070 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdf4c019430 con 0x5606dfcc3070 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdf4c019630 con 0x5606dfcc3070 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf6684b740 10 client.27842 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.349+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5606e0084860 con 0x5606dfcc3070 2024-09-21T02:00:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.350+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5004e070 con 0x5606dfcc3070 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fdf4c011ce0 con 0x5606dfcc3070 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 10 client.27842 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 10 client.27842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 20 client.27842 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc819f0 0x5606e0088230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 4 client.27842 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.351+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606dffe2df0 con 0x5606dfc819f0 2024-09-21T02:00:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.355+0000 7fdf5e073640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc819f0 0x5606e0088230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.366+0000 7fdf5e073640 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc819f0 0x5606e0088230 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de74cb0411a9717a server_cookie=47902887f46a9f51 in_seq=0 out_seq=0 2024-09-21T02:00:34.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.367+0000 7fdf41ffb640 10 client.27842 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.371+0000 7fdf41ffb640 1 -- 172.21.15.29:0/3797797209 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606dffe2df0 con 0x5606dfc819f0 2024-09-21T02:00:34.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.371+0000 7fdf41ffb640 10 client.27842 handle_command_reply: tid=0 2024-09-21T02:00:34.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.371+0000 7fdf6684b740 1 client.27842 shutdown 2024-09-21T02:00:34.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.375+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 msgr2=0x5606e0081d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.375+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0116f0 tx=0x7fdf4c011bb0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.377+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 shutdown_connections 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.377+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc819f0 0x5606e0088230 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081d60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 --2- 172.21.15.29:0/3797797209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0075a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf41ffb640 0 client.27842 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 >> 172.21.15.29:0/3797797209 conn(0x5606dfcb7460 msgr2=0x5606e006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 shutdown_connections 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 1 -- 172.21.15.29:0/3797797209 wait complete. 2024-09-21T02:00:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.378+0000 7fdf6684b740 20 client.27842 trim_cache size 0 max 16384 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 Processor -- start 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 -- start start 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0074910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b5a10 con 0x5606dfc9a1b0 2024-09-21T02:00:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b7610 con 0x5606dfcc3070 2024-09-21T02:00:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.409+0000 7fdf6684b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b60e0 con 0x5606dfca2390 2024-09-21T02:00:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51834/0 (socket says 172.21.15.29:51834) 2024-09-21T02:00:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 -- 172.21.15.29:0/1536356319 learned_addr learned my addr 172.21.15.29:0/1536356319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5e874640 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 -- 172.21.15.29:0/1536356319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 msgr2=0x5606e0081c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 -- 172.21.15.29:0/1536356319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 msgr2=0x5606e0074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.410+0000 7fdf5d872640 1 -- 172.21.15.29:0/1536356319 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e00623b0 con 0x5606dfca2390 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.411+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf540aa820 tx=0x7fdf54003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aace4ff6f2913ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.411+0000 7fdf43fff640 1 -- 172.21.15.29:0/1536356319 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ad0c0 con 0x5606dfca2390 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.412+0000 7fdf43fff640 1 -- 172.21.15.29:0/1536356319 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf540d4070 con 0x5606dfca2390 2024-09-21T02:00:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.412+0000 7fdf43fff640 1 -- 172.21.15.29:0/1536356319 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540b6030 con 0x5606dfca2390 2024-09-21T02:00:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.415+0000 7fdf6684b740 1 -- 172.21.15.29:0/1536356319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 msgr2=0x5606e00682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.415+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf540aa820 tx=0x7fdf54003e20 comp rx=0 tx=0).stop 2024-09-21T02:00:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.415+0000 7fdf6684b740 1 -- 172.21.15.29:0/1536356319 shutdown_connections 2024-09-21T02:00:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.415+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e0081c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.416+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfca2390 0x5606e00682b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.416+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1536356319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfc9a1b0 0x5606e0074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.416+0000 7fdf6684b740 1 -- 172.21.15.29:0/1536356319 >> 172.21.15.29:0/1536356319 conn(0x5606dfcb7460 msgr2=0x5606e0067ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.417+0000 7fdf6684b740 1 -- 172.21.15.29:0/1536356319 shutdown_connections 2024-09-21T02:00:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.417+0000 7fdf6684b740 1 -- 172.21.15.29:0/1536356319 wait complete. 2024-09-21T02:00:34.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.418+0000 7fdf6684b740 1 -- start start 2024-09-21T02:00:34.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.418+0000 7fdf6684b740 1 Processor -- start 2024-09-21T02:00:34.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.419+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 0x5606e007e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.419+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.419+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 0x5606e007e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.420+0000 7fdf6684b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e007a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.420+0000 7fdf5e073640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 0x5606e007e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51846/0 (socket says 172.21.15.29:51846) 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.420+0000 7fdf5d872640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.420+0000 7fdf5d872640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45084/0 (socket says 172.21.15.29:45084) 2024-09-21T02:00:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.420+0000 7fdf5e073640 1 -- 172.21.15.29:0/1816547272 learned_addr learned my addr 172.21.15.29:0/1816547272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.421+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b6e80 con 0x5606dfca2390 2024-09-21T02:00:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.421+0000 7fdf5e874640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e007a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.421+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b67b0 con 0x5606dfcc3070 2024-09-21T02:00:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.421+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b39f0 con 0x5606dfc9a1b0 2024-09-21T02:00:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.421+0000 7fdf6684b740 10 client.0 _ll_register_callbacks cb 0x7fdf4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.422+0000 7fdf5d872640 1 -- 172.21.15.29:0/1816547272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 msgr2=0x5606e007e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.422+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 0x5606e007e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.422+0000 7fdf5d872640 1 -- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 msgr2=0x5606e007a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.422+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e007a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.422+0000 7fdf5d872640 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e007a810 con 0x5606dfca2390 2024-09-21T02:00:34.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.423+0000 7fdf5d872640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdf540aa820 tx=0x7fdf540b6270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9281322677939a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.423+0000 7fdf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:34.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.423+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ad0c0 con 0x5606dfca2390 2024-09-21T02:00:34.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.424+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fdf540cf290 con 0x5606dfca2390 2024-09-21T02:00:34.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.424+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf540d4070 con 0x5606dfca2390 2024-09-21T02:00:34.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.424+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdf540b2090 con 0x5606dfca2390 2024-09-21T02:00:34.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.424+0000 7fdf6684b740 10 client.18009 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:34.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.424+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5606e00f2c40 con 0x5606dfca2390 2024-09-21T02:00:34.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.425+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf540d4070 con 0x5606dfca2390 2024-09-21T02:00:34.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.425+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fdf540b6510 con 0x5606dfca2390 2024-09-21T02:00:34.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.426+0000 7fdf6684b740 10 client.18009 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:34.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.427+0000 7fdf6684b740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:34.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.427+0000 7fdf6684b740 20 client.18009 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:34.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.427+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc56da0 0x5606e0106420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.427+0000 7fdf6684b740 4 client.18009 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:00:34.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.427+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606dffde8a0 con 0x5606dfc56da0 2024-09-21T02:00:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.430+0000 7fdf5e073640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc56da0 0x5606e0106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.436+0000 7fdf41ffb640 10 client.18009 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.436+0000 7fdf5e073640 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc56da0 0x5606e0106420 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654bccb311aa74e4 server_cookie=7527e41b4743c2de in_seq=0 out_seq=0 2024-09-21T02:00:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf41ffb640 1 -- 172.21.15.29:0/1816547272 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606dffde8a0 con 0x5606dfc56da0 2024-09-21T02:00:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf41ffb640 10 client.18009 handle_command_reply: tid=0 2024-09-21T02:00:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 client.18009 shutdown 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 msgr2=0x5606e0079a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdf540aa820 tx=0x7fdf540b6270 comp rx=0 tx=0).stop 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 shutdown_connections 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5606dfc56da0 0x5606e0106420 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5606dfcc3070 0x5606e007a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5606dfca2390 0x5606e0079a50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 --2- 172.21.15.29:0/1816547272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606dfc9a1b0 0x5606e007e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 >> 172.21.15.29:0/1816547272 conn(0x5606dfcb7460 msgr2=0x5606e0067970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf41ffb640 0 client.18009 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 shutdown_connections 2024-09-21T02:00:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 1 -- 172.21.15.29:0/1816547272 wait complete. 2024-09-21T02:00:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.437+0000 7fdf6684b740 20 client.18009 trim_cache size 0 max 16384 2024-09-21T02:00:34.441 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:00:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.438+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 msgr2=0x7fdf440b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.438+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 0x7fdf440b8b20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdf50000f80 tx=0x7fdf50004a60 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.438+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 msgr2=0x7fdf581a32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.438+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 0x7fdf581a32e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf5806c6e0 tx=0x7fdf5400ee50 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.438+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 shutdown_connections 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fdf440b6610 0x7fdf440b8b20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf58116640 0x7fdf581a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5810aee0 0x7fdf581a32e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 --2- 172.21.15.29:0/4204587821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf5810a590 0x7fdf581a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 >> 172.21.15.29:0/4204587821 conn(0x7fdf58070ec0 msgr2=0x7fdf58073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 shutdown_connections 2024-09-21T02:00:34.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:34.439+0000 7fdf602d7640 1 -- 172.21.15.29:0/4204587821 wait complete. 2024-09-21T02:00:34.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:34 smithi190 ceph-mon[34368]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-09-21T02:00:34.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:34 smithi066 ceph-mon[35001]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-09-21T02:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.932+0000 7fc26b21d640 1 -- 172.21.15.29:0/2921020350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410a3c0 msgr2=0x7fc26410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.932+0000 7fc26b21d640 1 --2- 172.21.15.29:0/2921020350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410a3c0 0x7fc26410a7d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc254009860 tx=0x7fc25402a340 comp rx=0 tx=0).stop 2024-09-21T02:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.932+0000 7fc26b21d640 1 -- 172.21.15.29:0/2921020350 shutdown_connections 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.932+0000 7fc26b21d640 1 --2- 172.21.15.29:0/2921020350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc264111f90 0x7fc2641143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.932+0000 7fc26b21d640 1 --2- 172.21.15.29:0/2921020350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26410ada0 0x7fc26410b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.933+0000 7fc26b21d640 1 --2- 172.21.15.29:0/2921020350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410a3c0 0x7fc26410a7d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.933+0000 7fc26b21d640 1 -- 172.21.15.29:0/2921020350 >> 172.21.15.29:0/2921020350 conn(0x7fc264070810 msgr2=0x7fc264072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.933+0000 7fc26b21d640 1 -- 172.21.15.29:0/2921020350 shutdown_connections 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.933+0000 7fc26b21d640 1 -- 172.21.15.29:0/2921020350 wait complete. 2024-09-21T02:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.934+0000 7fc26b21d640 1 Processor -- start 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.934+0000 7fc26b21d640 1 -- start start 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 0x7fc26410d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410ada0 0x7fc26410be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 0x7fc26410c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc264117ab0 con 0x7fc26410a3c0 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc264117930 con 0x7fc264111f90 2024-09-21T02:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc26b21d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc264117c30 con 0x7fc26410ada0 2024-09-21T02:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 0x7fc26410d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 0x7fc26410d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45106/0 (socket says 172.21.15.29:45106) 2024-09-21T02:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc268fb9640 1 -- 172.21.15.29:0/3941676843 learned_addr learned my addr 172.21.15.29:0/3941676843 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 0x7fc26410c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.935+0000 7fc263fff640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410ada0 0x7fc26410be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 -- 172.21.15.29:0/3941676843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410ada0 msgr2=0x7fc26410be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410ada0 0x7fc26410be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 -- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 msgr2=0x7fc26410d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 0x7fc26410d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 -- 172.21.15.29:0/3941676843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641a65d0 con 0x7fc264111f90 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 0x7fc26410c3c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00efc0 tx=0x7fc25c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.936+0000 7fc261ffb640 1 -- 172.21.15.29:0/3941676843 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c003970 con 0x7fc264111f90 2024-09-21T02:00:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.937+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26410ca20 con 0x7fc264111f90 2024-09-21T02:00:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.937+0000 7fc261ffb640 1 -- 172.21.15.29:0/3941676843 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c004020 con 0x7fc264111f90 2024-09-21T02:00:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.937+0000 7fc261ffb640 1 -- 172.21.15.29:0/3941676843 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00b940 con 0x7fc264111f90 2024-09-21T02:00:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.937+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641b11d0 con 0x7fc264111f90 2024-09-21T02:00:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.941+0000 7fc272264740 1 Processor -- start 2024-09-21T02:00:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.941+0000 7fc261ffb640 1 -- 172.21.15.29:0/3941676843 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc25c01d020 con 0x7fc264111f90 2024-09-21T02:00:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.941+0000 7fc272264740 1 -- start start 2024-09-21T02:00:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.942+0000 7fc261ffb640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 0x7fc24c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.943+0000 7fc261ffb640 1 -- 172.21.15.29:0/3941676843 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc25c0e71c0 con 0x7fc264111f90 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.944+0000 7fc263fff640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 0x7fc24c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 0x55b10e966e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5962f0 0x55b10e9674c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b10e8d96f0 con 0x55b10e5962f0 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b10e8c8b20 con 0x55b10e5bf1b0 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc272264740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b10e8cc930 con 0x55b10e59e4d0 2024-09-21T02:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc263fff640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 0x7fc24c0b8b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc258009bd0 tx=0x7fc258005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 0x55b10e966e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc263fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5962f0 0x55b10e9674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51870/0 (socket says 172.21.15.29:51870) 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 0x55b10e966e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51438/0 (socket says 172.21.15.29:51438) 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 -- 172.21.15.29:0/4186451177 learned_addr learned my addr 172.21.15.29:0/4186451177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 -- 172.21.15.29:0/4186451177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 msgr2=0x55b10e966e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 0x55b10e966e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 -- 172.21.15.29:0/4186451177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5962f0 msgr2=0x55b10e9674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5962f0 0x55b10e9674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 -- 172.21.15.29:0/4186451177 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10e88e530 con 0x55b10e59e4d0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.945+0000 7fc2697ba640 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0a98d0 tx=0x7fc25c0acd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77671a0fc6257b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.946+0000 7fc24affd640 1 -- 172.21.15.29:0/4186451177 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0ada00 con 0x55b10e59e4d0 2024-09-21T02:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.946+0000 7fc24affd640 1 -- 172.21.15.29:0/4186451177 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c0b1de0 con 0x55b10e59e4d0 2024-09-21T02:00:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.946+0000 7fc24affd640 1 -- 172.21.15.29:0/4186451177 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0a8c50 con 0x55b10e59e4d0 2024-09-21T02:00:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- 172.21.15.29:0/4186451177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 msgr2=0x55b10e968010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0a98d0 tx=0x7fc25c0acd10 comp rx=0 tx=0).stop 2024-09-21T02:00:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- 172.21.15.29:0/4186451177 shutdown_connections 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e968010 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5962f0 0x55b10e9674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 --2- 172.21.15.29:0/4186451177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5bf1b0 0x55b10e966e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- 172.21.15.29:0/4186451177 >> 172.21.15.29:0/4186451177 conn(0x55b10e5b3300 msgr2=0x55b10e9628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- 172.21.15.29:0/4186451177 shutdown_connections 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- 172.21.15.29:0/4186451177 wait complete. 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 -- start start 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.947+0000 7fc272264740 1 Processor -- start 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e59e4d0 0x55b10e970c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e9717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b10e800ab0 con 0x55b10e5bf1b0 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc263fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e59e4d0 0x55b10e970c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51872/0 (socket says 172.21.15.29:51872) 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc268fb9640 1 -- 172.21.15.29:0/2494568165 learned_addr learned my addr 172.21.15.29:0/2494568165 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b10e86e730 con 0x55b10e59e4d0 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b10e968b30 con 0x55b10e5962f0 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc2697ba640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e9717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.948+0000 7fc272264740 10 client.0 _ll_register_callbacks cb 0x55b10e8c26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.949+0000 7fc268fb9640 1 -- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e59e4d0 msgr2=0x55b10e970c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.949+0000 7fc268fb9640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e59e4d0 0x55b10e970c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.949+0000 7fc268fb9640 1 -- 172.21.15.29:0/2494568165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 msgr2=0x55b10e9717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.949+0000 7fc268fb9640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e9717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.949+0000 7fc268fb9640 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10e62a790 con 0x55b10e5962f0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc268fb9640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc254009980 tx=0x7fc254030dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23715e3cc1fedaa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc243fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254042070 con 0x55b10e5962f0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc254030810 con 0x55b10e5962f0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25c0b1de0 con 0x55b10e5962f0 2024-09-21T02:00:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.950+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc254037040 con 0x55b10e5962f0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc272264740 10 client.27811 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b10e6a3620 con 0x55b10e5962f0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25404e070 con 0x55b10e5962f0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fc254003c60 con 0x55b10e5962f0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc272264740 10 client.27811 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.951+0000 7fc272264740 10 client.27811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.952+0000 7fc272264740 20 client.27811 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.952+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e57dd80 0x55b10e97df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.952+0000 7fc272264740 4 client.27811 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.952+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10e8db640 con 0x55b10e57dd80 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.952+0000 7fc263fff640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e57dd80 0x55b10e97df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.953+0000 7fc263fff640 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e57dd80 0x55b10e97df20 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c788945c9168d08 server_cookie=a27bfba2e99d9854 in_seq=0 out_seq=0 2024-09-21T02:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.953+0000 7fc243fff640 10 client.27811 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.953+0000 7fc243fff640 1 -- 172.21.15.29:0/2494568165 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b10e8db640 con 0x55b10e57dd80 2024-09-21T02:00:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.953+0000 7fc243fff640 10 client.27811 handle_command_reply: tid=0 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 client.27811 shutdown 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 msgr2=0x55b10e9705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc254009980 tx=0x7fc254030dd0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 shutdown_connections 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e57dd80 0x55b10e97df20 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e9717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e59e4d0 0x55b10e970c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 --2- 172.21.15.29:0/2494568165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e5962f0 0x55b10e9705c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 >> 172.21.15.29:0/2494568165 conn(0x55b10e5b3300 msgr2=0x55b10e960ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc243fff640 0 client.27811 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 shutdown_connections 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 1 -- 172.21.15.29:0/2494568165 wait complete. 2024-09-21T02:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.954+0000 7fc272264740 20 client.27811 trim_cache size 0 max 16384 2024-09-21T02:00:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.995+0000 7fc272264740 1 Processor -- start 2024-09-21T02:00:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.996+0000 7fc272264740 1 -- start start 2024-09-21T02:00:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc272264740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc272264740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc272264740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e961650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51454/0 (socket says 172.21.15.29:51454) 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc263fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc263fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51882/0 (socket says 172.21.15.29:51882) 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.997+0000 7fc268fb9640 1 -- 172.21.15.29:0/3834293071 learned_addr learned my addr 172.21.15.29:0/3834293071 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.998+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641b9d40 con 0x55b10e5bf1b0 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.998+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641bb940 con 0x55b10e5962f0 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.998+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ba410 con 0x55b10e59e4d0 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.998+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e961650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.998+0000 7fc2697ba640 1 -- 172.21.15.29:0/3834293071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 msgr2=0x55b10e9786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.999+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.999+0000 7fc2697ba640 1 -- 172.21.15.29:0/3834293071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 msgr2=0x55b10e96f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.999+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.999+0000 7fc2697ba640 1 -- 172.21.15.29:0/3834293071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10e57dfb0 con 0x55b10e5bf1b0 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:34.999+0000 7fc2697ba640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e961650 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0acfb0 tx=0x7fc25c0a9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129403079c560b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.000+0000 7fc24a7fc640 1 -- 172.21.15.29:0/3834293071 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0ada00 con 0x55b10e5bf1b0 2024-09-21T02:00:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.000+0000 7fc268fb9640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.000+0000 7fc263fff640 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.000+0000 7fc24a7fc640 1 -- 172.21.15.29:0/3834293071 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c0b6d20 con 0x55b10e5bf1b0 2024-09-21T02:00:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.001+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 msgr2=0x55b10e961650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.001+0000 7fc272264740 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e961650 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0acfb0 tx=0x7fc25c0a9a40 comp rx=0 tx=0).stop 2024-09-21T02:00:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.002+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 shutdown_connections 2024-09-21T02:00:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.002+0000 7fc272264740 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e961650 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.002+0000 7fc272264740 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e9786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.002+0000 7fc272264740 1 --2- 172.21.15.29:0/3834293071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e96f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.002+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 >> 172.21.15.29:0/3834293071 conn(0x55b10e5b3300 msgr2=0x55b10e971d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.008+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 shutdown_connections 2024-09-21T02:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.014+0000 7fc272264740 1 -- 172.21.15.29:0/3834293071 wait complete. 2024-09-21T02:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.014+0000 7fc272264740 1 -- start start 2024-09-21T02:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.014+0000 7fc272264740 1 Processor -- start 2024-09-21T02:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e975870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e976aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641bb1b0 con 0x55b10e5bf1b0 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641baae0 con 0x55b10e5962f0 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641b7d20 con 0x55b10e59e4d0 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc272264740 10 client.0 _ll_register_callbacks cb 0x7fc258005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.015+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e975870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc2697ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e976aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc263fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc2697ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e976aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45162/0 (socket says 172.21.15.29:45162) 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc2697ba640 1 -- 172.21.15.29:0/4276670461 learned_addr learned my addr 172.21.15.29:0/4276670461 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc263fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51898/0 (socket says 172.21.15.29:51898) 2024-09-21T02:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.019+0000 7fc268fb9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e975870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51462/0 (socket says 172.21.15.29:51462) 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.021+0000 7fc263fff640 1 -- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 msgr2=0x55b10e975870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.021+0000 7fc263fff640 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e975870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.021+0000 7fc263fff640 1 -- 172.21.15.29:0/4276670461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 msgr2=0x55b10e976aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.021+0000 7fc263fff640 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e976aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.021+0000 7fc263fff640 1 -- 172.21.15.29:0/4276670461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10e977200 con 0x55b10e59e4d0 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.022+0000 7fc263fff640 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b10e97e650 tx=0x7fc258004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856fe3aa53da218e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.022+0000 7fc243fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.027+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258005020 con 0x55b10e59e4d0 2024-09-21T02:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.027+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25c0b6d20 con 0x55b10e59e4d0 2024-09-21T02:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.027+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc2580051c0 con 0x55b10e59e4d0 2024-09-21T02:00:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.033+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc2580053c0 con 0x55b10e59e4d0 2024-09-21T02:00:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.033+0000 7fc272264740 10 client.27814 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.034+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b10e9784d0 con 0x55b10e59e4d0 2024-09-21T02:00:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.035+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c0b6d20 con 0x55b10e59e4d0 2024-09-21T02:00:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fc258030750 con 0x55b10e59e4d0 2024-09-21T02:00:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 10 client.27814 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 10 client.27814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 20 client.27814 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e5531c0 0x55b10e9eac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 4 client.27814 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:00:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.036+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b10e8d70f0 con 0x55b10e5531c0 2024-09-21T02:00:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.038+0000 7fc268fb9640 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e5531c0 0x55b10e9eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.043+0000 7fc268fb9640 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e5531c0 0x55b10e9eac30 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74dec169c55ee6c4 server_cookie=d6f7d38574e3e215 in_seq=0 out_seq=0 2024-09-21T02:00:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.043+0000 7fc243fff640 10 client.27814 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.069+0000 7fc243fff640 1 -- 172.21.15.29:0/4276670461 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b10e8d70f0 con 0x55b10e5531c0 2024-09-21T02:00:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.070+0000 7fc243fff640 10 client.27814 handle_command_reply: tid=0 2024-09-21T02:00:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.070+0000 7fc272264740 1 client.27814 shutdown 2024-09-21T02:00:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.070+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 msgr2=0x55b10e975f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.070+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b10e97e650 tx=0x7fc258004730 comp rx=0 tx=0).stop 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 shutdown_connections 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b10e5531c0 0x55b10e9eac30 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b10e5bf1b0 0x55b10e976aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b10e59e4d0 0x55b10e975f30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 --2- 172.21.15.29:0/4276670461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b10e5962f0 0x55b10e975870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 >> 172.21.15.29:0/4276670461 conn(0x55b10e5b3300 msgr2=0x55b10e9615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.071+0000 7fc243fff640 0 client.27814 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.075+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 shutdown_connections 2024-09-21T02:00:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.075+0000 7fc272264740 1 -- 172.21.15.29:0/4276670461 wait complete. 2024-09-21T02:00:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.076+0000 7fc272264740 20 client.27814 trim_cache size 0 max 16384 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 msgr2=0x7fc24c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 0x7fc24c0b8b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc258009bd0 tx=0x7fc258005f90 comp rx=0 tx=0).stop 2024-09-21T02:00:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 msgr2=0x7fc26410c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 0x7fc26410c3c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00efc0 tx=0x7fc25c00c800 comp rx=0 tx=0).stop 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 shutdown_connections 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc24c0b6610 0x7fc24c0b8b20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc264111f90 0x7fc26410c3c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26410ada0 0x7fc26410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 --2- 172.21.15.29:0/3941676843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc26410a3c0 0x7fc26410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 >> 172.21.15.29:0/3941676843 conn(0x7fc264070810 msgr2=0x7fc26410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 shutdown_connections 2024-09-21T02:00:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:35.079+0000 7fc26b21d640 1 -- 172.21.15.29:0/3941676843 wait complete. 2024-09-21T02:00:36.207 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:00:36.207 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:00:36.422 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:36 smithi029 ceph-mon[31228]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-09-21T02:00:36.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:36 smithi190 ceph-mon[34368]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-09-21T02:00:36.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:36 smithi066 ceph-mon[35001]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-09-21T02:00:36.647 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.249+0000 7f00953c0640 1 -- 172.21.15.29:0/446094018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090102a60 msgr2=0x7f0090102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.249+0000 7f00953c0640 1 --2- 172.21.15.29:0/446094018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090102a60 0x7f0090102e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0080009ca0 tx=0x7f008002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.251+0000 7f00953c0640 1 -- 172.21.15.29:0/446094018 shutdown_connections 2024-09-21T02:00:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.251+0000 7f00953c0640 1 --2- 172.21.15.29:0/446094018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0090108cf0 0x7f009010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.251+0000 7f00953c0640 1 --2- 172.21.15.29:0/446094018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00901083a0 0x7f00901087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.251+0000 7f00953c0640 1 --2- 172.21.15.29:0/446094018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090102a60 0x7f0090102e70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.251+0000 7f00953c0640 1 -- 172.21.15.29:0/446094018 >> 172.21.15.29:0/446094018 conn(0x7f0090079730 msgr2=0x7f0090079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.252+0000 7f00953c0640 1 -- 172.21.15.29:0/446094018 shutdown_connections 2024-09-21T02:00:37.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.252+0000 7f00953c0640 1 -- 172.21.15.29:0/446094018 wait complete. 2024-09-21T02:00:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.253+0000 7f00953c0640 1 Processor -- start 2024-09-21T02:00:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.253+0000 7f00953c0640 1 -- start start 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.253+0000 7f00953c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 0x7f00901126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.253+0000 7f00953c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 0x7f0090112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f00953c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f00953c0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f009006b1f0 con 0x7f0090108cf0 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f00953c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f009006b070 con 0x7f00901083a0 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f00953c0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f009006b370 con 0x7f0090102a60 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 0x7f00901126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45176/0 (socket says 172.21.15.29:45176) 2024-09-21T02:00:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 0x7f00901126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51910/0 (socket says 172.21.15.29:51910) 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 -- 172.21.15.29:0/3414757503 learned_addr learned my addr 172.21.15.29:0/3414757503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 0x7f0090112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 -- 172.21.15.29:0/3414757503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 msgr2=0x7f00901126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 0x7f00901126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 -- 172.21.15.29:0/3414757503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 msgr2=0x7f0090112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 0x7f0090112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008f7fe640 1 -- 172.21.15.29:0/3414757503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00901a86d0 con 0x7f0090108cf0 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.254+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 0x7f0090112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f008f7fe640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f008800e970 tx=0x7f008800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f0087fff640 1 -- 172.21.15.29:0/3414757503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008800ccc0 con 0x7f0090108cf0 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00901a89c0 con 0x7f0090108cf0 2024-09-21T02:00:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f0087fff640 1 -- 172.21.15.29:0/3414757503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008800ce60 con 0x7f0090108cf0 2024-09-21T02:00:37.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f0087fff640 1 -- 172.21.15.29:0/3414757503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088010430 con 0x7f0090108cf0 2024-09-21T02:00:37.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.255+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a8f00 con 0x7f0090108cf0 2024-09-21T02:00:37.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.259+0000 7f009b934740 1 Processor -- start 2024-09-21T02:00:37.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f0087fff640 1 -- 172.21.15.29:0/3414757503 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f00880105d0 con 0x7f0090108cf0 2024-09-21T02:00:37.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 -- start start 2024-09-21T02:00:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9136350 0x55b0d94dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b0d9453e60 con 0x55b0d9115670 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b0d9443290 con 0x55b0d910d490 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.260+0000 7f009b934740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b0d94470a0 con 0x55b0d9136350 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51490/0 (socket says 172.21.15.29:51490) 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 -- 172.21.15.29:0/3793322563 learned_addr learned my addr 172.21.15.29:0/3793322563 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9136350 0x55b0d94dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 -- 172.21.15.29:0/3793322563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9136350 msgr2=0x55b0d94dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9136350 0x55b0d94dd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 -- 172.21.15.29:0/3793322563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 msgr2=0x55b0d94ddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94ddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f008effd640 1 -- 172.21.15.29:0/3793322563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d91a1930 con 0x55b0d910d490 2024-09-21T02:00:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.261+0000 7f0087fff640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 0x7f007c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.262+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 0x7f007c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.262+0000 7f0087fff640 1 -- 172.21.15.29:0/3414757503 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f00880e7760 con 0x7f0090108cf0 2024-09-21T02:00:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.262+0000 7f008effd640 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0080009ca0 tx=0x7f0080009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d95a34687ce80f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.269+0000 7f008e7fc640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 0x7f007c0b8aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00780058c0 tx=0x7f0078005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.269+0000 7f00857fa640 1 -- 172.21.15.29:0/3793322563 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080031cf0 con 0x55b0d910d490 2024-09-21T02:00:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.269+0000 7f00857fa640 1 -- 172.21.15.29:0/3793322563 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080036030 con 0x55b0d910d490 2024-09-21T02:00:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.269+0000 7f00857fa640 1 -- 172.21.15.29:0/3793322563 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800042a0 con 0x55b0d910d490 2024-09-21T02:00:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.276+0000 7f009b934740 1 -- 172.21.15.29:0/3793322563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 msgr2=0x55b0d94dd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.276+0000 7f009b934740 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0080009ca0 tx=0x7f0080009410 comp rx=0 tx=0).stop 2024-09-21T02:00:37.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.282+0000 7f009b934740 1 -- 172.21.15.29:0/3793322563 shutdown_connections 2024-09-21T02:00:37.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.282+0000 7f009b934740 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.282+0000 7f009b934740 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94dd970 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.282+0000 7f009b934740 1 --2- 172.21.15.29:0/3793322563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9136350 0x55b0d94dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.282+0000 7f009b934740 1 -- 172.21.15.29:0/3793322563 >> 172.21.15.29:0/3793322563 conn(0x55b0d912a740 msgr2=0x55b0d94db320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.283+0000 7f009b934740 1 -- 172.21.15.29:0/3793322563 shutdown_connections 2024-09-21T02:00:37.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.283+0000 7f009b934740 1 -- 172.21.15.29:0/3793322563 wait complete. 2024-09-21T02:00:37.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.284+0000 7f009b934740 1 -- start start 2024-09-21T02:00:37.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.284+0000 7f009b934740 1 Processor -- start 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b0d92a4890 con 0x55b0d9136350 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b0d93ddd40 con 0x55b0d910d490 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b0d94eba30 con 0x55b0d9115670 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f008effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f009b934740 10 client.0 _ll_register_callbacks cb 0x55b0d9441c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f008effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:51502/0 (socket says 172.21.15.29:51502) 2024-09-21T02:00:37.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.285+0000 7f008effd640 1 -- 172.21.15.29:0/2523479821 learned_addr learned my addr 172.21.15.29:0/2523479821 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:37.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 -- 172.21.15.29:0/2523479821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 msgr2=0x55b0d94e95f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:37.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e95f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 -- 172.21.15.29:0/2523479821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 msgr2=0x55b0d94f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 -- 172.21.15.29:0/2523479821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d94f53c0 con 0x55b0d910d490 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f008effd640 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f00800316b0 tx=0x7f008002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34693b99001f8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.286+0000 7f00767fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.287+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008004a040 con 0x55b0d910d490 2024-09-21T02:00:37.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.287+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f008002f660 con 0x55b0d910d490 2024-09-21T02:00:37.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.291+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0080036030 con 0x55b0d910d490 2024-09-21T02:00:37.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.291+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0080003970 con 0x55b0d910d490 2024-09-21T02:00:37.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.296+0000 7f009b934740 10 client.27869 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:37.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.296+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b0d94f7cd0 con 0x55b0d910d490 2024-09-21T02:00:37.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.299+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080053070 con 0x55b0d910d490 2024-09-21T02:00:37.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.299+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f008004d070 con 0x55b0d910d490 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 10 client.27869 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 10 client.27869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 20 client.27869 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90f4cd0 0x55b0d94fb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 4 client.27869 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:37.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.300+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0080036030 con 0x55b0d90f4cd0 2024-09-21T02:00:37.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.302+0000 7f008e7fc640 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90f4cd0 0x55b0d94fb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.319+0000 7f00767fc640 10 client.27869 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:37.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.319+0000 7f008e7fc640 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90f4cd0 0x55b0d94fb2b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e77fca8b4f9e4c2 server_cookie=84a335a55ba3444a in_seq=0 out_seq=0 2024-09-21T02:00:37.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.320+0000 7f00767fc640 1 -- 172.21.15.29:0/2523479821 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0080036030 con 0x55b0d90f4cd0 2024-09-21T02:00:37.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.320+0000 7f00767fc640 10 client.27869 handle_command_reply: tid=0 2024-09-21T02:00:37.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.321+0000 7f009b934740 1 client.27869 shutdown 2024-09-21T02:00:37.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 msgr2=0x55b0d94e9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f00800316b0 tx=0x7f008002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 shutdown_connections 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90f4cd0 0x55b0d94fb2b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 --2- 172.21.15.29:0/2523479821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94e9050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 >> 172.21.15.29:0/2523479821 conn(0x55b0d912a740 msgr2=0x55b0d94e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.323+0000 7f00767fc640 0 client.27869 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:37.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.328+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 shutdown_connections 2024-09-21T02:00:37.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.328+0000 7f009b934740 1 -- 172.21.15.29:0/2523479821 wait complete. 2024-09-21T02:00:37.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.328+0000 7f009b934740 20 client.27869 trim_cache size 0 max 16384 2024-09-21T02:00:37.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.366+0000 7f009b934740 1 Processor -- start 2024-09-21T02:00:37.368 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:37 smithi029 ceph-mon[31228]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 766 op/s 2024-09-21T02:00:37.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.366+0000 7f009b934740 1 -- start start 2024-09-21T02:00:37.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b14b0 con 0x55b0d9136350 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b30b0 con 0x55b0d910d490 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f009b934740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b1b80 con 0x55b0d9115670 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45210/0 (socket says 172.21.15.29:45210) 2024-09-21T02:00:37.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.367+0000 7f008f7fe640 1 -- 172.21.15.29:0/4150127713 learned_addr learned my addr 172.21.15.29:0/4150127713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 -- 172.21.15.29:0/4150127713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 msgr2=0x55b0d94e0530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 -- 172.21.15.29:0/4150127713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 msgr2=0x55b0d94f4c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94f4c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 -- 172.21.15.29:0/4150127713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d94e46e0 con 0x55b0d9136350 2024-09-21T02:00:37.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.368+0000 7f008f7fe640 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f00880ad5a0 tx=0x7f00880ad2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb528def0481b522 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.369+0000 7f0084ff9640 1 -- 172.21.15.29:0/4150127713 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880046c0 con 0x55b0d9136350 2024-09-21T02:00:37.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.369+0000 7f0084ff9640 1 -- 172.21.15.29:0/4150127713 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00880ad3b0 con 0x55b0d9136350 2024-09-21T02:00:37.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.369+0000 7f0084ff9640 1 -- 172.21.15.29:0/4150127713 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880aee30 con 0x55b0d9136350 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 -- 172.21.15.29:0/4150127713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 msgr2=0x55b0d94e0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f00880ad5a0 tx=0x7f00880ad2b0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 -- 172.21.15.29:0/4150127713 shutdown_connections 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9136350 0x55b0d94e0a70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d9115670 0x55b0d94e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 --2- 172.21.15.29:0/4150127713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d910d490 0x55b0d94f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.370+0000 7f009b934740 1 -- 172.21.15.29:0/4150127713 >> 172.21.15.29:0/4150127713 conn(0x55b0d912a740 msgr2=0x55b0d94e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.371+0000 7f009b934740 1 -- 172.21.15.29:0/4150127713 shutdown_connections 2024-09-21T02:00:37.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.371+0000 7f009b934740 1 -- 172.21.15.29:0/4150127713 wait complete. 2024-09-21T02:00:37.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.372+0000 7f009b934740 1 -- start start 2024-09-21T02:00:37.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.373+0000 7f009b934740 1 Processor -- start 2024-09-21T02:00:37.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.373+0000 7f009b934740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.373+0000 7f009b934740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.374+0000 7f008effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.374+0000 7f008effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51944/0 (socket says 172.21.15.29:51944) 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.374+0000 7f008effd640 1 -- 172.21.15.29:0/1427298210 learned_addr learned my addr 172.21.15.29:0/1427298210 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d9136350 0x55b0d94ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b2920 con 0x55b0d9115670 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b2250 con 0x55b0d9136350 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00901af490 con 0x55b0d910d490 2024-09-21T02:00:37.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f009b934740 10 client.0 _ll_register_callbacks cb 0x7f0078011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f008e7fc640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.375+0000 7f008f7fe640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d9136350 0x55b0d94ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 -- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d9136350 msgr2=0x55b0d94ebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d9136350 0x55b0d94ebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 -- 172.21.15.29:0/1427298210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 msgr2=0x55b0d94f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d94dfcc0 con 0x55b0d910d490 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f008effd640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0080031680 tx=0x7f0080030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e02c8b33181f0dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.376+0000 7f00767fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080030eb0 con 0x55b0d910d490 2024-09-21T02:00:37.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f008002f660 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00880ad3b0 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0080004b40 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f009b934740 10 client.27826 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.377+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b0d956eb60 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080036020 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f008004d040 con 0x55b0d910d490 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 10 client.27826 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 10 client.27826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 20 client.27826 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90ca080 0x55b0d95713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 4 client.27826 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:00:37.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.378+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f00880ad3b0 con 0x55b0d90ca080 2024-09-21T02:00:37.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.379+0000 7f008e7fc640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90ca080 0x55b0d95713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:37.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.380+0000 7f008e7fc640 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90ca080 0x55b0d95713b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78a9c5d7c18db94 server_cookie=611ee85ee3dcc9d in_seq=0 out_seq=0 2024-09-21T02:00:37.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.380+0000 7f00767fc640 10 client.27826 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:37.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.382+0000 7f00767fc640 1 -- 172.21.15.29:0/1427298210 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f00880ad3b0 con 0x55b0d90ca080 2024-09-21T02:00:37.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.382+0000 7f00767fc640 10 client.27826 handle_command_reply: tid=0 2024-09-21T02:00:37.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.382+0000 7f009b934740 1 client.27826 shutdown 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 msgr2=0x55b0d94f0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0080031680 tx=0x7f0080030a30 comp rx=0 tx=0).stop 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 shutdown_connections 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b0d90ca080 0x55b0d95713b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f00767fc640 0 client.27826 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0d9136350 0x55b0d94ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.383+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0d9115670 0x55b0d94f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.384+0000 7f009b934740 1 --2- 172.21.15.29:0/1427298210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0d910d490 0x55b0d94f0ef0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.384+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 >> 172.21.15.29:0/1427298210 conn(0x55b0d912a740 msgr2=0x55b0d94e0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.384+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 shutdown_connections 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.384+0000 7f009b934740 1 -- 172.21.15.29:0/1427298210 wait complete. 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.384+0000 7f009b934740 20 client.27826 trim_cache size 0 max 16384 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "49a9abae-6e84-4db7-9902-1340c785ba9b", 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T02:00:37.387 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 msgr2=0x7f007c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 0x7f007c0b8aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00780058c0 tx=0x7f0078005f90 comp rx=0 tx=0).stop 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 msgr2=0x7f0090113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f008800e970 tx=0x7f008800ed70 comp rx=0 tx=0).stop 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 shutdown_connections 2024-09-21T02:00:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f007c0b6590 0x7f007c0b8aa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0090108cf0 0x7f0090113170 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00901083a0 0x7f0090112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 --2- 172.21.15.29:0/3414757503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090102a60 0x7f00901126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 >> 172.21.15.29:0/3414757503 conn(0x7f0090079730 msgr2=0x7f0090106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 shutdown_connections 2024-09-21T02:00:37.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:37.385+0000 7f00953c0640 1 -- 172.21.15.29:0/3414757503 wait complete. 2024-09-21T02:00:37.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:37 smithi190 ceph-mon[34368]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 766 op/s 2024-09-21T02:00:37.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:37 smithi066 ceph-mon[35001]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 766 op/s 2024-09-21T02:00:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 720.0 s, 1266.6 tps, lat 78.975 ms stddev 93.135 2024-09-21T02:00:38.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:00:38.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:38 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:00:38.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:38 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:00:38.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:38 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:38 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:00:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:38 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:00:38.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:38 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:38.586 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:38.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:38 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:00:38.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:38 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:00:38.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:38 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.169+0000 7f2f35a50640 1 -- 172.21.15.29:0/2268376610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 msgr2=0x7f2f300761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.169+0000 7f2f35a50640 1 --2- 172.21.15.29:0/2268376610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f300761e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009860 tx=0x7f2f1802a380 comp rx=0 tx=0).stop 2024-09-21T02:00:39.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.179+0000 7f2f35a50640 1 -- 172.21.15.29:0/2268376610 shutdown_connections 2024-09-21T02:00:39.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.179+0000 7f2f35a50640 1 --2- 172.21.15.29:0/2268376610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30114470 0x7f2f30116940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.179+0000 7f2f35a50640 1 --2- 172.21.15.29:0/2268376610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f300761e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.179+0000 7f2f35a50640 1 --2- 172.21.15.29:0/2268376610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30077380 0x7f2f300757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.179+0000 7f2f35a50640 1 -- 172.21.15.29:0/2268376610 >> 172.21.15.29:0/2268376610 conn(0x7f2f300ff630 msgr2=0x7f2f30101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.180+0000 7f2f35a50640 1 -- 172.21.15.29:0/2268376610 shutdown_connections 2024-09-21T02:00:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.180+0000 7f2f35a50640 1 -- 172.21.15.29:0/2268376610 wait complete. 2024-09-21T02:00:39.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.181+0000 7f2f35a50640 1 Processor -- start 2024-09-21T02:00:39.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 -- start start 2024-09-21T02:00:39.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f30113d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30114470 0x7f2f30110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010a430 con 0x7f2f30075dd0 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010a2b0 con 0x7f2f30114470 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.187+0000 7f2f35a50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010a5b0 con 0x7f2f30077380 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.188+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.188+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51966/0 (socket says 172.21.15.29:51966) 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.188+0000 7f2f27fff640 1 -- 172.21.15.29:0/1508980958 learned_addr learned my addr 172.21.15.29:0/1508980958 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:39.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.190+0000 7f2f2effd640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f30113d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30114470 0x7f2f30110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2effd640 1 -- 172.21.15.29:0/1508980958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 msgr2=0x7f2f3010fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2effd640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2effd640 1 -- 172.21.15.29:0/1508980958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30114470 msgr2=0x7f2f30110670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2effd640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30114470 0x7f2f30110670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.192+0000 7f2f2effd640 1 -- 172.21.15.29:0/1508980958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3006cde0 con 0x7f2f30075dd0 2024-09-21T02:00:39.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.193+0000 7f2f2effd640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f30113d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000b3f0 tx=0x7f2f2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.194+0000 7f2f34a4e640 1 -- 172.21.15.29:0/1508980958 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20015020 con 0x7f2f30075dd0 2024-09-21T02:00:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.194+0000 7f2f34a4e640 1 -- 172.21.15.29:0/1508980958 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f20011ce0 con 0x7f2f30075dd0 2024-09-21T02:00:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.195+0000 7f2f34a4e640 1 -- 172.21.15.29:0/1508980958 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20010dd0 con 0x7f2f30075dd0 2024-09-21T02:00:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.196+0000 7f2f27fff640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.196+0000 7f2f35a50640 1 -- 172.21.15.29:0/1508980958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3006d0d0 con 0x7f2f30075dd0 2024-09-21T02:00:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.197+0000 7f2f35a50640 1 -- 172.21.15.29:0/1508980958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3006d590 con 0x7f2f30075dd0 2024-09-21T02:00:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.200+0000 7f2f3bfc4740 1 Processor -- start 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 -- start start 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff801b0 0x565190350a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ff88390 0x565190350fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5651902c6e50 con 0x56518ff801b0 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5651902b6280 con 0x56518ff88390 2024-09-21T02:00:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.201+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5651902ba090 con 0x56518ffa9070 2024-09-21T02:00:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.202+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.202+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51974/0 (socket says 172.21.15.29:51974) 2024-09-21T02:00:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.202+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/3551371689 learned_addr learned my addr 172.21.15.29:0/3551371689 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.202+0000 7f2f27fff640 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ff88390 0x565190350fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/3551371689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ff88390 msgr2=0x565190350fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ff88390 0x565190350fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/3551371689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff801b0 msgr2=0x565190350a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff801b0 0x565190350a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/3551371689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565190014650 con 0x56518ffa9070 2024-09-21T02:00:39.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.206+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f28020490 tx=0x7f2f28020890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e39091b2c3dcda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.209+0000 7f2f257fa640 1 -- 172.21.15.29:0/3551371689 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2802a420 con 0x56518ffa9070 2024-09-21T02:00:39.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.209+0000 7f2f257fa640 1 -- 172.21.15.29:0/3551371689 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2801d8f0 con 0x56518ffa9070 2024-09-21T02:00:39.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.209+0000 7f2f34a4e640 1 -- 172.21.15.29:0/1508980958 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2f20018020 con 0x7f2f30075dd0 2024-09-21T02:00:39.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.210+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/3551371689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 msgr2=0x565190350530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.210+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f28020490 tx=0x7f2f28020890 comp rx=0 tx=0).stop 2024-09-21T02:00:39.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.214+0000 7f2f34a4e640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 0x7f2f1c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.215+0000 7f2f34a4e640 1 -- 172.21.15.29:0/1508980958 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2f200e8910 con 0x7f2f30075dd0 2024-09-21T02:00:39.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.219+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 0x7f2f1c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.220+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/3551371689 shutdown_connections 2024-09-21T02:00:39.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.220+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ff88390 0x565190350fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.220+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff801b0 0x565190350a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.220+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/3551371689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ffa9070 0x565190350530 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.220+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/3551371689 >> 172.21.15.29:0/3551371689 conn(0x56518ff9d460 msgr2=0x56519034e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.224+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/3551371689 shutdown_connections 2024-09-21T02:00:39.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.226+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/3551371689 wait complete. 2024-09-21T02:00:39.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.226+0000 7f2f3bfc4740 1 -- start start 2024-09-21T02:00:39.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.226+0000 7f2f3bfc4740 1 Processor -- start 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 0x7f2f1c0b8b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2f28020460 tx=0x7f2f2801ac80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x56519035b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565190117700 con 0x56518ff88390 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565190251110 con 0x56518ffa9070 2024-09-21T02:00:39.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56519035ec80 con 0x56518ff801b0 2024-09-21T02:00:39.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.227+0000 7f2f3bfc4740 10 client.0 _ll_register_callbacks cb 0x56519026a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:39.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.229+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.229+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45252/0 (socket says 172.21.15.29:45252) 2024-09-21T02:00:39.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.229+0000 7f2f27fff640 1 -- 172.21.15.29:0/2076798177 learned_addr learned my addr 172.21.15.29:0/2076798177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:39.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.231+0000 7f2f27fff640 1 -- 172.21.15.29:0/2076798177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 msgr2=0x56519035b970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:39.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.231+0000 7f2f27fff640 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x56519035b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.231+0000 7f2f27fff640 1 -- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 msgr2=0x56519035c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:39.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.231+0000 7f2f27fff640 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.231+0000 7f2f27fff640 1 -- 172.21.15.29:0/2076798177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519027ec40 con 0x56518ff88390 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.233+0000 7f2f27fff640 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009830 tx=0x7f2f180044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6913e8b70fd89ded server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18042070 con 0x56518ff88390 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2f18005030 con 0x56518ff88390 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2801d8f0 con 0x56518ff88390 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2f1804e050 con 0x56518ff88390 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f3bfc4740 10 client.18051 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:39.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.234+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56519036a750 con 0x56518ff88390 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2801d8f0 con 0x56518ff88390 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2f180035f0 con 0x56518ff88390 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 10 client.18051 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 20 client.18051 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff679f0 0x56519036e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 4 client.18051 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651902c8da0 con 0x56518ff679f0 2024-09-21T02:00:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.235+0000 7f2f2effd640 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff679f0 0x56519036e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.236+0000 7f2f16ffd640 10 client.18051 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:39.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.237+0000 7f2f2effd640 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff679f0 0x56519036e1a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b4e7a8ccd886f3 server_cookie=683136b9cd3c713d in_seq=0 out_seq=0 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.238+0000 7f2f16ffd640 1 -- 172.21.15.29:0/2076798177 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651902c8da0 con 0x56518ff679f0 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.238+0000 7f2f16ffd640 10 client.18051 handle_command_reply: tid=0 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.238+0000 7f2f3bfc4740 1 client.18051 shutdown 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 msgr2=0x56519035beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009830 tx=0x7f2f180044f0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 shutdown_connections 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff679f0 0x56519036e1a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035beb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2076798177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x56519035b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 >> 172.21.15.29:0/2076798177 conn(0x56518ff9d460 msgr2=0x565190353710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f16ffd640 0 client.18051 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 shutdown_connections 2024-09-21T02:00:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2076798177 wait complete. 2024-09-21T02:00:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.239+0000 7f2f3bfc4740 20 client.18051 trim_cache size 0 max 16384 2024-09-21T02:00:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.279+0000 7f2f3bfc4740 1 Processor -- start 2024-09-21T02:00:39.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.279+0000 7f2f3bfc4740 1 -- start start 2024-09-21T02:00:39.281 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.282 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:39 smithi029 ceph-mon[31228]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.280+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.280+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f2effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f2effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51984/0 (socket says 172.21.15.29:51984) 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45256/0 (socket says 172.21.15.29:45256) 2024-09-21T02:00:39.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f2effd640 1 -- 172.21.15.29:0/188265897 learned_addr learned my addr 172.21.15.29:0/188265897 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519034d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301b5890 con 0x56518ff88390 2024-09-21T02:00:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301b7490 con 0x56518ffa9070 2024-09-21T02:00:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301b5f60 con 0x56518ff801b0 2024-09-21T02:00:39.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.281+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519034d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/188265897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 msgr2=0x565190351f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/188265897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 msgr2=0x56519035d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/188265897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519035c730 con 0x56518ffa9070 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2effd640 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519034d9d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f280203e0 tx=0x7f2f2801a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bd41463f1fc329 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f24ff9640 1 -- 172.21.15.29:0/188265897 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2801ec40 con 0x56518ffa9070 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f24ff9640 1 -- 172.21.15.29:0/188265897 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2801ede0 con 0x56518ffa9070 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.282+0000 7f2f24ff9640 1 -- 172.21.15.29:0/188265897 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2801d560 con 0x56518ffa9070 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.283+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 msgr2=0x56519034d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.283+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519034d9d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f280203e0 tx=0x7f2f2801a5d0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 shutdown_connections 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519034d9d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/188265897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x565190351f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 >> 172.21.15.29:0/188265897 conn(0x56518ff9d460 msgr2=0x56519035ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 shutdown_connections 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/188265897 wait complete. 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 -- start start 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.284+0000 7f2f3bfc4740 1 Processor -- start 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.285+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x5651903644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.285+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.285+0000 7f2f3bfc4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x5651903644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x5651903644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:51998/0 (socket says 172.21.15.29:51998) 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2effd640 1 -- 172.21.15.29:0/2014667693 learned_addr learned my addr 172.21.15.29:0/2014667693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f27fff640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301b6d00 con 0x56518ff88390 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301b6630 con 0x56518ffa9070 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30073890 con 0x56518ff801b0 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f3bfc4740 10 client.0 _ll_register_callbacks cb 0x7f2f180096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/2014667693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 msgr2=0x5651903644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x5651903644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/2014667693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 msgr2=0x56519035f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f2f7fe640 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651903603d0 con 0x56518ffa9070 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.286+0000 7f2f27fff640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f2f7fe640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035fe90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f2801ab90 tx=0x7f2f2802bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb473acb95c099d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f167fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:00:39.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2801ec40 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2f2802b7e0 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2801ede0 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2f28003970 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f3bfc4740 10 client.27884 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.287+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5651903d8770 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2801ede0 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2f28033070 con 0x56518ffa9070 2024-09-21T02:00:39.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 10 client.27884 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 10 client.27884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 20 client.27884 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff3cda0 0x5651903dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 4 client.27884 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.288+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651902c4850 con 0x56518ff3cda0 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f2effd640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff3cda0 0x5651903dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f2effd640 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff3cda0 0x5651903dad00 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29f4887842c0843 server_cookie=119c2177fc426867 in_seq=0 out_seq=0 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f167fc640 10 client.27884 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f167fc640 1 -- 172.21.15.29:0/2014667693 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x5651902c4850 con 0x56518ff3cda0 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f167fc640 10 client.27884 handle_command_reply: tid=0 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f3bfc4740 1 client.27884 shutdown 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 msgr2=0x56519035fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.289+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035fe90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f2801ab90 tx=0x7f2f2802bd60 comp rx=0 tx=0).stop 2024-09-21T02:00:39.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 shutdown_connections 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56518ff3cda0 0x5651903dad00 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56518ffa9070 0x56519035fe90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56518ff88390 0x56519035f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 --2- 172.21.15.29:0/2014667693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56518ff801b0 0x5651903644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 >> 172.21.15.29:0/2014667693 conn(0x56518ff9d460 msgr2=0x56519035b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f167fc640 0 client.27884 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 shutdown_connections 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 1 -- 172.21.15.29:0/2014667693 wait complete. 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.290+0000 7f2f3bfc4740 20 client.27884 trim_cache size 0 max 16384 2024-09-21T02:00:39.296 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "49a9abae-6e84-4db7-9902-1340c785ba9b", 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:00:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:00:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:00:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:00:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:00:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:00:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 msgr2=0x7f2f1c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 0x7f2f1c0b8b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2f28020460 tx=0x7f2f2801ac80 comp rx=0 tx=0).stop 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 msgr2=0x7f2f30113d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f30113d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000b3f0 tx=0x7f2f2000b8b0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 shutdown_connections 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2f1c0b65f0 0x7f2f1c0b8b00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f30114470 0x7f2f30110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f30077380 0x7f2f3010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 --2- 172.21.15.29:0/1508980958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f30075dd0 0x7f2f30113d10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 >> 172.21.15.29:0/1508980958 conn(0x7f2f300ff630 msgr2=0x7f2f301035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 shutdown_connections 2024-09-21T02:00:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:00:39.291+0000 7f2f267fc640 1 -- 172.21.15.29:0/1508980958 wait complete. 2024-09-21T02:00:39.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T02:00:39.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:39 smithi190 ceph-mon[34368]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-21T02:00:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T02:00:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:00:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:00:39.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:00:39.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:00:39.550 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:39 smithi066 ceph-mon[35001]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-21T02:00:40.170 INFO:tasks.cephfs.filesystem:scrub status for tag:49a9abae-6e84-4db7-9902-1340c785ba9b - {'path': '/', 'tag': '49a9abae-6e84-4db7-9902-1340c785ba9b', 'options': 'recursive,force'} 2024-09-21T02:00:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:41 smithi066 ceph-mon[35001]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 875 op/s 2024-09-21T02:00:41.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:41 smithi029 ceph-mon[31228]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 875 op/s 2024-09-21T02:00:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:41 smithi190 ceph-mon[34368]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 875 op/s 2024-09-21T02:00:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 725.0 s, 1463.4 tps, lat 68.549 ms stddev 79.997 2024-09-21T02:00:43.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:43 smithi066 ceph-mon[35001]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-09-21T02:00:43.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:43 smithi029 ceph-mon[31228]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-09-21T02:00:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:43 smithi190 ceph-mon[34368]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-09-21T02:00:45.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:45 smithi066 ceph-mon[35001]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-21T02:00:45.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:45 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:45 smithi029 ceph-mon[31228]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-21T02:00:45.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:45 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:45 smithi190 ceph-mon[34368]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-21T02:00:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:45 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:46.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:46.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:46.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:00:46.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:46.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:46.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:00:46.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:46.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:46.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:00:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 730.0 s, 1489.4 tps, lat 67.678 ms stddev 73.392 2024-09-21T02:00:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 -- 172.21.15.29:0/2443728644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 msgr2=0x7f0dfc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 --2- 172.21.15.29:0/2443728644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc102ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df002a420 comp rx=0 tx=0).stop 2024-09-21T02:00:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 -- 172.21.15.29:0/2443728644 shutdown_connections 2024-09-21T02:00:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 --2- 172.21.15.29:0/2443728644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc10ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 --2- 172.21.15.29:0/2443728644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dfc1034c0 0x7f0dfc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 --2- 172.21.15.29:0/2443728644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc102ef0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 -- 172.21.15.29:0/2443728644 >> 172.21.15.29:0/2443728644 conn(0x7f0dfc0fd690 msgr2=0x7f0dfc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 -- 172.21.15.29:0/2443728644 shutdown_connections 2024-09-21T02:00:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.229+0000 7f0e031da640 1 -- 172.21.15.29:0/2443728644 wait complete. 2024-09-21T02:00:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.230+0000 7f0e031da640 1 Processor -- start 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.230+0000 7f0e031da640 1 -- start start 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dfc1034c0 0x7f0dfc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc10e2d0 con 0x7f0dfc102ae0 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc10e150 con 0x7f0dfc1040d0 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e031da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc10e450 con 0x7f0dfc1034c0 2024-09-21T02:00:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.231+0000 7f0e00f76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59050/0 (socket says 172.21.15.29:59050) 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 -- 172.21.15.29:0/408290408 learned_addr learned my addr 172.21.15.29:0/408290408 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 -- 172.21.15.29:0/408290408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dfc1034c0 msgr2=0x7f0dfc199560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dfc1034c0 0x7f0dfc199560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 -- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 msgr2=0x7f0dfc198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.232+0000 7f0e01777640 1 -- 172.21.15.29:0/408290408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc19a930 con 0x7f0dfc1040d0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0e01777640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0df400eb10 tx=0x7f0df400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0df9ffb640 1 -- 172.21.15.29:0/408290408 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4004b20 con 0x7f0dfc1040d0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0df9ffb640 1 -- 172.21.15.29:0/408290408 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df4004cc0 con 0x7f0dfc1040d0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0df9ffb640 1 -- 172.21.15.29:0/408290408 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4010670 con 0x7f0dfc1040d0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc19ac20 con 0x7f0dfc1040d0 2024-09-21T02:00:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.233+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc101a50 con 0x7f0dfc1040d0 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.237+0000 7f0df9ffb640 1 -- 172.21.15.29:0/408290408 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0df4005020 con 0x7f0dfc1040d0 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.238+0000 7f0df9ffb640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 0x7f0dec0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.239+0000 7f0df9ffb640 1 -- 172.21.15.29:0/408290408 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0df40e7910 con 0x7f0dfc1040d0 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.240+0000 7f0e0a222740 1 Processor -- start 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 -- start start 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0ab61b0 0x5570c0e594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a8d2f0 0x5570c0e59b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5570c0dd06f0 con 0x5570c0ab61b0 2024-09-21T02:00:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5570c0dbfb20 con 0x5570c0a954d0 2024-09-21T02:00:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.241+0000 7f0e0a222740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5570c0dc3930 con 0x5570c0a8d2f0 2024-09-21T02:00:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.244+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.244+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59060/0 (socket says 172.21.15.29:59060) 2024-09-21T02:00:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.244+0000 7f0e01777640 1 -- 172.21.15.29:0/301897403 learned_addr learned my addr 172.21.15.29:0/301897403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.244+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a8d2f0 0x5570c0e59b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.245+0000 7f0e01777640 1 -- 172.21.15.29:0/301897403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a8d2f0 msgr2=0x5570c0e59b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.245+0000 7f0e01777640 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a8d2f0 0x5570c0e59b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.245+0000 7f0e01777640 1 -- 172.21.15.29:0/301897403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0ab61b0 msgr2=0x5570c0e594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.245+0000 7f0e01777640 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0ab61b0 0x5570c0e594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.245+0000 7f0e01777640 1 -- 172.21.15.29:0/301897403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c0d85530 con 0x5570c0a954d0 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.247+0000 7f0e01777640 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0df40aa020 tx=0x7f0df400cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c738b243999f6eda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.249+0000 7f0e00f76640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 0x7f0dec0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.249+0000 7f0de6ffd640 1 -- 172.21.15.29:0/301897403 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df40ad560 con 0x5570c0a954d0 2024-09-21T02:00:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.249+0000 7f0de6ffd640 1 -- 172.21.15.29:0/301897403 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df40d10e0 con 0x5570c0a954d0 2024-09-21T02:00:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.250+0000 7f0e0a222740 1 -- 172.21.15.29:0/301897403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 msgr2=0x5570c0e5a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.250+0000 7f0e0a222740 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0df40aa020 tx=0x7f0df400cee0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.255+0000 7f0e0a222740 1 -- 172.21.15.29:0/301897403 shutdown_connections 2024-09-21T02:00:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.255+0000 7f0e0a222740 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e5a660 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.255+0000 7f0e0a222740 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a8d2f0 0x5570c0e59b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.255+0000 7f0e0a222740 1 --2- 172.21.15.29:0/301897403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0ab61b0 0x5570c0e594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.255+0000 7f0e0a222740 1 -- 172.21.15.29:0/301897403 >> 172.21.15.29:0/301897403 conn(0x5570c0aaa300 msgr2=0x5570c0e620f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.257+0000 7f0e0a222740 1 -- 172.21.15.29:0/301897403 shutdown_connections 2024-09-21T02:00:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.258+0000 7f0e00f76640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 0x7f0dec0b8b60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0df0031500 tx=0x7f0df0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.258+0000 7f0e0a222740 1 -- 172.21.15.29:0/301897403 wait complete. 2024-09-21T02:00:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.258+0000 7f0e0a222740 1 -- start start 2024-09-21T02:00:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.258+0000 7f0e0a222740 1 Processor -- start 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 0x5570c0e67c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 0x5570c0e68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5570c0cf7ab0 con 0x5570c0a8d2f0 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5570c0d65730 con 0x5570c0ab61b0 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5570c0e5af40 con 0x5570c0a954d0 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e0a222740 10 client.0 _ll_register_callbacks cb 0x5570c0db96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.259+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 0x5570c0e68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.260+0000 7f0e01777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 0x5570c0e68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59072/0 (socket says 172.21.15.29:59072) 2024-09-21T02:00:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.260+0000 7f0e01777640 1 -- 172.21.15.29:0/1424234553 learned_addr learned my addr 172.21.15.29:0/1424234553 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.260+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 0x5570c0e67c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.260+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.263+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 msgr2=0x5570c0e68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.263+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 0x5570c0e68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.263+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1424234553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 msgr2=0x5570c0e675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.263+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.263+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1424234553 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c0b21790 con 0x5570c0a954d0 2024-09-21T02:00:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.264+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 0x5570c0e67c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0de800bf30 tx=0x7f0de800d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f08667744de2d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.266+0000 7f0ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:00:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.266+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8014430 con 0x5570c0a954d0 2024-09-21T02:00:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.266+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0de80145d0 con 0x5570c0a954d0 2024-09-21T02:00:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.266+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df40d10e0 con 0x5570c0a954d0 2024-09-21T02:00:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.270+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0de80147d0 con 0x5570c0a954d0 2024-09-21T02:00:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.270+0000 7f0e0a222740 10 client.27844 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.270+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5570c0b9a620 con 0x5570c0a954d0 2024-09-21T02:00:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df40d10e0 con 0x5570c0a954d0 2024-09-21T02:00:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0de8013070 con 0x5570c0a954d0 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 10 client.27844 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 10 client.27844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 20 client.27844 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a74d80 0x5570c0e74e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 4 client.27844 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:00:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.271+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570c0dd2640 con 0x5570c0a74d80 2024-09-21T02:00:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.275+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a74d80 0x5570c0e74e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.276+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a74d80 0x5570c0e74e90 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d04855cf3a476e server_cookie=f17ba9f695bc0e97 in_seq=0 out_seq=0 2024-09-21T02:00:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.276+0000 7f0ddbfff640 10 client.27844 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.277+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1424234553 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570c0dd2640 con 0x5570c0a74d80 2024-09-21T02:00:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.277+0000 7f0ddbfff640 10 client.27844 handle_command_reply: tid=0 2024-09-21T02:00:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 client.27844 shutdown 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 msgr2=0x5570c0e67c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 0x5570c0e67c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0de800bf30 tx=0x7f0de800d5f0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 shutdown_connections 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a74d80 0x5570c0e74e90 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0ab61b0 0x5570c0e68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0a954d0 0x5570c0e67c90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1424234553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 >> 172.21.15.29:0/1424234553 conn(0x5570c0aaa300 msgr2=0x5570c0e57ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0ddbfff640 0 client.27844 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 shutdown_connections 2024-09-21T02:00:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 1 -- 172.21.15.29:0/1424234553 wait complete. 2024-09-21T02:00:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.278+0000 7f0e0a222740 20 client.27844 trim_cache size 0 max 16384 2024-09-21T02:00:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.305+0000 7f0e0a222740 1 Processor -- start 2024-09-21T02:00:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.305+0000 7f0e0a222740 1 -- start start 2024-09-21T02:00:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.305+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.305+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e61a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.305+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e0a222740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1a8f10 con 0x5570c0a8d2f0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e0a222740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1aab10 con 0x5570c0a954d0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e0a222740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1a95e0 con 0x5570c0ab61b0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51724/0 (socket says 172.21.15.29:51724) 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 -- 172.21.15.29:0/1935083203 learned_addr learned my addr 172.21.15.29:0/1935083203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e01777640 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e61a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 -- 172.21.15.29:0/1935083203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 msgr2=0x5570c0e667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 -- 172.21.15.29:0/1935083203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 msgr2=0x5570c0e61a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:00:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e61a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.306+0000 7f0e00f76640 1 -- 172.21.15.29:0/1935083203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c0e68b70 con 0x5570c0a8d2f0 2024-09-21T02:00:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.307+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbd819508187c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.307+0000 7f0de67fc640 1 -- 172.21.15.29:0/1935083203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0055070 con 0x5570c0a8d2f0 2024-09-21T02:00:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.308+0000 7f0de67fc640 1 -- 172.21.15.29:0/1935083203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df0040070 con 0x5570c0a8d2f0 2024-09-21T02:00:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.308+0000 7f0de67fc640 1 -- 172.21.15.29:0/1935083203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df003c6a0 con 0x5570c0a8d2f0 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 -- 172.21.15.29:0/1935083203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 msgr2=0x5570c0e74e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df003e000 comp rx=0 tx=0).stop 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 -- 172.21.15.29:0/1935083203 shutdown_connections 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a954d0 0x5570c0e61a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1935083203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a8d2f0 0x5570c0e74e20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 -- 172.21.15.29:0/1935083203 >> 172.21.15.29:0/1935083203 conn(0x5570c0aaa300 msgr2=0x5570c0e546f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 -- 172.21.15.29:0/1935083203 shutdown_connections 2024-09-21T02:00:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.309+0000 7f0e0a222740 1 -- 172.21.15.29:0/1935083203 wait complete. 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.310+0000 7f0e0a222740 1 -- start start 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.310+0000 7f0e0a222740 1 Processor -- start 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.310+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a8d2f0 0x5570c0e6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.310+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0e0a222740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0e0a222740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1aa380 con 0x5570c0a954d0 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0e0a222740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1a9cb0 con 0x5570c0a8d2f0 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0e0a222740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1a6ef0 con 0x5570c0ab61b0 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0e0a222740 10 client.0 _ll_register_callbacks cb 0x7f0de8006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51726/0 (socket says 172.21.15.29:51726) 2024-09-21T02:00:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1691733144 learned_addr learned my addr 172.21.15.29:0/1691733144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1691733144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 msgr2=0x5570c0e6b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e6b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.311+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a8d2f0 msgr2=0x5570c0e6a810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a8d2f0 0x5570c0e6a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0dfbfff640 1 -- 172.21.15.29:0/1691733144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c0e6c130 con 0x5570c0a954d0 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0dfbfff640 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006a50 tx=0x7f0de80146e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7b73273d9df4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:00:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8014b80 con 0x5570c0a954d0 2024-09-21T02:00:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0de8014d20 con 0x5570c0a954d0 2024-09-21T02:00:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.312+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df0040070 con 0x5570c0a954d0 2024-09-21T02:00:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0de801e450 con 0x5570c0a954d0 2024-09-21T02:00:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 10 client.18072 fetch_fsmap learned FSMap version 47 2024-09-21T02:00:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5570c0e6f4d0 con 0x5570c0a954d0 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df0040070 con 0x5570c0a954d0 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0de8031030 con 0x5570c0a954d0 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 10 client.18072 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 20 client.18072 populate_metadata read hostname 'smithi029' 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a4a1c0 0x5570c0f039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 4 client.18072 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.313+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570c0dce0f0 con 0x5570c0a4a1c0 2024-09-21T02:00:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.314+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a4a1c0 0x5570c0f039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.315+0000 7f0e00f76640 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a4a1c0 0x5570c0f039b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e53bba81d3bd8f8 server_cookie=41a3aafcb9cfec6a in_seq=0 out_seq=0 2024-09-21T02:00:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.315+0000 7f0ddbfff640 10 client.18072 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0ddbfff640 1 -- 172.21.15.29:0/1691733144 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570c0dce0f0 con 0x5570c0a4a1c0 2024-09-21T02:00:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0ddbfff640 10 client.18072 handle_command_reply: tid=0 2024-09-21T02:00:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 client.18072 shutdown 2024-09-21T02:00:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 msgr2=0x5570c0e6aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006a50 tx=0x7f0de80146e0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 shutdown_connections 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5570c0a4a1c0 0x5570c0f039b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570c0ab61b0 0x5570c0e6b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5570c0a954d0 0x5570c0e6aed0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 --2- 172.21.15.29:0/1691733144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570c0a8d2f0 0x5570c0e6a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 >> 172.21.15.29:0/1691733144 conn(0x5570c0aaa300 msgr2=0x5570c0e61a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.323+0000 7f0ddbfff640 0 client.18072 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.324+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 shutdown_connections 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.324+0000 7f0e0a222740 1 -- 172.21.15.29:0/1691733144 wait complete. 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.324+0000 7f0e0a222740 20 client.18072 trim_cache size 0 max 16384 2024-09-21T02:00:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:00:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:00:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:00:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.325+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 msgr2=0x7f0dec0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.325+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 0x7f0dec0b8b60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0df0031500 tx=0x7f0df0005fd0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.325+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 msgr2=0x7f0dfc19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.325+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0df400eb10 tx=0x7f0df400c2d0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 shutdown_connections 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0dec0b6650 0x7f0dec0b8b60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dfc1040d0 0x7f0dfc19a110 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dfc1034c0 0x7f0dfc199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 --2- 172.21.15.29:0/408290408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dfc102ae0 0x7f0dfc198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 >> 172.21.15.29:0/408290408 conn(0x7f0dfc0fd690 msgr2=0x7f0dfc106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 shutdown_connections 2024-09-21T02:00:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:00:48.326+0000 7f0e031da640 1 -- 172.21.15.29:0/408290408 wait complete. 2024-09-21T02:00:48.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:48 smithi029 ceph-mon[31228]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 912 op/s 2024-09-21T02:00:48.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:48 smithi190 ceph-mon[34368]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 912 op/s 2024-09-21T02:00:48.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:48 smithi066 ceph-mon[35001]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 912 op/s 2024-09-21T02:00:50.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:50 smithi029 ceph-mon[31228]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 695 op/s 2024-09-21T02:00:50.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:50 smithi190 ceph-mon[34368]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 695 op/s 2024-09-21T02:00:50.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:50 smithi066 ceph-mon[35001]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 695 op/s 2024-09-21T02:00:51.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:00:51.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:00:51.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:00:51.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:00:51.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:51 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:00:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:00:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:00:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:00:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:00:51.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:51 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:00:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:00:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:00:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:00:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:00:51.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:51 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:00:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:52 smithi029 ceph-mon[31228]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 17 MiB/s wr, 855 op/s 2024-09-21T02:00:52.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:52 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:00:52.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:52 smithi190 ceph-mon[34368]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 17 MiB/s wr, 855 op/s 2024-09-21T02:00:52.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:52 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:00:52.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:52 smithi066 ceph-mon[35001]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 17 MiB/s wr, 855 op/s 2024-09-21T02:00:52.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:52 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:00:52.967 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 735.0 s, 1728.8 tps, lat 57.506 ms stddev 66.357 2024-09-21T02:00:54.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:54 smithi029 ceph-mon[31228]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-21T02:00:54.351 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:54 smithi066 ceph-mon[35001]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-21T02:00:54.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:54 smithi190 ceph-mon[34368]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-21T02:00:56.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:56 smithi029 ceph-mon[31228]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 611 op/s 2024-09-21T02:00:56.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:56 smithi190 ceph-mon[34368]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 611 op/s 2024-09-21T02:00:56.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:56 smithi066 ceph-mon[35001]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 611 op/s 2024-09-21T02:00:57.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:57 smithi190 ceph-mon[34368]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 684 op/s 2024-09-21T02:00:57.549 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:57 smithi066 ceph-mon[35001]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 684 op/s 2024-09-21T02:00:57.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:57 smithi029 ceph-mon[31228]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 684 op/s 2024-09-21T02:00:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 740.0 s, 1733.2 tps, lat 57.071 ms stddev 65.948 2024-09-21T02:00:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:00:59 smithi066 ceph-mon[35001]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-21T02:00:59.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:00:59 smithi029 ceph-mon[31228]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-21T02:00:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:00:59 smithi190 ceph-mon[34368]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-21T02:01:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.457+0000 7f389be9b640 1 -- 172.21.15.29:0/2245758574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103650 msgr2=0x7f3894103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.457+0000 7f389be9b640 1 --2- 172.21.15.29:0/2245758574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103650 0x7f3894103a60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402a410 comp rx=0 tx=0).stop 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.458+0000 7f389be9b640 1 -- 172.21.15.29:0/2245758574 shutdown_connections 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.458+0000 7f389be9b640 1 --2- 172.21.15.29:0/2245758574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3894103ff0 0x7f389410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.458+0000 7f389be9b640 1 --2- 172.21.15.29:0/2245758574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103650 0x7f3894103a60 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402a410 comp rx=0 tx=0).stop 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.458+0000 7f389be9b640 1 --2- 172.21.15.29:0/2245758574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f3894102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.458+0000 7f389be9b640 1 -- 172.21.15.29:0/2245758574 >> 172.21.15.29:0/2245758574 conn(0x7f38940fdf30 msgr2=0x7f38941003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.459+0000 7f389be9b640 1 -- 172.21.15.29:0/2245758574 shutdown_connections 2024-09-21T02:01:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.459+0000 7f389be9b640 1 -- 172.21.15.29:0/2245758574 wait complete. 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 Processor -- start 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 -- start start 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f389419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389419b220 0x7f389419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f389410e220 con 0x7f3894103ff0 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f389410e0a0 con 0x7f38941025d0 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f389be9b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f389410e3a0 con 0x7f389419b220 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57562/0 (socket says 172.21.15.29:57562) 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.460+0000 7f3899436640 1 -- 172.21.15.29:0/4092956258 learned_addr learned my addr 172.21.15.29:0/4092956258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f389419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f389a438640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389419b220 0x7f389419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 -- 172.21.15.29:0/4092956258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389419b220 msgr2=0x7f389419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389419b220 0x7f389419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 -- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 msgr2=0x7f389419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 -- 172.21.15.29:0/4092956258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389419bf40 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899436640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f3899c37640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f389419a0f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f389000e9f0 tx=0x7f389000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f388affd640 1 -- 172.21.15.29:0/4092956258 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389000cd80 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38941a22d0 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f388affd640 1 -- 172.21.15.29:0/4092956258 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38900035f0 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f388affd640 1 -- 172.21.15.29:0/4092956258 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890010690 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.461+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38941a2870 con 0x7f38941025d0 2024-09-21T02:01:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.464+0000 7f388affd640 1 -- 172.21.15.29:0/4092956258 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f38900037a0 con 0x7f38941025d0 2024-09-21T02:01:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.465+0000 7f38a2e09740 1 Processor -- start 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.465+0000 7f38a2e09740 1 -- start start 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560bcd34e6f0 con 0x560bcd0134d0 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560bcd33db20 con 0x560bcd00b2f0 2024-09-21T02:01:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f38a2e09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560bcd341930 con 0x560bcd0341b0 2024-09-21T02:01:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f389a438640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f389a438640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56146/0 (socket says 172.21.15.29:56146) 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.466+0000 7f389a438640 1 -- 172.21.15.29:0/998060630 learned_addr learned my addr 172.21.15.29:0/998060630 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 -- 172.21.15.29:0/998060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 msgr2=0x560bcd3dc480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f388affd640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 0x7f38800b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f3899c37640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3dc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 -- 172.21.15.29:0/998060630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 msgr2=0x560bcd3dcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3dcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 -- 172.21.15.29:0/998060630 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd303530 con 0x560bcd0341b0 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f3899436640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f3899c37640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f388affd640 1 -- 172.21.15.29:0/4092956258 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f38900e7600 con 0x7f38941025d0 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627cf8f4eb86c171 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.467+0000 7f389a438640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 0x7f38800b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.468+0000 7f387ffff640 1 -- 172.21.15.29:0/998060630 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c004250 con 0x560bcd0341b0 2024-09-21T02:01:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.468+0000 7f387ffff640 1 -- 172.21.15.29:0/998060630 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c00f4e0 con 0x560bcd0341b0 2024-09-21T02:01:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.468+0000 7f387ffff640 1 -- 172.21.15.29:0/998060630 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c01b670 con 0x560bcd0341b0 2024-09-21T02:01:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 -- 172.21.15.29:0/998060630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 msgr2=0x560bcd3dbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c0118d0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f389a438640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 0x7f38800b8a50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f388c020000 tx=0x7f388c01bd70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 -- 172.21.15.29:0/998060630 shutdown_connections 2024-09-21T02:01:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 --2- 172.21.15.29:0/998060630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3dbe10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 -- 172.21.15.29:0/998060630 >> 172.21.15.29:0/998060630 conn(0x560bcd028300 msgr2=0x560bcd3d78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 -- 172.21.15.29:0/998060630 shutdown_connections 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.469+0000 7f38a2e09740 1 -- 172.21.15.29:0/998060630 wait complete. 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 -- start start 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 Processor -- start 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3ede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560bcd275ab0 con 0x560bcd0134d0 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560bcd2e3730 con 0x560bcd00b2f0 2024-09-21T02:01:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560bcd3ddaf0 con 0x560bcd0341b0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57580/0 (socket says 172.21.15.29:57580) 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f3899436640 1 -- 172.21.15.29:0/2364121684 learned_addr learned my addr 172.21.15.29:0/2364121684 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.470+0000 7f38a2e09740 10 client.0 _ll_register_callbacks cb 0x560bcd3376b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f3899c37640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 -- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 msgr2=0x560bcd3ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3ede30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 -- 172.21.15.29:0/2364121684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 msgr2=0x560bcd3ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 -- 172.21.15.29:0/2364121684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd09f790 con 0x560bcd0341b0 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f3899436640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f389a438640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3ef060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fa0 tx=0x7f388c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cff739dfa029b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f387d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c019b50 con 0x560bcd0341b0 2024-09-21T02:01:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f388c019cf0 con 0x560bcd0341b0 2024-09-21T02:01:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.471+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388c00f4e0 con 0x560bcd0341b0 2024-09-21T02:01:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.472+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f388c009c60 con 0x560bcd0341b0 2024-09-21T02:01:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.475+0000 7f38a2e09740 10 client.27859 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.475+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560bcd118620 con 0x560bcd0341b0 2024-09-21T02:01:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.476+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c02c040 con 0x560bcd0341b0 2024-09-21T02:01:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.476+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f388c03b440 con 0x560bcd0341b0 2024-09-21T02:01:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.476+0000 7f38a2e09740 10 client.27859 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.477+0000 7f38a2e09740 10 client.27859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.477+0000 7f38a2e09740 20 client.27859 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.477+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccff2d80 0x560bcd3fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.477+0000 7f38a2e09740 4 client.27859 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.477+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f388c00f4e0 con 0x560bccff2d80 2024-09-21T02:01:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.480+0000 7f3899c37640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccff2d80 0x560bcd3fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.489+0000 7f3899c37640 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccff2d80 0x560bcd3fb360 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73239ab0809e51f server_cookie=38256099db3d7768 in_seq=0 out_seq=0 2024-09-21T02:01:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.491+0000 7f387d7fa640 10 client.27859 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.491+0000 7f387d7fa640 1 -- 172.21.15.29:0/2364121684 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f388c00f4e0 con 0x560bccff2d80 2024-09-21T02:01:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.491+0000 7f387d7fa640 10 client.27859 handle_command_reply: tid=0 2024-09-21T02:01:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.491+0000 7f38a2e09740 1 client.27859 shutdown 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 msgr2=0x560bcd3ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3ef060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fa0 tx=0x7f388c017000 comp rx=0 tx=0).stop 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 shutdown_connections 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccff2d80 0x560bcd3fb360 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f387d7fa640 0 client.27859 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0341b0 0x560bcd3ef060 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 --2- 172.21.15.29:0/2364121684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd00b2f0 0x560bcd3ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.501+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 >> 172.21.15.29:0/2364121684 conn(0x560bcd028300 msgr2=0x560bcd3d5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.507+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 shutdown_connections 2024-09-21T02:01:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.507+0000 7f38a2e09740 1 -- 172.21.15.29:0/2364121684 wait complete. 2024-09-21T02:01:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.508+0000 7f38a2e09740 20 client.27859 trim_cache size 0 max 16384 2024-09-21T02:01:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.540+0000 7f38a2e09740 1 Processor -- start 2024-09-21T02:01:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.540+0000 7f38a2e09740 1 -- start start 2024-09-21T02:01:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd00b2f0 0x560bcd3d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38941a9400 con 0x560bcd0134d0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38941aaf70 con 0x560bcd0341b0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f38a2e09740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f38941a9ad0 con 0x560bcd00b2f0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f389a438640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f3899436640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57592/0 (socket says 172.21.15.29:57592) 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f389a438640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53592/0 (socket says 172.21.15.29:53592) 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f3899436640 1 -- 172.21.15.29:0/1031368888 learned_addr learned my addr 172.21.15.29:0/1031368888 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.541+0000 7f3899c37640 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd00b2f0 0x560bcd3d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 -- 172.21.15.29:0/1031368888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd00b2f0 msgr2=0x560bcd3d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd00b2f0 0x560bcd3d8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 -- 172.21.15.29:0/1031368888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 msgr2=0x560bcd3ec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 -- 172.21.15.29:0/1031368888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd3ec060 con 0x560bcd0134d0 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f389a438640 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f3899436640 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3884009230 tx=0x7f3884031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde9223c51ec3eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f387f7fe640 1 -- 172.21.15.29:0/1031368888 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884035070 con 0x560bcd0134d0 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f387f7fe640 1 -- 172.21.15.29:0/1031368888 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3884031c40 con 0x560bcd0134d0 2024-09-21T02:01:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.542+0000 7f387f7fe640 1 -- 172.21.15.29:0/1031368888 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884003d70 con 0x560bcd0134d0 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 -- 172.21.15.29:0/1031368888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 msgr2=0x560bcd3d7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3884009230 tx=0x7f3884031ad0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 -- 172.21.15.29:0/1031368888 shutdown_connections 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd0134d0 0x560bcd3d7220 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1031368888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd00b2f0 0x560bcd3d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.544+0000 7f38a2e09740 1 -- 172.21.15.29:0/1031368888 >> 172.21.15.29:0/1031368888 conn(0x560bcd028300 msgr2=0x560bcd3ed050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.545+0000 7f38a2e09740 1 -- 172.21.15.29:0/1031368888 shutdown_connections 2024-09-21T02:01:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.545+0000 7f38a2e09740 1 -- 172.21.15.29:0/1031368888 wait complete. 2024-09-21T02:01:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.545+0000 7f38a2e09740 1 -- start start 2024-09-21T02:01:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.545+0000 7f38a2e09740 1 Processor -- start 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 0x560bcd3f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f38a2e09740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f3899c37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 0x560bcd3f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f3899c37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 0x560bcd3f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57596/0 (socket says 172.21.15.29:57596) 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f3899436640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.546+0000 7f3899436640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:56178/0 (socket says 172.21.15.29:56178) 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899c37640 1 -- 172.21.15.29:0/1400926373 learned_addr learned my addr 172.21.15.29:0/1400926373 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38941aa870 con 0x560bcd00b2f0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38941aa1a0 con 0x560bcd0341b0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f389a438640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f38941a73e0 con 0x560bcd0134d0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f38a2e09740 10 client.0 _ll_register_callbacks cb 0x7f388c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 -- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 msgr2=0x560bcd3f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 -- 172.21.15.29:0/1400926373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 msgr2=0x560bcd3f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 0x560bcd3f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd3f5490 con 0x560bcd0134d0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f389a438640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f3899436640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f3884003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ed90866fa2f367 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.547+0000 7f387d7fa640 10 client.27865 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884031c40 con 0x560bcd0134d0 2024-09-21T02:01:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f387d7fa640 1 -- 172.21.15.29:0/1400926373 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884035040 con 0x560bcd0134d0 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f387d7fa640 1 -- 172.21.15.29:0/1400926373 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3884003ef0 con 0x560bcd0134d0 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f387d7fa640 1 -- 172.21.15.29:0/1400926373 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f38840040f0 con 0x560bcd0134d0 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f38a2e09740 10 client.27865 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.548+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=48}) -- 0x560bcd465e60 con 0x560bcd0134d0 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f387d7fa640 1 -- 172.21.15.29:0/1400926373 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3884004290 con 0x560bcd0134d0 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 10 client.27865 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 10 client.27865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 20 client.27865 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccfc81c0 0x560bcd4687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 4 client.27865 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3884031c40 con 0x560bccfc81c0 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.549+0000 7f3899c37640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccfc81c0 0x560bcd4687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.550+0000 7f3899c37640 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccfc81c0 0x560bcd4687d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed08042a9e76b90e server_cookie=dd0a118c0be6a67 in_seq=0 out_seq=0 2024-09-21T02:01:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.550+0000 7f387d7fa640 10 client.27865 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.557+0000 7f387d7fa640 1 -- 172.21.15.29:0/1400926373 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3884031c40 con 0x560bccfc81c0 2024-09-21T02:01:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.557+0000 7f387d7fa640 10 client.27865 handle_command_reply: tid=0 2024-09-21T02:01:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.557+0000 7f38a2e09740 1 client.27865 shutdown 2024-09-21T02:01:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.558+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 msgr2=0x560bcd3f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.558+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f3884003af0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 shutdown_connections 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560bccfc81c0 0x560bcd4687d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bcd0341b0 0x560bcd3f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bcd0134d0 0x560bcd3f45f0 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f3884003af0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 --2- 172.21.15.29:0/1400926373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560bcd00b2f0 0x560bcd3f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 >> 172.21.15.29:0/1400926373 conn(0x560bcd028300 msgr2=0x560bcd3ed050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f387d7fa640 0 client.27865 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 shutdown_connections 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 1 -- 172.21.15.29:0/1400926373 wait complete. 2024-09-21T02:01:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.559+0000 7f38a2e09740 20 client.27865 trim_cache size 0 max 16384 2024-09-21T02:01:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:01:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:01:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:01:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 msgr2=0x7f38800b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 0x7f38800b8a50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f388c020000 tx=0x7f388c01bd70 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 msgr2=0x7f389419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f389419a0f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f389000e9f0 tx=0x7f389000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 shutdown_connections 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38800b6540 0x7f38800b8a50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389419b220 0x7f389419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3894103ff0 0x7f389419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 --2- 172.21.15.29:0/4092956258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38941025d0 0x7f389419a0f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 >> 172.21.15.29:0/4092956258 conn(0x7f38940fdf30 msgr2=0x7f38940ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 shutdown_connections 2024-09-21T02:01:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:01.560+0000 7f389be9b640 1 -- 172.21.15.29:0/4092956258 wait complete. 2024-09-21T02:01:01.778 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:01 smithi066 ceph-mon[35001]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-09-21T02:01:01.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:01 smithi029 ceph-mon[31228]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-09-21T02:01:01.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:01 smithi190 ceph-mon[34368]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-09-21T02:01:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 745.0 s, 1740.0 tps, lat 58.458 ms stddev 76.443 2024-09-21T02:01:03.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:03 smithi066 ceph-mon[35001]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-09-21T02:01:03.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:03 smithi029 ceph-mon[31228]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-09-21T02:01:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:03 smithi190 ceph-mon[34368]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-09-21T02:01:05.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:05 smithi066 ceph-mon[35001]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 644 op/s 2024-09-21T02:01:05.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:05 smithi029 ceph-mon[31228]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 644 op/s 2024-09-21T02:01:05.907 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:05 smithi190 ceph-mon[34368]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 644 op/s 2024-09-21T02:01:07.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:07 smithi066 ceph-mon[35001]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 707 op/s 2024-09-21T02:01:07.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:07 smithi029 ceph-mon[31228]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 707 op/s 2024-09-21T02:01:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 750.0 s, 1396.2 tps, lat 70.886 ms stddev 82.670 2024-09-21T02:01:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:07 smithi190 ceph-mon[34368]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 707 op/s 2024-09-21T02:01:09.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:09 smithi066 ceph-mon[35001]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-09-21T02:01:09.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:09 smithi029 ceph-mon[31228]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-09-21T02:01:09.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:09 smithi190 ceph-mon[34368]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-09-21T02:01:10.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:01:10.543 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 -- 172.21.15.29:0/3524743844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d0410de10 msgr2=0x7f2d04110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 --2- 172.21.15.29:0/3524743844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d0410de10 0x7f2d04110300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009860 tx=0x7f2cfc02a340 comp rx=0 tx=0).stop 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 -- 172.21.15.29:0/3524743844 shutdown_connections 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 --2- 172.21.15.29:0/3524743844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d0410de10 0x7f2d04110300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 --2- 172.21.15.29:0/3524743844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d0410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 --2- 172.21.15.29:0/3524743844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d041049b0 0x7f2d04104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 -- 172.21.15.29:0/3524743844 >> 172.21.15.29:0/3524743844 conn(0x7f2d04079730 msgr2=0x7f2d04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 -- 172.21.15.29:0/3524743844 shutdown_connections 2024-09-21T02:01:11.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.164+0000 7f2d0a911640 1 -- 172.21.15.29:0/3524743844 wait complete. 2024-09-21T02:01:11.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.165+0000 7f2d0a911640 1 Processor -- start 2024-09-21T02:01:11.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.165+0000 7f2d0a911640 1 -- start start 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d041049b0 0x7f2d04198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d0410de10 0x7f2d0419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111880 con 0x7f2d0410de10 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111700 con 0x7f2d04105300 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d0a911640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111a00 con 0x7f2d041049b0 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d08eae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d0410de10 0x7f2d0419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46206/0 (socket says 172.21.15.29:46206) 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d037fe640 1 -- 172.21.15.29:0/2484266050 learned_addr learned my addr 172.21.15.29:0/2484266050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.166+0000 7f2d03fff640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d041049b0 0x7f2d04198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 -- 172.21.15.29:0/2484266050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d041049b0 msgr2=0x7f2d04198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d041049b0 0x7f2d04198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 -- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d0410de10 msgr2=0x7f2d0419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d0410de10 0x7f2d0419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 -- 172.21.15.29:0/2484266050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0419e1a0 con 0x7f2d04105300 2024-09-21T02:01:11.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.167+0000 7f2d037fe640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bc70 tx=0x7f2cf8008600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.169+0000 7f2d017fa640 1 -- 172.21.15.29:0/2484266050 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8008cb0 con 0x7f2d04105300 2024-09-21T02:01:11.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.169+0000 7f2d017fa640 1 -- 172.21.15.29:0/2484266050 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf8008e50 con 0x7f2d04105300 2024-09-21T02:01:11.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.169+0000 7f2d017fa640 1 -- 172.21.15.29:0/2484266050 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8014700 con 0x7f2d04105300 2024-09-21T02:01:11.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.171+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0419e490 con 0x7f2d04105300 2024-09-21T02:01:11.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.171+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d041a4bf0 con 0x7f2d04105300 2024-09-21T02:01:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.173+0000 7f2d10e85740 1 Processor -- start 2024-09-21T02:01:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 -- start start 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a177b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39daf390 0x557f3a1780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557f3a0ede50 con 0x557f39da71b0 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f3a0dd280 con 0x557f39dd0070 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d10e85740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f3a0e1090 con 0x557f39daf390 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d08eae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d08eae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46218/0 (socket says 172.21.15.29:46218) 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a177b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.174+0000 7f2d08eae640 1 -- 172.21.15.29:0/3449493144 learned_addr learned my addr 172.21.15.29:0/3449493144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d017fa640 1 -- 172.21.15.29:0/2484266050 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2cf8015400 con 0x7f2d04105300 2024-09-21T02:01:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 -- 172.21.15.29:0/3449493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39daf390 msgr2=0x557f3a1780d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39daf390 0x557f3a1780d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 -- 172.21.15.29:0/3449493144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 msgr2=0x557f3a177b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a177b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 -- 172.21.15.29:0/3449493144 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f39e3b650 con 0x557f39dd0070 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a177b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d08eae640 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc035040 tx=0x7f2cfc0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcdfb730536e877 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.175+0000 7f2d017fa640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 0x7f2cec0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.176+0000 7f2d017fa640 1 -- 172.21.15.29:0/2484266050 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2cf80e7100 con 0x7f2d04105300 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.176+0000 7f2d03fff640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 0x7f2cec0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.176+0000 7f2cea7fc640 1 -- 172.21.15.29:0/3449493144 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc037040 con 0x557f39dd0070 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.176+0000 7f2cea7fc640 1 -- 172.21.15.29:0/3449493144 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc030e10 con 0x557f39dd0070 2024-09-21T02:01:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.176+0000 7f2cea7fc640 1 -- 172.21.15.29:0/3449493144 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc037040 con 0x557f39dd0070 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 -- 172.21.15.29:0/3449493144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 msgr2=0x557f3a177650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc035040 tx=0x7f2cfc0095a0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d03fff640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 0x7f2cec0b8b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x557f3a1762d0 tx=0x7f2cf40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 -- 172.21.15.29:0/3449493144 shutdown_connections 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39daf390 0x557f3a1780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a177b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3449493144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a177650 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.177+0000 7f2d10e85740 1 -- 172.21.15.29:0/3449493144 >> 172.21.15.29:0/3449493144 conn(0x557f39dc4460 msgr2=0x557f3a175540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.178+0000 7f2d10e85740 1 -- 172.21.15.29:0/3449493144 shutdown_connections 2024-09-21T02:01:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.178+0000 7f2d10e85740 1 -- 172.21.15.29:0/3449493144 wait complete. 2024-09-21T02:01:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.178+0000 7f2d10e85740 1 -- start start 2024-09-21T02:01:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.178+0000 7f2d10e85740 1 Processor -- start 2024-09-21T02:01:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a183320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a18f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:47246/0 (socket says 172.21.15.29:47246) 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d03fff640 1 -- 172.21.15.29:0/841798544 learned_addr learned my addr 172.21.15.29:0/841798544 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557f39f3e700 con 0x557f39daf390 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f3a078110 con 0x557f39dd0070 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f3a185dd0 con 0x557f39da71b0 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d037fe640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a183320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.179+0000 7f2d08eae640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a18f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d10e85740 10 client.0 _ll_register_callbacks cb 0x557f3a091990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 -- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 msgr2=0x557f3a18f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a18f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 -- 172.21.15.29:0/841798544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 msgr2=0x557f3a183320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a183320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3a0a5c40 con 0x557f39da71b0 2024-09-21T02:01:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d08eae640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a18f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc030e10 con 0x557f39da71b0 2024-09-21T02:01:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d03fff640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4020970 tx=0x7f2cf4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8328499d0752f076 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2ce37fe640 10 client.27874 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2ce37fe640 1 -- 172.21.15.29:0/841798544 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4031030 con 0x557f39da71b0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2ce37fe640 1 -- 172.21.15.29:0/841798544 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2cf4003970 con 0x557f39da71b0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2ce37fe640 1 -- 172.21.15.29:0/841798544 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2cf4003b70 con 0x557f39da71b0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d10e85740 10 client.27874 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.180+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=48}) -- 0x557f3a191ab0 con 0x557f39da71b0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2ce37fe640 1 -- 172.21.15.29:0/841798544 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f2cf401f890 con 0x557f39da71b0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 10 client.27874 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 10 client.27874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 20 client.27874 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d8e9f0 0x557f3a194f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 4 client.27874 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3a0efda0 con 0x557f39d8e9f0 2024-09-21T02:01:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.181+0000 7f2d037fe640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d8e9f0 0x557f3a194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2ce37fe640 10 client.27874 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d037fe640 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d8e9f0 0x557f3a194f40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e0ed9bb69de967 server_cookie=3564a5f07002c316 in_seq=0 out_seq=0 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2ce37fe640 1 -- 172.21.15.29:0/841798544 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f3a0efda0 con 0x557f39d8e9f0 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2ce37fe640 10 client.27874 handle_command_reply: tid=0 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 client.27874 shutdown 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 msgr2=0x557f3a182de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4020970 tx=0x7f2cf4004a40 comp rx=0 tx=0).stop 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 shutdown_connections 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d8e9f0 0x557f3a194f40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a18f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a183320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 --2- 172.21.15.29:0/841798544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a182de0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 >> 172.21.15.29:0/841798544 conn(0x557f39dc4460 msgr2=0x557f3a17a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.182+0000 7f2ce37fe640 0 client.27874 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:11.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.183+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 shutdown_connections 2024-09-21T02:01:11.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.183+0000 7f2d10e85740 1 -- 172.21.15.29:0/841798544 wait complete. 2024-09-21T02:01:11.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.183+0000 7f2d10e85740 20 client.27874 trim_cache size 0 max 16384 2024-09-21T02:01:11.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.218+0000 7f2d10e85740 1 Processor -- start 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 -- start start 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39daf390 0x557f3a184cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39dd0070 0x557f3a18eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ad110 con 0x557f39da71b0 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041aec80 con 0x557f39daf390 2024-09-21T02:01:11.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.219+0000 7f2d10e85740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ad7e0 con 0x557f39dd0070 2024-09-21T02:01:11.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d08eae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39dd0070 0x557f3a18eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36696/0 (socket says 172.21.15.29:36696) 2024-09-21T02:01:11.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d03fff640 1 -- 172.21.15.29:0/3331217895 learned_addr learned my addr 172.21.15.29:0/3331217895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:11.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d03fff640 1 -- 172.21.15.29:0/3331217895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39dd0070 msgr2=0x557f3a18eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.220+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39dd0070 0x557f3a18eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.221+0000 7f2d03fff640 1 -- 172.21.15.29:0/3331217895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39daf390 msgr2=0x557f3a184cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.221+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39daf390 0x557f3a184cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.221+0000 7f2d03fff640 1 -- 172.21.15.29:0/3331217895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3a1812e0 con 0x557f39da71b0 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.221+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40206e0 tx=0x7f2cf402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952e2bbc3fbc7e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.222+0000 7f2ce9ffb640 1 -- 172.21.15.29:0/3331217895 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4044070 con 0x557f39da71b0 2024-09-21T02:01:11.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.222+0000 7f2ce9ffb640 1 -- 172.21.15.29:0/3331217895 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4032040 con 0x557f39da71b0 2024-09-21T02:01:11.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.222+0000 7f2ce9ffb640 1 -- 172.21.15.29:0/3331217895 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4031070 con 0x557f39da71b0 2024-09-21T02:01:11.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.223+0000 7f2d10e85740 1 -- 172.21.15.29:0/3331217895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 msgr2=0x557f3a179150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.223+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40206e0 tx=0x7f2cf402c000 comp rx=0 tx=0).stop 2024-09-21T02:01:11.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.224+0000 7f2d10e85740 1 -- 172.21.15.29:0/3331217895 shutdown_connections 2024-09-21T02:01:11.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.224+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39dd0070 0x557f3a18eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.224+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39daf390 0x557f3a184cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.224+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3331217895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39da71b0 0x557f3a179150 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.224+0000 7f2d10e85740 1 -- 172.21.15.29:0/3331217895 >> 172.21.15.29:0/3331217895 conn(0x557f39dc4460 msgr2=0x557f3a17c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.227+0000 7f2d10e85740 1 -- 172.21.15.29:0/3331217895 shutdown_connections 2024-09-21T02:01:11.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.227+0000 7f2d10e85740 1 -- 172.21.15.29:0/3331217895 wait complete. 2024-09-21T02:01:11.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.228+0000 7f2d10e85740 1 -- start start 2024-09-21T02:01:11.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.228+0000 7f2d10e85740 1 Processor -- start 2024-09-21T02:01:11.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.229+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.229+0000 7f2d10e85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a187c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.229+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:47262/0 (socket says 172.21.15.29:47262) 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d03fff640 1 -- 172.21.15.29:0/3883708706 learned_addr learned my addr 172.21.15.29:0/3883708706 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a1fe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ae580 con 0x557f39daf390 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041adeb0 con 0x557f39dd0070 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ab0f0 con 0x557f39da71b0 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.230+0000 7f2d10e85740 10 client.0 _ll_register_callbacks cb 0x7f2cf4009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d08eae640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a1fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d03fff640 1 -- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 msgr2=0x557f3a1fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a1fe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d03fff640 1 -- 172.21.15.29:0/3883708706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 msgr2=0x557f3a187c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:11.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a187c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.231+0000 7f2d03fff640 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3a1fee40 con 0x557f39da71b0 2024-09-21T02:01:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.237+0000 7f2d03fff640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2cf401fd60 tx=0x7f2cf4004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c47dc4b1a80509d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.237+0000 7f2ce37fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.237+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf404c060 con 0x557f39da71b0 2024-09-21T02:01:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.237+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2cf4005250 con 0x557f39da71b0 2024-09-21T02:01:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.238+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf4032040 con 0x557f39da71b0 2024-09-21T02:01:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.238+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2cf4031070 con 0x557f39da71b0 2024-09-21T02:01:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.238+0000 7f2d10e85740 10 client.27880 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.238+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557f3a200060 con 0x557f39da71b0 2024-09-21T02:01:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc030e10 con 0x557f39da71b0 2024-09-21T02:01:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f2cf40368c0 con 0x557f39da71b0 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 10 client.27880 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 10 client.27880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 20 client.27880 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d63da0 0x557f3a2026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 4 client.27880 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:01:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.239+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2cf4032040 con 0x557f39d63da0 2024-09-21T02:01:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.240+0000 7f2d037fe640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d63da0 0x557f3a2026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.241+0000 7f2ce37fe640 10 client.27880 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.241+0000 7f2d037fe640 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d63da0 0x557f3a2026b0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35649b6dead6d94 server_cookie=f9c9862253e83fd4 in_seq=0 out_seq=0 2024-09-21T02:01:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.241+0000 7f2ce37fe640 1 -- 172.21.15.29:0/3883708706 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x7f2cf4032040 con 0x557f39d63da0 2024-09-21T02:01:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.241+0000 7f2ce37fe640 10 client.27880 handle_command_reply: tid=0 2024-09-21T02:01:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.241+0000 7f2d10e85740 1 client.27880 shutdown 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 msgr2=0x557f3a1876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2cf401fd60 tx=0x7f2cf4004ce0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 shutdown_connections 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x557f39d63da0 0x557f3a2026b0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f39dd0070 0x557f3a1fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f39daf390 0x557f3a187c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2ce37fe640 0 client.27880 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 --2- 172.21.15.29:0/3883708706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f39da71b0 0x557f3a1876f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.242+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 >> 172.21.15.29:0/3883708706 conn(0x557f39dc4460 msgr2=0x557f3a184a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.243+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 shutdown_connections 2024-09-21T02:01:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.243+0000 7f2d10e85740 1 -- 172.21.15.29:0/3883708706 wait complete. 2024-09-21T02:01:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.243+0000 7f2d10e85740 20 client.27880 trim_cache size 0 max 16384 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:01:11.246 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.247 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.250 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:01:11.251 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.252 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.253 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:01:11.255 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:01:11.256 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 msgr2=0x7f2cec0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 0x7f2cec0b8b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x557f3a1762d0 tx=0x7f2cf40095c0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 msgr2=0x7f2d041987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bc70 tx=0x7f2cf8008600 comp rx=0 tx=0).stop 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 shutdown_connections 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2cec0b6660 0x7f2cec0b8b70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d0410de10 0x7f2d0419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d04105300 0x7f2d041987d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 --2- 172.21.15.29:0/2484266050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d041049b0 0x7f2d04198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:11.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 >> 172.21.15.29:0/2484266050 conn(0x7f2d04079730 msgr2=0x7f2d040ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:11.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.246+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 shutdown_connections 2024-09-21T02:01:11.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:11.247+0000 7f2d0a911640 1 -- 172.21.15.29:0/2484266050 wait complete. 2024-09-21T02:01:11.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:11 smithi066 ceph-mon[35001]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-21T02:01:11.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:11 smithi029 ceph-mon[31228]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-21T02:01:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:11 smithi190 ceph-mon[34368]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-21T02:01:12.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:01:12.179 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:01:12.589 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 755.0 s, 1444.4 tps, lat 68.493 ms stddev 84.725 2024-09-21T02:01:13.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 -- 172.21.15.29:0/1193485128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 msgr2=0x7ff2a410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:13.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/1193485128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a410d850 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009860 tx=0x7ff28c02a340 comp rx=0 tx=0).stop 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 -- 172.21.15.29:0/1193485128 shutdown_connections 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/1193485128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a410de10 0x7ff2a4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/1193485128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a410d850 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/1193485128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a41049b0 0x7ff2a4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 -- 172.21.15.29:0/1193485128 >> 172.21.15.29:0/1193485128 conn(0x7ff2a4079730 msgr2=0x7ff2a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 -- 172.21.15.29:0/1193485128 shutdown_connections 2024-09-21T02:01:13.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.273+0000 7ff2a9b95640 1 -- 172.21.15.29:0/1193485128 wait complete. 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 Processor -- start 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 -- start start 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 0x7ff2a419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 0x7ff2a419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4113a70 con 0x7ff2a41049b0 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a9b95640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41138f0 con 0x7ff2a4105300 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.274+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 0x7ff2a419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:13.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 0x7ff2a419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53534/0 (socket says 172.21.15.29:53534) 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff2a37fe640 1 -- 172.21.15.29:0/2981100772 learned_addr learned my addr 172.21.15.29:0/2981100772 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4113bf0 con 0x7ff2a410de10 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff2a3fff640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 0x7ff2a419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 -- 172.21.15.29:0/2981100772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 msgr2=0x7ff2a419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 0x7ff2a419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 -- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 msgr2=0x7ff2a419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 0x7ff2a419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a41a04f0 con 0x7ff2a4105300 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff2a3fff640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 0x7ff2a419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.275+0000 7ff29bfff640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a419ab20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009780 tx=0x7ff28c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:13.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.276+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c037040 con 0x7ff2a4105300 2024-09-21T02:01:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.276+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28c030c10 con 0x7ff2a4105300 2024-09-21T02:01:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.276+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41a0780 con 0x7ff2a4105300 2024-09-21T02:01:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.276+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0035f0 con 0x7ff2a4105300 2024-09-21T02:01:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.276+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a6f40 con 0x7ff2a4105300 2024-09-21T02:01:13.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.278+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a406b560 con 0x7ff2a4105300 2024-09-21T02:01:13.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.279+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff28c043020 con 0x7ff2a4105300 2024-09-21T02:01:13.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.280+0000 7ff2a17fa640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 0x7ff2900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.281+0000 7ff2a37fe640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 0x7ff2900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.281+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff28c106660 con 0x7ff2a4105300 2024-09-21T02:01:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.281+0000 7ff2a37fe640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 0x7ff2900b8aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff2940058c0 tx=0x7ff294005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:13.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.283+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff28c04a060 con 0x7ff2a4105300 2024-09-21T02:01:13.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.658+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2a4104dc0 con 0x7ff2a4105300 2024-09-21T02:01:13.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.659+0000 7ff2a17fa640 1 -- 172.21.15.29:0/2981100772 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11971 (secure 0 0 0) 0x7ff2a4104dc0 con 0x7ff2a4105300 2024-09-21T02:01:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:01:13.660 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":48,"btime":"2024-09-21T02:00:51:076111+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:00:50.250040+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T02:01:13.660 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 48 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 msgr2=0x7ff2900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 0x7ff2900b8aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff2940058c0 tx=0x7ff294005f90 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 msgr2=0x7ff2a419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a419ab20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009780 tx=0x7ff28c032d50 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 shutdown_connections 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff2900b6590 0x7ff2900b8aa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410de10 0x7ff2a419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a4105300 0x7ff2a419ab20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 --2- 172.21.15.29:0/2981100772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2a41049b0 0x7ff2a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 >> 172.21.15.29:0/2981100772 conn(0x7ff2a4079730 msgr2=0x7ff2a4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 shutdown_connections 2024-09-21T02:01:13.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:13.662+0000 7ff2a9b95640 1 -- 172.21.15.29:0/2981100772 wait complete. 2024-09-21T02:01:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:13 smithi066 ceph-mon[35001]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 603 op/s 2024-09-21T02:01:13.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:13 smithi029 ceph-mon[31228]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 603 op/s 2024-09-21T02:01:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:13 smithi190 ceph-mon[34368]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 603 op/s 2024-09-21T02:01:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3698614381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac106c20 msgr2=0x7f2cac109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3698614381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac106c20 0x7f2cac109040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009900 tx=0x7f2ca002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3698614381 shutdown_connections 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3698614381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac109580 0x7f2cac10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3698614381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac106c20 0x7f2cac109040 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3698614381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.697+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3698614381 >> 172.21.15.29:0/3698614381 conn(0x7f2cac0fd690 msgr2=0x7f2cac0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.698+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3698614381 shutdown_connections 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.698+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3698614381 wait complete. 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.698+0000 7f2cb23c3640 1 Processor -- start 2024-09-21T02:01:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 -- start start 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 0x7f2cac19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f350 con 0x7f2cac109580 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f1d0 con 0x7f2cac101ad0 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.699+0000 7f2cb23c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f4d0 con 0x7f2cac106c20 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 0x7f2cac19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46684/0 (socket says 172.21.15.29:46684) 2024-09-21T02:01:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60738/0 (socket says 172.21.15.29:60738) 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 -- 172.21.15.29:0/3787887172 learned_addr learned my addr 172.21.15.29:0/3787887172 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 0x7f2cac19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53552/0 (socket says 172.21.15.29:53552) 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 -- 172.21.15.29:0/3787887172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 msgr2=0x7f2cac19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 -- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 msgr2=0x7f2cac19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 0x7f2cac19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 -- 172.21.15.29:0/3787887172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac19b9c0 con 0x7f2cac101ad0 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2cabfff640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9f0 tx=0x7f2c9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.700+0000 7f2ca97fa640 1 -- 172.21.15.29:0/3787887172 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00cd80 con 0x7f2cac101ad0 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.701+0000 7f2ca97fa640 1 -- 172.21.15.29:0/3787887172 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9c0108e0 con 0x7f2cac101ad0 2024-09-21T02:01:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.701+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac19bcb0 con 0x7f2cac101ad0 2024-09-21T02:01:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.701+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a23a0 con 0x7f2cac101ad0 2024-09-21T02:01:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.701+0000 7f2ca97fa640 1 -- 172.21.15.29:0/3787887172 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c018980 con 0x7f2cac101ad0 2024-09-21T02:01:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.703+0000 7f2cb940d740 1 Processor -- start 2024-09-21T02:01:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.703+0000 7f2ca97fa640 1 -- 172.21.15.29:0/3787887172 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2c9c018b20 con 0x7f2cac101ad0 2024-09-21T02:01:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.703+0000 7f2cb940d740 1 -- start start 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x560059522cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059523820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56005948c6f0 con 0x5600591514d0 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56005947bb20 con 0x5600591721b0 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb940d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56005947f930 con 0x5600591492f0 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x560059522cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46698/0 (socket says 172.21.15.29:46698) 2024-09-21T02:01:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 -- 172.21.15.29:0/650179406 learned_addr learned my addr 172.21.15.29:0/650179406 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059523820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 -- 172.21.15.29:0/650179406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 msgr2=0x560059522cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x560059522cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 -- 172.21.15.29:0/650179406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 msgr2=0x560059523820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059523820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.704+0000 7f2cb0960640 1 -- 172.21.15.29:0/650179406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560059441530 con 0x5600591721b0 2024-09-21T02:01:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2cb0960640 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009fd0 tx=0x7f2ca4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c5024c732230ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2ca97fa640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 0x7f2c940b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2c927fc640 1 -- 172.21.15.29:0/650179406 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4004d20 con 0x5600591721b0 2024-09-21T02:01:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2c927fc640 1 -- 172.21.15.29:0/650179406 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca4004030 con 0x5600591721b0 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2c927fc640 1 -- 172.21.15.29:0/650179406 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4005280 con 0x5600591721b0 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2ca97fa640 1 -- 172.21.15.29:0/3787887172 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2c9c0e7590 con 0x7f2cac101ad0 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.705+0000 7f2cabfff640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 0x7f2c940b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.706+0000 7f2cb940d740 1 -- 172.21.15.29:0/650179406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 msgr2=0x560059522660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.706+0000 7f2cb940d740 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009fd0 tx=0x7f2ca4011810 comp rx=0 tx=0).stop 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.706+0000 7f2cabfff640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 0x7f2c940b8aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0a9db0 tx=0x7f2c9c0ed0e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 -- 172.21.15.29:0/650179406 shutdown_connections 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059523820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x560059522cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 --2- 172.21.15.29:0/650179406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 -- 172.21.15.29:0/650179406 >> 172.21.15.29:0/650179406 conn(0x560059166300 msgr2=0x5600595158e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 -- 172.21.15.29:0/650179406 shutdown_connections 2024-09-21T02:01:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.707+0000 7f2cb940d740 1 -- 172.21.15.29:0/650179406 wait complete. 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.708+0000 7f2cb940d740 1 -- start start 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.708+0000 7f2cb940d740 1 Processor -- start 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.708+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591514d0 0x56005952c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591721b0 0x56005952d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5600593b3ab0 con 0x5600591721b0 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560059421730 con 0x5600591492f0 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560059524340 con 0x5600591514d0 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb940d740 10 client.0 _ll_register_callbacks cb 0x5600594756b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46706/0 (socket says 172.21.15.29:46706) 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 -- 172.21.15.29:0/619311430 learned_addr learned my addr 172.21.15.29:0/619311430 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591514d0 0x56005952c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591721b0 0x56005952d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 -- 172.21.15.29:0/619311430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591514d0 msgr2=0x56005952c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591514d0 0x56005952c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 -- 172.21.15.29:0/619311430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591721b0 msgr2=0x56005952d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591721b0 0x56005952d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 -- 172.21.15.29:0/619311430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600591dd790 con 0x5600591492f0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2cb0960640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009270 tx=0x7f2ca4009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93acd560a128726a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.709+0000 7f2c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca402d060 con 0x5600591492f0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2ca4003c40 con 0x5600591492f0 2024-09-21T02:01:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca4004030 con 0x5600591492f0 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2ca4003e60 con 0x5600591492f0 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 10 client.27947 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560059256620 con 0x5600591492f0 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca4004030 con 0x5600591492f0 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f2ca4024070 con 0x5600591492f0 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 10 client.27947 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 10 client.27947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 20 client.27947 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560059130d80 0x560059539300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 4 client.27947 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.710+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005948e640 con 0x560059130d80 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.711+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560059130d80 0x560059539300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.711+0000 7f2c8b7fe640 10 client.27947 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.711+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560059130d80 0x560059539300 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b748cddd3c8190d server_cookie=454b2bf0ff385457 in_seq=0 out_seq=0 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.712+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/619311430 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56005948e640 con 0x560059130d80 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.712+0000 7f2c8b7fe640 10 client.27947 handle_command_reply: tid=0 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.712+0000 7f2cb940d740 1 client.27947 shutdown 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.712+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 msgr2=0x56005952be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.712+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009270 tx=0x7f2ca4009cd0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 shutdown_connections 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560059130d80 0x560059539300 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591721b0 0x56005952d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591514d0 0x56005952c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 --2- 172.21.15.29:0/619311430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591492f0 0x56005952be20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 >> 172.21.15.29:0/619311430 conn(0x560059166300 msgr2=0x560059513ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2c8b7fe640 0 client.27947 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 shutdown_connections 2024-09-21T02:01:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 1 -- 172.21.15.29:0/619311430 wait complete. 2024-09-21T02:01:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.713+0000 7f2cb940d740 20 client.27947 trim_cache size 0 max 16384 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.731+0000 7f2cb940d740 1 Processor -- start 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 -- start start 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x56005952aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1a8dc0 con 0x5600591514d0 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1aa930 con 0x5600591721b0 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb940d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1a9490 con 0x5600591492f0 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53586/0 (socket says 172.21.15.29:53586) 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cab7fe640 1 -- 172.21.15.29:0/103430431 learned_addr learned my addr 172.21.15.29:0/103430431 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb0960640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x56005952aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.732+0000 7f2cb0960640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 -- 172.21.15.29:0/103430431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 msgr2=0x56005952aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x56005952aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 -- 172.21.15.29:0/103430431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 msgr2=0x560059522b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 -- 172.21.15.29:0/103430431 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560059529fb0 con 0x5600591514d0 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cb0960640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.733+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2ca002a9d0 tx=0x7f2ca00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a396208f1c35b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.734+0000 7f2c91ffb640 1 -- 172.21.15.29:0/103430431 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca004a040 con 0x5600591514d0 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.734+0000 7f2c91ffb640 1 -- 172.21.15.29:0/103430431 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0003bc0 con 0x5600591514d0 2024-09-21T02:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.734+0000 7f2c91ffb640 1 -- 172.21.15.29:0/103430431 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0035040 con 0x5600591514d0 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 -- 172.21.15.29:0/103430431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 msgr2=0x560059522600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2ca002a9d0 tx=0x7f2ca00314a0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 -- 172.21.15.29:0/103430431 shutdown_connections 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591721b0 0x560059522b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591514d0 0x560059522600 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 --2- 172.21.15.29:0/103430431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591492f0 0x56005952aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 -- 172.21.15.29:0/103430431 >> 172.21.15.29:0/103430431 conn(0x560059166300 msgr2=0x56005952a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 -- 172.21.15.29:0/103430431 shutdown_connections 2024-09-21T02:01:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.735+0000 7f2cb940d740 1 -- 172.21.15.29:0/103430431 wait complete. 2024-09-21T02:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 -- start start 2024-09-21T02:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 Processor -- start 2024-09-21T02:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 0x560059531280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 0x560059531940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 0x5600595324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1aa230 con 0x5600591492f0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1a9b60 con 0x5600591061c0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1a6da0 con 0x5600591721b0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.736+0000 7f2cb940d740 10 client.0 _ll_register_callbacks cb 0x7f2ca40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 0x560059531280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cb0960640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 0x560059531280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53596/0 (socket says 172.21.15.29:53596) 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cb0960640 1 -- 172.21.15.29:0/1699983632 learned_addr learned my addr 172.21.15.29:0/1699983632 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cb0960640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 0x5600595324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 0x560059531940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 -- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 msgr2=0x5600595324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 0x5600595324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 -- 172.21.15.29:0/1699983632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 msgr2=0x560059531280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 0x560059531280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cab7fe640 1 -- 172.21.15.29:0/1699983632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560059532c10 con 0x5600591721b0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.737+0000 7f2cb0960640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 0x5600595324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cab7fe640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 0x560059531940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x560059523910 tx=0x7f2ca0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da42bd74be411331 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca005e060 con 0x5600591721b0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0003bc0 con 0x5600591721b0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f2ca00039c0 con 0x5600591721b0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2ca0035050 con 0x5600591721b0 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 10 client.27904 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560059533ee0 con 0x5600591721b0 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0003bc0 con 0x5600591721b0 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f2ca4004460 con 0x5600591721b0 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 10 client.27904 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 10 client.27904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 20 client.27904 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5600595a3fd0 0x5600595a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 4 client.27904 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.738+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56005948a0f0 con 0x5600595a3fd0 2024-09-21T02:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.739+0000 7f2cb0960640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5600595a3fd0 0x5600595a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.739+0000 7f2c8b7fe640 10 client.27904 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.739+0000 7f2cb0960640 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5600595a3fd0 0x5600595a63f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546bec7321328f6f server_cookie=8704dfd985f75769 in_seq=0 out_seq=0 2024-09-21T02:01:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.748+0000 7f2c8b7fe640 1 -- 172.21.15.29:0/1699983632 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56005948a0f0 con 0x5600595a3fd0 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.748+0000 7f2c8b7fe640 10 client.27904 handle_command_reply: tid=0 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.748+0000 7f2cb940d740 1 client.27904 shutdown 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 msgr2=0x560059531940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 0x560059531940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x560059523910 tx=0x7f2ca0009680 comp rx=0 tx=0).stop 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 shutdown_connections 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5600595a3fd0 0x5600595a63f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600591061c0 0x5600595324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2c8b7fe640 0 client.27904 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600591721b0 0x560059531940 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 --2- 172.21.15.29:0/1699983632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600591492f0 0x560059531280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 >> 172.21.15.29:0/1699983632 conn(0x560059166300 msgr2=0x56005952a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 shutdown_connections 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.749+0000 7f2cb940d740 1 -- 172.21.15.29:0/1699983632 wait complete. 2024-09-21T02:01:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb940d740 20 client.27904 trim_cache size 0 max 16384 2024-09-21T02:01:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:01:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:01:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:01:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 msgr2=0x7f2c940b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 0x7f2c940b8aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0a9db0 tx=0x7f2c9c0ed0e0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 msgr2=0x7f2cac199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9f0 tx=0x7f2c9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 shutdown_connections 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f2c940b6590 0x7f2c940b8aa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cac109580 0x7f2cac19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac106c20 0x7f2cac19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 --2- 172.21.15.29:0/3787887172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac101ad0 0x7f2cac199f80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 >> 172.21.15.29:0/3787887172 conn(0x7f2cac0fd690 msgr2=0x7f2cac107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.750+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 shutdown_connections 2024-09-21T02:01:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:14.751+0000 7f2cb23c3640 1 -- 172.21.15.29:0/3787887172 wait complete. 2024-09-21T02:01:14.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:14 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2981100772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:14.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:14 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2981100772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:14.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:14 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2981100772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:15.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:01:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:15.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:15 smithi066 ceph-mon[35001]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T02:01:15.820 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:15 smithi029 ceph-mon[31228]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T02:01:15.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:15 smithi190 ceph-mon[34368]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T02:01:16.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 -- 172.21.15.29:0/238634216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39041049b0 msgr2=0x7f3904104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 --2- 172.21.15.29:0/238634216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39041049b0 0x7f3904104dc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3900009900 tx=0x7f390002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 -- 172.21.15.29:0/238634216 shutdown_connections 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 --2- 172.21.15.29:0/238634216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f390410de10 0x7f3904110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 --2- 172.21.15.29:0/238634216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 0x7f390410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 --2- 172.21.15.29:0/238634216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39041049b0 0x7f3904104dc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 -- 172.21.15.29:0/238634216 >> 172.21.15.29:0/238634216 conn(0x7f3904079730 msgr2=0x7f3904079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 -- 172.21.15.29:0/238634216 shutdown_connections 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.310+0000 7f390bb1e640 1 -- 172.21.15.29:0/238634216 wait complete. 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 Processor -- start 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 -- start start 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 0x7f390419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3904113a90 con 0x7f390410de10 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3904113910 con 0x7f3904105300 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390bb1e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3904113c10 con 0x7f39041049b0 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f390a0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f39098ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.311+0000 7f39090b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 0x7f390419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53614/0 (socket says 172.21.15.29:53614) 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 -- 172.21.15.29:0/2386683129 learned_addr learned my addr 172.21.15.29:0/2386683129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f39098ba640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60796/0 (socket says 172.21.15.29:60796) 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 -- 172.21.15.29:0/2386683129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 msgr2=0x7f390419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 -- 172.21.15.29:0/2386683129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 msgr2=0x7f390419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 0x7f390419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 -- 172.21.15.29:0/2386683129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041a0430 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f39098ba640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390a0bb640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00eb10 tx=0x7f38fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f38faffd640 1 -- 172.21.15.29:0/2386683129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc004b20 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f38faffd640 1 -- 172.21.15.29:0/2386683129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc004cc0 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041a0720 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f38faffd640 1 -- 172.21.15.29:0/2386683129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc010670 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.312+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a6e80 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.315+0000 7f3912092740 1 Processor -- start 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.315+0000 7f38faffd640 1 -- 172.21.15.29:0/2386683129 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f38fc005020 con 0x7f390410de10 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.315+0000 7f3912092740 1 -- start start 2024-09-21T02:01:16.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848c2070 0x55e984c692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848a1390 0x55e984c69d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e984bdfea0 con 0x55e9848a1390 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e984bcf2d0 con 0x55e9848991b0 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f3912092740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e984bd30e0 con 0x55e9848c2070 2024-09-21T02:01:16.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39090b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848c2070 0x55e984c692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46740/0 (socket says 172.21.15.29:46740) 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39090b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848a1390 0x55e984c69d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 -- 172.21.15.29:0/2431171823 learned_addr learned my addr 172.21.15.29:0/2431171823 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 -- 172.21.15.29:0/2431171823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848c2070 msgr2=0x55e984c692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f38faffd640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 0x7f38f00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848c2070 0x55e984c692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 -- 172.21.15.29:0/2431171823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848a1390 msgr2=0x55e984c69d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848a1390 0x55e984c69d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 -- 172.21.15.29:0/2431171823 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98492d650 con 0x55e9848991b0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f390a0bb640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 0x7f38f00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f39098ba640 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f390406bc60 tx=0x7f3900005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64ada49df596d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.316+0000 7f38faffd640 1 -- 172.21.15.29:0/2386683129 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f38fc0e7790 con 0x7f390410de10 2024-09-21T02:01:16.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.317+0000 7f38effff640 1 -- 172.21.15.29:0/2431171823 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900040070 con 0x55e9848991b0 2024-09-21T02:01:16.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.317+0000 7f38effff640 1 -- 172.21.15.29:0/2431171823 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000317a0 con 0x55e9848991b0 2024-09-21T02:01:16.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.317+0000 7f38effff640 1 -- 172.21.15.29:0/2431171823 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035040 con 0x55e9848991b0 2024-09-21T02:01:16.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- 172.21.15.29:0/2431171823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 msgr2=0x55e984c69800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f390a0bb640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 0x7f38f00b8aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0ad630 tx=0x7f38fc00cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f390406bc60 tx=0x7f3900005ed0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- 172.21.15.29:0/2431171823 shutdown_connections 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848a1390 0x55e984c69d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c69800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 --2- 172.21.15.29:0/2431171823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848c2070 0x55e984c692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- 172.21.15.29:0/2431171823 >> 172.21.15.29:0/2431171823 conn(0x55e9848b6460 msgr2=0x55e984c671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- 172.21.15.29:0/2431171823 shutdown_connections 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- 172.21.15.29:0/2431171823 wait complete. 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 -- start start 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.318+0000 7f3912092740 1 Processor -- start 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c78d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848a1390 0x55e984c792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e984a30700 con 0x55e9848c2070 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e984b6a160 con 0x55e9848991b0 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e984c7bd70 con 0x55e9848a1390 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f3912092740 10 client.0 _ll_register_callbacks cb 0x55e984b839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c78d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53642/0 (socket says 172.21.15.29:53642) 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 -- 172.21.15.29:0/2353146791 learned_addr learned my addr 172.21.15.29:0/2353146791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39098ba640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848a1390 0x55e984c792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 -- 172.21.15.29:0/2353146791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848a1390 msgr2=0x55e984c792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848a1390 0x55e984c792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 -- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 msgr2=0x55e984c78d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c78d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.319+0000 7f39090b9640 1 -- 172.21.15.29:0/2353146791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e984b97c90 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f39090b9640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f390406ad30 tx=0x7f38f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6225ee1047caad8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f38ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f400dea0 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f38f4004020 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39000317a0 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f38f4004220 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f3912092740 10 client.18141 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e984c87b50 con 0x55e9848c2070 2024-09-21T02:01:16.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.320+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000317a0 con 0x55e9848c2070 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f38f40049a0 con 0x55e9848c2070 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 10 client.18141 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 10 client.18141 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 20 client.18141 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e9848809f0 0x55e984c8b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 4 client.18141 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e984be1df0 con 0x55e9848809f0 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f39098ba640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e9848809f0 0x55e984c8b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f38ed7fa640 10 client.18141 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.321+0000 7f39098ba640 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e9848809f0 0x55e984c8b150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e49ad303cebf8623 server_cookie=f622ed63859657df in_seq=0 out_seq=0 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2353146791 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e984be1df0 con 0x55e9848809f0 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f38ed7fa640 10 client.18141 handle_command_reply: tid=0 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 client.18141 shutdown 2024-09-21T02:01:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 msgr2=0x55e984c850d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f390406ad30 tx=0x7f38f4004730 comp rx=0 tx=0).stop 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 shutdown_connections 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e9848809f0 0x55e984c8b150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c850d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848a1390 0x55e984c792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 --2- 172.21.15.29:0/2353146791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848991b0 0x55e984c78d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f38ed7fa640 0 client.18141 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:01:16.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 >> 172.21.15.29:0/2353146791 conn(0x55e9848b6460 msgr2=0x55e984c65750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.322+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 shutdown_connections 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.323+0000 7f3912092740 1 -- 172.21.15.29:0/2353146791 wait complete. 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.323+0000 7f3912092740 20 client.18141 trim_cache size 0 max 16384 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 Processor -- start 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 -- start start 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c77c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ad1d0 con 0x55e9848c2070 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f39041aed40 con 0x55e9848a1390 2024-09-21T02:01:16.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.341+0000 7f3912092740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ad8a0 con 0x55e9848991b0 2024-09-21T02:01:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.346+0000 7f39090b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.346+0000 7f39098ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c77c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.346+0000 7f39090b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60806/0 (socket says 172.21.15.29:60806) 2024-09-21T02:01:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.346+0000 7f39090b9640 1 -- 172.21.15.29:0/1877126144 learned_addr learned my addr 172.21.15.29:0/1877126144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 -- 172.21.15.29:0/1877126144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 msgr2=0x55e984c77c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c77c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 -- 172.21.15.29:0/1877126144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 msgr2=0x55e984c6ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c6ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.347+0000 7f39090b9640 1 -- 172.21.15.29:0/1877126144 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e984c6f290 con 0x55e9848991b0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.349+0000 7f39090b9640 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b510 tx=0x7f38f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9637ecc70a56ac27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.351+0000 7f38ef7fe640 1 -- 172.21.15.29:0/1877126144 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4016030 con 0x55e9848991b0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.354+0000 7f38ef7fe640 1 -- 172.21.15.29:0/1877126144 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f4004440 con 0x55e9848991b0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.354+0000 7f38ef7fe640 1 -- 172.21.15.29:0/1877126144 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4005360 con 0x55e9848991b0 2024-09-21T02:01:16.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 -- 172.21.15.29:0/1877126144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 msgr2=0x55e984c73620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b510 tx=0x7f38f40046c0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 -- 172.21.15.29:0/1877126144 shutdown_connections 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c77c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 --2- 172.21.15.29:0/1877126144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.355+0000 7f3912092740 1 -- 172.21.15.29:0/1877126144 >> 172.21.15.29:0/1877126144 conn(0x55e9848b6460 msgr2=0x55e984c6e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.360+0000 7f3912092740 1 -- 172.21.15.29:0/1877126144 shutdown_connections 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.361+0000 7f3912092740 1 -- 172.21.15.29:0/1877126144 wait complete. 2024-09-21T02:01:16.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.361+0000 7f3912092740 1 -- start start 2024-09-21T02:01:16.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.361+0000 7f3912092740 1 Processor -- start 2024-09-21T02:01:16.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ae640 con 0x55e9848c2070 2024-09-21T02:01:16.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f39041adf70 con 0x55e9848a1390 2024-09-21T02:01:16.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ab1b0 con 0x55e9848991b0 2024-09-21T02:01:16.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.363+0000 7f3912092740 10 client.0 _ll_register_callbacks cb 0x7f38f4012b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:16.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.366+0000 7f39098ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.366+0000 7f39098ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:46780/0 (socket says 172.21.15.29:46780) 2024-09-21T02:01:16.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.366+0000 7f39098ba640 1 -- 172.21.15.29:0/2704467529 learned_addr learned my addr 172.21.15.29:0/2704467529 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:16.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.367+0000 7f39090b9640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.367+0000 7f39090b9640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.368+0000 7f39098ba640 1 -- 172.21.15.29:0/2704467529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 msgr2=0x55e984c73200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.368+0000 7f39098ba640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.368+0000 7f39098ba640 1 -- 172.21.15.29:0/2704467529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 msgr2=0x55e984c81540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:16.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.368+0000 7f39098ba640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c81540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.368+0000 7f39098ba640 1 -- 172.21.15.29:0/2704467529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e984c81b00 con 0x55e9848a1390 2024-09-21T02:01:16.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.370+0000 7f39098ba640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3900005530 tx=0x7f3900032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4193a73f8380e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.370+0000 7f38ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:16.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.370+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390005a060 con 0x55e9848a1390 2024-09-21T02:01:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.370+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f4004440 con 0x55e9848a1390 2024-09-21T02:01:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.370+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f39000317f0 con 0x55e9848a1390 2024-09-21T02:01:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.371+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3900035040 con 0x55e9848a1390 2024-09-21T02:01:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.371+0000 7f3912092740 10 client.27971 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.371+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e984c714d0 con 0x55e9848a1390 2024-09-21T02:01:16.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.371+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f4004440 con 0x55e9848a1390 2024-09-21T02:01:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3900003ae0 con 0x55e9848a1390 2024-09-21T02:01:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 10 client.27971 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 10 client.27971 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:01:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 20 client.27971 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e984855da0 0x55e984cf4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 4 client.27971 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:01:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.372+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e984bdd8a0 con 0x55e984855da0 2024-09-21T02:01:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.374+0000 7f39090b9640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e984855da0 0x55e984cf4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.375+0000 7f39090b9640 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e984855da0 0x55e984cf4080 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65e9f3ac2d53e98 server_cookie=2b282e5f8958f5ac in_seq=0 out_seq=0 2024-09-21T02:01:16.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.375+0000 7f38ed7fa640 10 client.27971 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:01:16.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.375+0000 7f38ed7fa640 1 -- 172.21.15.29:0/2704467529 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e984bdd8a0 con 0x55e984855da0 2024-09-21T02:01:16.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.376+0000 7f38ed7fa640 10 client.27971 handle_command_reply: tid=0 2024-09-21T02:01:16.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.376+0000 7f3912092740 1 client.27971 shutdown 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.377+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 msgr2=0x55e984c81000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.377+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3900005530 tx=0x7f3900032f90 comp rx=0 tx=0).stop 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 shutdown_connections 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55e984855da0 0x55e984cf4080 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e9848c2070 0x55e984c81540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9848a1390 0x55e984c81000 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3900005530 tx=0x7f3900032f90 comp rx=0 tx=0).stop 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 --2- 172.21.15.29:0/2704467529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9848991b0 0x55e984c73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 >> 172.21.15.29:0/2704467529 conn(0x55e9848b6460 msgr2=0x55e984c6a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f38ed7fa640 0 client.27971 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:01:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 shutdown_connections 2024-09-21T02:01:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 1 -- 172.21.15.29:0/2704467529 wait complete. 2024-09-21T02:01:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.378+0000 7f3912092740 20 client.27971 trim_cache size 0 max 16384 2024-09-21T02:01:16.381 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:01:16.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 msgr2=0x7f38f00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 0x7f38f00b8aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0ad630 tx=0x7f38fc00cf70 comp rx=0 tx=0).stop 2024-09-21T02:01:16.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 msgr2=0x7f390419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00eb10 tx=0x7f38fc00c2d0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 shutdown_connections 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f38f00b6590 0x7f38f00b8aa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f390410de10 0x7f390419fd00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3904105300 0x7f390419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 --2- 172.21.15.29:0/2386683129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39041049b0 0x7f390419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 >> 172.21.15.29:0/2386683129 conn(0x7f3904079730 msgr2=0x7f3904100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 shutdown_connections 2024-09-21T02:01:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:16.382+0000 7f390bb1e640 1 -- 172.21.15.29:0/2386683129 wait complete. 2024-09-21T02:01:17.255 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:01:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:17.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:17 smithi029 ceph-mon[31228]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-09-21T02:01:17.970 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 760.0 s, 852.8 tps, lat 92.110 ms stddev 147.614 2024-09-21T02:01:17.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:17 smithi190 ceph-mon[34368]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-09-21T02:01:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:17 smithi066 ceph-mon[35001]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 -- 172.21.15.29:0/1826030490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5508104a30 msgr2=0x7f5508104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 --2- 172.21.15.29:0/1826030490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5508104a30 0x7f5508104e40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802a410 comp rx=0 tx=0).stop 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 -- 172.21.15.29:0/1826030490 shutdown_connections 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 --2- 172.21.15.29:0/1826030490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f550810de60 0x7f5508110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 --2- 172.21.15.29:0/1826030490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 --2- 172.21.15.29:0/1826030490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5508104a30 0x7f5508104e40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 -- 172.21.15.29:0/1826030490 >> 172.21.15.29:0/1826030490 conn(0x7f55080797b0 msgr2=0x7f5508079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 -- 172.21.15.29:0/1826030490 shutdown_connections 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.462+0000 7f550d909640 1 -- 172.21.15.29:0/1826030490 wait complete. 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 Processor -- start 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 -- start start 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 0x7f550819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5508113ac0 con 0x7f550810de60 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5508113940 con 0x7f5508104a30 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f550d909640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5508113c40 con 0x7f5508105380 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.463+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53676/0 (socket says 172.21.15.29:53676) 2024-09-21T02:01:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55077fe640 1 -- 172.21.15.29:0/2987335030 learned_addr learned my addr 172.21.15.29:0/2987335030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 0x7f550819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 -- 172.21.15.29:0/2987335030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 msgr2=0x7f550819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 0x7f550819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 -- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 msgr2=0x7f550819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 -- 172.21.15.29:0/2987335030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55081a0400 con 0x7f5508105380 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 0x7f550819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55077fe640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f55067fc640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550819aa30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f550806ad30 tx=0x7f54fc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f550c907640 1 -- 172.21.15.29:0/2987335030 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00cc40 con 0x7f5508105380 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55081a06f0 con 0x7f5508105380 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f550c907640 1 -- 172.21.15.29:0/2987335030 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54fc00cde0 con 0x7f5508105380 2024-09-21T02:01:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.464+0000 7f550c907640 1 -- 172.21.15.29:0/2987335030 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc004a00 con 0x7f5508105380 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.465+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a6e50 con 0x7f5508105380 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 Processor -- start 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 -- start start 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f550c907640 1 -- 172.21.15.29:0/2987335030 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f54fc004ba0 con 0x7f5508105380 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc554be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc555120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc4caea0 con 0x55a1dc1ad070 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc4ba2d0 con 0x55a1dc18c390 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f5513e7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc4be0e0 con 0x55a1dc1841b0 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53692/0 (socket says 172.21.15.29:53692) 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.467+0000 7f55077fe640 1 -- 172.21.15.29:0/1943623823 learned_addr learned my addr 172.21.15.29:0/1943623823 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f5506ffd640 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc554be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55067fc640 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc555120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 -- 172.21.15.29:0/1943623823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 msgr2=0x55a1dc554be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc554be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 -- 172.21.15.29:0/1943623823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 msgr2=0x55a1dc555120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc555120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 -- 172.21.15.29:0/1943623823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1dc218650 con 0x55a1dc1ad070 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f55077fe640 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f55000118b0 tx=0x7f5500011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412ed55f4eba93eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f54edffb640 1 -- 172.21.15.29:0/1943623823 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550001e070 con 0x55a1dc1ad070 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f54edffb640 1 -- 172.21.15.29:0/1943623823 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550000fe10 con 0x55a1dc1ad070 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.468+0000 7f54edffb640 1 -- 172.21.15.29:0/1943623823 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500019d00 con 0x55a1dc1ad070 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f550c907640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 0x7f54f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 0x7f54f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f550c907640 1 -- 172.21.15.29:0/2987335030 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54fc0e7270 con 0x7f5508105380 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 -- 172.21.15.29:0/1943623823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 msgr2=0x55a1dc5546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f55000118b0 tx=0x7f5500011d70 comp rx=0 tx=0).stop 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 -- 172.21.15.29:0/1943623823 shutdown_connections 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc555120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc554be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1943623823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5546a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 -- 172.21.15.29:0/1943623823 >> 172.21.15.29:0/1943623823 conn(0x55a1dc1a1460 msgr2=0x55a1dc552590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 0x7f54f00b8a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54f8031440 tx=0x7f54f8032ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 -- 172.21.15.29:0/1943623823 shutdown_connections 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.469+0000 7f5513e7d740 1 -- 172.21.15.29:0/1943623823 wait complete. 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 -- start start 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 Processor -- start 2024-09-21T02:01:18.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc5645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 0x55a1dc564b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc31b700 con 0x55a1dc1841b0 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc455160 con 0x55a1dc1ad070 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.470+0000 7f5513e7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1dc5671e0 con 0x55a1dc18c390 2024-09-21T02:01:18.540 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5513e7d740 10 client.0 _ll_register_callbacks cb 0x55a1dc46e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53706/0 (socket says 172.21.15.29:53706) 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 -- 172.21.15.29:0/3109135452 learned_addr learned my addr 172.21.15.29:0/3109135452 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f55077fe640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 0x55a1dc564b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f55067fc640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc5645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 -- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 msgr2=0x55a1dc5645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc5645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 -- 172.21.15.29:0/3109135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 msgr2=0x55a1dc564b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 0x55a1dc564b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 -- 172.21.15.29:0/3109135452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1dc482c90 con 0x55a1dc1841b0 2024-09-21T02:01:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f55077fe640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 0x55a1dc564b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5506ffd640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f54f8033000 tx=0x7f54f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962c947b38e083c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f54e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f804d030 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550000fe10 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.471+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54f80055c0 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f54f8035040 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 10 client.18159 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a1dc572e80 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550000fe10 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f54f803d940 con 0x55a1dc1841b0 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 10 client.18159 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 10 client.18159 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:01:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 20 client.18159 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc16b9f0 0x55a1dc5763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 4 client.18159 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1dc4ccdf0 con 0x55a1dc16b9f0 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.472+0000 7f55067fc640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc16b9f0 0x55a1dc5763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.473+0000 7f55067fc640 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc16b9f0 0x55a1dc5763c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db77789d93452948 server_cookie=c53d688ef044c125 in_seq=0 out_seq=0 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.473+0000 7f54e6ffd640 10 client.18159 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f54e6ffd640 1 -- 172.21.15.29:0/3109135452 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1dc4ccdf0 con 0x55a1dc16b9f0 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f54e6ffd640 10 client.18159 handle_command_reply: tid=0 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 client.18159 shutdown 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 msgr2=0x55a1dc5640a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f54f8033000 tx=0x7f54f8009260 comp rx=0 tx=0).stop 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 shutdown_connections 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1ad070 0x55a1dc564b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc16b9f0 0x55a1dc5763c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc5645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 --2- 172.21.15.29:0/3109135452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1841b0 0x55a1dc5640a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f54e6ffd640 0 client.18159 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 >> 172.21.15.29:0/3109135452 conn(0x55a1dc1a1460 msgr2=0x55a1dc550bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 shutdown_connections 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 1 -- 172.21.15.29:0/3109135452 wait complete. 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.474+0000 7f5513e7d740 20 client.18159 trim_cache size 0 max 16384 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.487+0000 7f5513e7d740 1 Processor -- start 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.487+0000 7f5513e7d740 1 -- start start 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1841b0 0x55a1dc552510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc555de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ad030 con 0x55a1dc1ad070 2024-09-21T02:01:18.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55081aeba0 con 0x55a1dc1841b0 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5513e7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ad700 con 0x55a1dc18c390 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53710/0 (socket says 172.21.15.29:53710) 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 -- 172.21.15.29:0/1336551810 learned_addr learned my addr 172.21.15.29:0/1336551810 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55067fc640 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc555de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f5506ffd640 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1841b0 0x55a1dc552510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 -- 172.21.15.29:0/1336551810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 msgr2=0x55a1dc555de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc555de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 -- 172.21.15.29:0/1336551810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1841b0 msgr2=0x55a1dc552510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1841b0 0x55a1dc552510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 -- 172.21.15.29:0/1336551810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1dc552a50 con 0x55a1dc1ad070 2024-09-21T02:01:18.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.488+0000 7f55077fe640 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5500019910 tx=0x7f550000fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d5b904af771a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.489+0000 7f54ed7fa640 1 -- 172.21.15.29:0/1336551810 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500028020 con 0x55a1dc1ad070 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.489+0000 7f54ed7fa640 1 -- 172.21.15.29:0/1336551810 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550001f040 con 0x55a1dc1ad070 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.489+0000 7f54ed7fa640 1 -- 172.21.15.29:0/1336551810 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550001e070 con 0x55a1dc1ad070 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.489+0000 7f5513e7d740 1 -- 172.21.15.29:0/1336551810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 msgr2=0x55a1dc561290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.489+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5500019910 tx=0x7f550000fe80 comp rx=0 tx=0).stop 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 -- 172.21.15.29:0/1336551810 shutdown_connections 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc561290 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc18c390 0x55a1dc555de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 --2- 172.21.15.29:0/1336551810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc1841b0 0x55a1dc552510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 -- 172.21.15.29:0/1336551810 >> 172.21.15.29:0/1336551810 conn(0x55a1dc1a1460 msgr2=0x55a1dc5556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 -- 172.21.15.29:0/1336551810 shutdown_connections 2024-09-21T02:01:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 -- 172.21.15.29:0/1336551810 wait complete. 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 -- start start 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 Processor -- start 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.490+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc568b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc569080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ae4a0 con 0x55a1dc1ad070 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55081addd0 con 0x55a1dc18c390 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ab030 con 0x55a1dc1841b0 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5513e7d740 10 client.0 _ll_register_callbacks cb 0x7f54f80317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc568b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc569080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc568b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60884/0 (socket says 172.21.15.29:60884) 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5dfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53718/0 (socket says 172.21.15.29:53718) 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f5506ffd640 1 -- 172.21.15.29:0/2457284183 learned_addr learned my addr 172.21.15.29:0/2457284183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 -- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 msgr2=0x55a1dc568b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc568b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 -- 172.21.15.29:0/2457284183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 msgr2=0x55a1dc5dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 -- 172.21.15.29:0/2457284183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1dc5e0300 con 0x55a1dc18c390 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f55067fc640 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc569080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0e4d20 tx=0x7f54fc0053f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3366f3a58a70d65d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.491+0000 7f54e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc01c040 con 0x55a1dc18c390 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54fc005020 con 0x55a1dc18c390 2024-09-21T02:01:18.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550001f040 con 0x55a1dc18c390 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f54fc0acec0 con 0x55a1dc18c390 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 10 client.27989 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a1dc5e1540 con 0x55a1dc18c390 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550001f040 con 0x55a1dc18c390 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f54fc0ac610 con 0x55a1dc18c390 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 10 client.27989 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 10 client.27989 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 20 client.27989 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc140da0 0x55a1dc5e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 4 client.27989 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.492+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1dc4c88a0 con 0x55a1dc140da0 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc140da0 0x55a1dc5e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f54e6ffd640 10 client.27989 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:01:18.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f5506ffd640 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc140da0 0x55a1dc5e3d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4923f4770a7ee105 server_cookie=8a262c03f7911e68 in_seq=0 out_seq=0 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f54e6ffd640 1 -- 172.21.15.29:0/2457284183 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1dc4c88a0 con 0x55a1dc140da0 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f54e6ffd640 10 client.27989 handle_command_reply: tid=0 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f5513e7d740 1 client.27989 shutdown 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 msgr2=0x55a1dc569080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.493+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc569080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0e4d20 tx=0x7f54fc0053f0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 shutdown_connections 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1dc1ad070 0x55a1dc5dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1dc18c390 0x55a1dc569080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55a1dc140da0 0x55a1dc5e3d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 --2- 172.21.15.29:0/2457284183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1dc1841b0 0x55a1dc568b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f54e6ffd640 0 client.27989 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 >> 172.21.15.29:0/2457284183 conn(0x55a1dc1a1460 msgr2=0x55a1dc559840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 shutdown_connections 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 1 -- 172.21.15.29:0/2457284183 wait complete. 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f5513e7d740 20 client.27989 trim_cache size 0 max 16384 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 msgr2=0x7f54f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 0x7f54f00b8a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54f8031440 tx=0x7f54f8032ed0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 msgr2=0x7f550819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.494+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550819aa30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f550806ad30 tx=0x7f54fc00ee50 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 shutdown_connections 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f54f00b6510 0x7f54f00b8a20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550810de60 0x7f550819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5508105380 0x7f550819aa30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 --2- 172.21.15.29:0/2987335030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508104a30 0x7f550819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 >> 172.21.15.29:0/2987335030 conn(0x7f55080797b0 msgr2=0x7f5508100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 shutdown_connections 2024-09-21T02:01:18.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:18.495+0000 7f550d909640 1 -- 172.21.15.29:0/2987335030 wait complete. 2024-09-21T02:01:19.820 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:19 smithi029 ceph-mon[31228]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.6 MiB/s wr, 591 op/s 2024-09-21T02:01:19.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:01:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:19 smithi190 ceph-mon[34368]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.6 MiB/s wr, 591 op/s 2024-09-21T02:01:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:19 smithi066 ceph-mon[35001]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.6 MiB/s wr, 591 op/s 2024-09-21T02:01:20.120 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 -- 172.21.15.29:0/520188303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f78108cf0 msgr2=0x7f3f7810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 --2- 172.21.15.29:0/520188303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f78108cf0 0x7f3f7810f520 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009ca0 tx=0x7f3f6802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 -- 172.21.15.29:0/520188303 shutdown_connections 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 --2- 172.21.15.29:0/520188303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f78108cf0 0x7f3f7810f520 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 --2- 172.21.15.29:0/520188303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f781083a0 0x7f3f781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 --2- 172.21.15.29:0/520188303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f78102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 -- 172.21.15.29:0/520188303 >> 172.21.15.29:0/520188303 conn(0x7f3f78079730 msgr2=0x7f3f78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 -- 172.21.15.29:0/520188303 shutdown_connections 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.624+0000 7f3f802fe640 1 -- 172.21.15.29:0/520188303 wait complete. 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 Processor -- start 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 -- start start 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f7810ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f78108cf0 0x7f3f78110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78113a80 con 0x7f3f781083a0 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78113900 con 0x7f3f78102a60 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f802fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78113c00 con 0x7f3f78108cf0 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f7d899640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f7d899640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53736/0 (socket says 172.21.15.29:53736) 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f7d899640 1 -- 172.21.15.29:0/3676337522 learned_addr learned my addr 172.21.15.29:0/3676337522 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f7e89b640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f78108cf0 0x7f3f78110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.625+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f7810ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 -- 172.21.15.29:0/3676337522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f78108cf0 msgr2=0x7f3f78110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f78108cf0 0x7f3f78110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 -- 172.21.15.29:0/3676337522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 msgr2=0x7f3f7810ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f7810ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 -- 172.21.15.29:0/3676337522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f781a86d0 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f7810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f7d899640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d770 tx=0x7f3f6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f777fe640 1 -- 172.21.15.29:0/3676337522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c017070 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f777fe640 1 -- 172.21.15.29:0/3676337522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f6c009c30 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f777fe640 1 -- 172.21.15.29:0/3676337522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00f620 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f781a89c0 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.626+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f781a8f00 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 Processor -- start 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 -- start start 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f777fe640 1 -- 172.21.15.29:0/3676337522 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3f6c01a070 con 0x7f3f781083a0 2024-09-21T02:01:20.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.629+0000 7f3f86872740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557f5c64eea0 con 0x557f5c331070 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f86872740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f5c63e2d0 con 0x557f5c310390 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f86872740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f5c6420e0 con 0x557f5c3081b0 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e89b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60896/0 (socket says 172.21.15.29:60896) 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1534848607 learned_addr learned my addr 172.21.15.29:0/1534848607 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e89b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6d86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53744/0 (socket says 172.21.15.29:53744) 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1534848607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 msgr2=0x557f5c6d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1534848607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 msgr2=0x557f5c6d86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6d86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1534848607 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5c39c650 con 0x557f5c3081b0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ce0 tx=0x7f3f700117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7820495a3ee04720 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f74d9a640 1 -- 172.21.15.29:0/1534848607 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70012450 con 0x557f5c3081b0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f74d9a640 1 -- 172.21.15.29:0/1534848607 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f70012aa0 con 0x557f5c3081b0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.630+0000 7f3f74d9a640 1 -- 172.21.15.29:0/1534848607 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000fbd0 con 0x557f5c3081b0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f777fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 0x7f3f600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f7e89b640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 0x7f3f600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f777fe640 1 -- 172.21.15.29:0/3676337522 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3f6c0e6e10 con 0x7f3f781083a0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f86872740 1 -- 172.21.15.29:0/1534848607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 msgr2=0x557f5c6d8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f7e89b640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 0x7f3f600b8a20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802f500 tx=0x7f3f6802f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.631+0000 7f3f86872740 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ce0 tx=0x7f3f700117b0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- 172.21.15.29:0/1534848607 shutdown_connections 2024-09-21T02:01:20.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6d8be0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- 172.21.15.29:0/1534848607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- 172.21.15.29:0/1534848607 >> 172.21.15.29:0/1534848607 conn(0x557f5c325460 msgr2=0x557f5c6d6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- 172.21.15.29:0/1534848607 shutdown_connections 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- 172.21.15.29:0/1534848607 wait complete. 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- start start 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 Processor -- start 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557f5c49f700 con 0x557f5c331070 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f5c5d9160 con 0x557f5c310390 2024-09-21T02:01:20.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.632+0000 7f3f86872740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f5c6e6e80 con 0x557f5c3081b0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f86872740 10 client.0 _ll_register_callbacks cb 0x557f5c5f29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7e89b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7e89b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6e4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53754/0 (socket says 172.21.15.29:53754) 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7e89b640 1 -- 172.21.15.29:0/1887107922 learned_addr learned my addr 172.21.15.29:0/1887107922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 -- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 msgr2=0x557f5c6e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 -- 172.21.15.29:0/1887107922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 msgr2=0x557f5c6e4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6e4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 -- 172.21.15.29:0/1887107922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5c606c90 con 0x557f5c310390 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6e4240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0e4970 tx=0x7f3f6c0ea4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3b8eb02d3f6669 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.633+0000 7f3f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c021070 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70012aa0 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3f6c0940d0 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3f6c0a8e50 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 10 client.28001 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557f5c6f2b80 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f70012aa0 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3f6c0942d0 con 0x557f5c310390 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 10 client.28001 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 10 client.28001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 20 client.28001 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2ef9f0 0x557f5c6f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 4 client.28001 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:20.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.634+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f5c650df0 con 0x557f5c2ef9f0 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.635+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2ef9f0 0x557f5c6f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.635+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2ef9f0 0x557f5c6f6080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=562e0187f416684f server_cookie=35d7e0120810b5bb in_seq=0 out_seq=0 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.635+0000 7f3f5dffb640 10 client.28001 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1887107922 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f5c650df0 con 0x557f5c2ef9f0 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f5dffb640 10 client.28001 handle_command_reply: tid=0 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 client.28001 shutdown 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 msgr2=0x557f5c6e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6e4240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0e4970 tx=0x7f3f6c0ea4e0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 shutdown_connections 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2ef9f0 0x557f5c6f6080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c331070 0x557f5c6e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f5dffb640 0 client.28001 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c310390 0x557f5c6e4240 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 --2- 172.21.15.29:0/1887107922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c3081b0 0x557f5c6e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 >> 172.21.15.29:0/1887107922 conn(0x557f5c325460 msgr2=0x557f5c6db880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 shutdown_connections 2024-09-21T02:01:20.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 1 -- 172.21.15.29:0/1887107922 wait complete. 2024-09-21T02:01:20.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.636+0000 7f3f86872740 20 client.28001 trim_cache size 0 max 16384 2024-09-21T02:01:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 Processor -- start 2024-09-21T02:01:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 -- start start 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 0x557f5c6e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c331070 0x557f5c6e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781b1510 con 0x557f5c310390 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781b3080 con 0x557f5c3081b0 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f86872740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781b1be0 con 0x557f5c331070 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.650+0000 7f3f7d899640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7d899640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53766/0 (socket says 172.21.15.29:53766) 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 0x557f5c6e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7d899640 1 -- 172.21.15.29:0/3959568992 learned_addr learned my addr 172.21.15.29:0/3959568992 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e89b640 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c331070 0x557f5c6e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 -- 172.21.15.29:0/3959568992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c331070 msgr2=0x557f5c6e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c331070 0x557f5c6e1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 -- 172.21.15.29:0/3959568992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 msgr2=0x557f5c6e0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 -- 172.21.15.29:0/3959568992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5c6d6e80 con 0x557f5c3081b0 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7d899640 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 0x557f5c6e58c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3f7001c040 tx=0x7f3f70011d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355b6599b169e68c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f5ffff640 1 -- 172.21.15.29:0/3959568992 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700096c0 con 0x557f5c3081b0 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f5ffff640 1 -- 172.21.15.29:0/3959568992 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f70005040 con 0x557f5c3081b0 2024-09-21T02:01:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.651+0000 7f3f5ffff640 1 -- 172.21.15.29:0/3959568992 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700048a0 con 0x557f5c3081b0 2024-09-21T02:01:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 -- 172.21.15.29:0/3959568992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 msgr2=0x557f5c6e58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 0x557f5c6e58c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3f7001c040 tx=0x7f3f70011d60 comp rx=0 tx=0).stop 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 -- 172.21.15.29:0/3959568992 shutdown_connections 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c331070 0x557f5c6e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c310390 0x557f5c6e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 --2- 172.21.15.29:0/3959568992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c3081b0 0x557f5c6e58c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 -- 172.21.15.29:0/3959568992 >> 172.21.15.29:0/3959568992 conn(0x557f5c325460 msgr2=0x557f5c6e51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 -- 172.21.15.29:0/3959568992 shutdown_connections 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.652+0000 7f3f86872740 1 -- 172.21.15.29:0/3959568992 wait complete. 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 -- start start 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 Processor -- start 2024-09-21T02:01:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c310390 0x557f5c6e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c331070 0x557f5c6e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781b2980 con 0x557f5c3081b0 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781b22b0 con 0x557f5c331070 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781af4f0 con 0x557f5c310390 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f86872740 10 client.0 _ll_register_callbacks cb 0x7f3f6c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f7e09a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f7e09a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53780/0 (socket says 172.21.15.29:53780) 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1426757406 learned_addr learned my addr 172.21.15.29:0/1426757406 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.653+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c310390 0x557f5c6e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e89b640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c331070 0x557f5c6e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c310390 msgr2=0x557f5c6e86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c310390 0x557f5c6e86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1426757406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c331070 msgr2=0x557f5c6e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c331070 0x557f5c6e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 -- 172.21.15.29:0/1426757406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5c75fa20 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f7e09a640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3f700117f0 tx=0x7f3f70009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53e38c69b0dba29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70004470 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70005040 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3f700051f0 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3f700053f0 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f86872740 10 client.18183 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.654+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557f5c760ca0 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f70005040 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3f70040030 con 0x557f5c3081b0 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 10 client.18183 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 10 client.18183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:01:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 20 client.18183 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2c4da0 0x557f5c763300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 4 client.18183 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f5c64c8a0 con 0x557f5c2c4da0 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.655+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2c4da0 0x557f5c763300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f7d899640 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2c4da0 0x557f5c763300 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c90e4132699b028 server_cookie=a4cddf7bfce1c415 in_seq=0 out_seq=0 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f5dffb640 10 client.18183 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f5dffb640 1 -- 172.21.15.29:0/1426757406 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f5c64c8a0 con 0x557f5c2c4da0 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f5dffb640 10 client.18183 handle_command_reply: tid=0 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 client.18183 shutdown 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 msgr2=0x557f5c6e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3f700117f0 tx=0x7f3f70009880 comp rx=0 tx=0).stop 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 shutdown_connections 2024-09-21T02:01:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x557f5c2c4da0 0x557f5c763300 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f5c331070 0x557f5c6e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f5c310390 0x557f5c6e86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 --2- 172.21.15.29:0/1426757406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557f5c3081b0 0x557f5c6e81b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 >> 172.21.15.29:0/1426757406 conn(0x557f5c325460 msgr2=0x557f5c6e5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f5dffb640 0 client.18183 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 shutdown_connections 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 1 -- 172.21.15.29:0/1426757406 wait complete. 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.656+0000 7f3f86872740 20 client.18183 trim_cache size 0 max 16384 2024-09-21T02:01:20.661 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 msgr2=0x7f3f600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 0x7f3f600b8a20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802f500 tx=0x7f3f6802f310 comp rx=0 tx=0).stop 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 msgr2=0x7f3f781104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d770 tx=0x7f3f6c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 shutdown_connections 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3f600b6510 0x7f3f600b8a20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f78108cf0 0x7f3f78110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f781083a0 0x7f3f781104f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 --2- 172.21.15.29:0/3676337522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f78102a60 0x7f3f7810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:20.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 >> 172.21.15.29:0/3676337522 conn(0x7f3f78079730 msgr2=0x7f3f78104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:20.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 shutdown_connections 2024-09-21T02:01:20.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:20.657+0000 7f3f802fe640 1 -- 172.21.15.29:0/3676337522 wait complete. 2024-09-21T02:01:21.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:21 smithi029 ceph-mon[31228]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-09-21T02:01:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:21 smithi190 ceph-mon[34368]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-09-21T02:01:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:21 smithi066 ceph-mon[35001]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-09-21T02:01:22.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:01:22.508 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:22.970 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 765.0 s, 3.4 tps, lat 1932.994 ms stddev 22.668 2024-09-21T02:01:23.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 -- 172.21.15.29:0/1739013437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0102ae0 msgr2=0x7f12b0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/1739013437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0102ae0 0x7f12b0102ef0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009860 tx=0x7f12ac02a340 comp rx=0 tx=0).stop 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 -- 172.21.15.29:0/1739013437 shutdown_connections 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/1739013437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b0108d20 0x7f12b010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/1739013437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b01083d0 0x7f12b01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/1739013437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0102ae0 0x7f12b0102ef0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 -- 172.21.15.29:0/1739013437 >> 172.21.15.29:0/1739013437 conn(0x7f12b0079760 msgr2=0x7f12b0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 -- 172.21.15.29:0/1739013437 shutdown_connections 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.194+0000 7f12b7f7c640 1 -- 172.21.15.29:0/1739013437 wait complete. 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 Processor -- start 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 -- start start 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 0x7f12b019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 0x7f12b019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0112990 con 0x7f12b0102ae0 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0112810 con 0x7f12b0108d20 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b7f7c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0112b10 con 0x7f12b01083d0 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48448/0 (socket says 172.21.15.29:48448) 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b6519640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 0x7f12b019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b5d18640 1 -- 172.21.15.29:0/2360215858 learned_addr learned my addr 172.21.15.29:0/2360215858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.195+0000 7f12b5517640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 0x7f12b019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 -- 172.21.15.29:0/2360215858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 msgr2=0x7f12b019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 0x7f12b019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 -- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 msgr2=0x7f12b019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 -- 172.21.15.29:0/2360215858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a03c0 con 0x7f12b0108d20 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b5517640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 0x7f12b019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b5d18640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b6519640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 0x7f12b019fc90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009fd0 tx=0x7f12a800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12a6ffd640 1 -- 172.21.15.29:0/2360215858 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800ce20 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12a6ffd640 1 -- 172.21.15.29:0/2360215858 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8005020 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a06b0 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12a6ffd640 1 -- 172.21.15.29:0/2360215858 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8010690 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.196+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a6e10 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12be4f0740 1 Processor -- start 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12a6ffd640 1 -- 172.21.15.29:0/2360215858 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f12a80054d0 con 0x7f12b0108d20 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12be4f0740 1 -- start start 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.199+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca8390 0x556db4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12be4f0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556db3fe6ea0 con 0x556db3ca01b0 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12be4f0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556db3fd62d0 con 0x556db3cc9070 2024-09-21T02:01:23.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12be4f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556db3fda0e0 con 0x556db3ca8390 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57216/0 (socket says 172.21.15.29:57216) 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48450/0 (socket says 172.21.15.29:48450) 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 -- 172.21.15.29:0/2151637821 learned_addr learned my addr 172.21.15.29:0/2151637821 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca8390 0x556db4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 -- 172.21.15.29:0/2151637821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca8390 msgr2=0x556db4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca8390 0x556db4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 -- 172.21.15.29:0/2151637821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 msgr2=0x556db4070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 -- 172.21.15.29:0/2151637821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db3d34650 con 0x556db3cc9070 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5d18640 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12a6ffd640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 0x7f129c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5517640 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12a0011b80 tx=0x7f12a00095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e7ab8335bc9b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12b5d18640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 0x7f129c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f12a6ffd640 1 -- 172.21.15.29:0/2360215858 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12a80e79d0 con 0x7f12b0108d20 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f129bfff640 1 -- 172.21.15.29:0/2151637821 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a000fc50 con 0x556db3cc9070 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.200+0000 7f129bfff640 1 -- 172.21.15.29:0/2151637821 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a000fdf0 con 0x556db3cc9070 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f129bfff640 1 -- 172.21.15.29:0/2151637821 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0004a00 con 0x556db3cc9070 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 -- 172.21.15.29:0/2151637821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 msgr2=0x556db40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12b5d18640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 0x7f129c0b8a20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x556db406ef40 tx=0x7f12ac02c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12a0011b80 tx=0x7f12a00095b0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 -- 172.21.15.29:0/2151637821 shutdown_connections 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca8390 0x556db4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 --2- 172.21.15.29:0/2151637821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db40702c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.201+0000 7f12be4f0740 1 -- 172.21.15.29:0/2151637821 >> 172.21.15.29:0/2151637821 conn(0x556db3cbd460 msgr2=0x556db406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.202+0000 7f12be4f0740 1 -- 172.21.15.29:0/2151637821 shutdown_connections 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.202+0000 7f12be4f0740 1 -- 172.21.15.29:0/2151637821 wait complete. 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.202+0000 7f12be4f0740 1 -- start start 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.202+0000 7f12be4f0740 1 Processor -- start 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db407ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db4087dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556db3e37700 con 0x556db3ca8390 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556db3f71160 con 0x556db3cc9070 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556db407eaa0 con 0x556db3ca01b0 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12be4f0740 10 client.0 _ll_register_callbacks cb 0x556db3f8a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48452/0 (socket says 172.21.15.29:48452) 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 -- 172.21.15.29:0/1443250072 learned_addr learned my addr 172.21.15.29:0/1443250072 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b6519640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db4087dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5d18640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db407ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 -- 172.21.15.29:0/1443250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 msgr2=0x556db407ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db407ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 -- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 msgr2=0x556db4087dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db4087dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 -- 172.21.15.29:0/1443250072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556db3f9ec90 con 0x556db3ca8390 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b6519640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db4087dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12b5517640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f12a000ffc0 tx=0x7f12a0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8465806eb611feca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12997fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.203+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a002d060 con 0x556db3ca8390 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12a0025070 con 0x556db3ca8390 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a000fdf0 con 0x556db3ca8390 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12a000fc50 con 0x556db3ca8390 2024-09-21T02:01:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 10 client.18195 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556db408a810 con 0x556db3ca8390 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a000fdf0 con 0x556db3ca8390 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f12a001f440 con 0x556db3ca8390 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 10 client.18195 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 10 client.18195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 20 client.18195 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c879f0 0x556db408dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 4 client.18195 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556db3fe8df0 con 0x556db3c879f0 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.204+0000 7f12b5d18640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c879f0 0x556db408dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.205+0000 7f12997fa640 10 client.18195 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.205+0000 7f12b5d18640 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c879f0 0x556db408dc20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed3c149251e8af54 server_cookie=2999abc62952a734 in_seq=0 out_seq=0 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12997fa640 1 -- 172.21.15.29:0/1443250072 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556db3fe8df0 con 0x556db3c879f0 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12997fa640 10 client.18195 handle_command_reply: tid=0 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 client.18195 shutdown 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 msgr2=0x556db407bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f12a000ffc0 tx=0x7f12a0005a00 comp rx=0 tx=0).stop 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 shutdown_connections 2024-09-21T02:01:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c879f0 0x556db408dc20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db4087dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db407bf40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1443250072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db407ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12997fa640 0 client.18195 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 >> 172.21.15.29:0/1443250072 conn(0x556db3cbd460 msgr2=0x556db4073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 shutdown_connections 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 1 -- 172.21.15.29:0/1443250072 wait complete. 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.206+0000 7f12be4f0740 20 client.18195 trim_cache size 0 max 16384 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 Processor -- start 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 -- start start 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db4073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01af2d0 con 0x556db3ca8390 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0e40 con 0x556db3cc9070 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.219+0000 7f12be4f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01af9a0 con 0x556db3ca01b0 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48460/0 (socket says 172.21.15.29:48460) 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 -- 172.21.15.29:0/1384440266 learned_addr learned my addr 172.21.15.29:0/1384440266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b6519640 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5d18640 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db4073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 -- 172.21.15.29:0/1384440266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 msgr2=0x556db4073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db4073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 -- 172.21.15.29:0/1384440266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 msgr2=0x556db406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 -- 172.21.15.29:0/1384440266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db406eaa0 con 0x556db3ca8390 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f12b5517640 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12a0005630 tx=0x7f12a0009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab516e2f30eea381 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f129b7fe640 1 -- 172.21.15.29:0/1384440266 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0038070 con 0x556db3ca8390 2024-09-21T02:01:23.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f129b7fe640 1 -- 172.21.15.29:0/1384440266 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a000fdf0 con 0x556db3ca8390 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.220+0000 7f129b7fe640 1 -- 172.21.15.29:0/1384440266 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a002d030 con 0x556db3ca8390 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- 172.21.15.29:0/1384440266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 msgr2=0x556db4087d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12a0005630 tx=0x7f12a0009300 comp rx=0 tx=0).stop 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- 172.21.15.29:0/1384440266 shutdown_connections 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3cc9070 0x556db406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca8390 0x556db4087d30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 --2- 172.21.15.29:0/1384440266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3ca01b0 0x556db4073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- 172.21.15.29:0/1384440266 >> 172.21.15.29:0/1384440266 conn(0x556db3cbd460 msgr2=0x556db40733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- 172.21.15.29:0/1384440266 shutdown_connections 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- 172.21.15.29:0/1384440266 wait complete. 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 -- start start 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.221+0000 7f12be4f0740 1 Processor -- start 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 0x556db40efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3cc9070 0x556db408e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0740 con 0x556db3ca01b0 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0070 con 0x556db3ca8390 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ad2b0 con 0x556db3cc9070 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12be4f0740 10 client.0 _ll_register_callbacks cb 0x7f12a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48472/0 (socket says 172.21.15.29:48472) 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5517640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 0x556db40efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 -- 172.21.15.29:0/3210707491 learned_addr learned my addr 172.21.15.29:0/3210707491 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b6519640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3cc9070 0x556db408e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 -- 172.21.15.29:0/3210707491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3cc9070 msgr2=0x556db408e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3cc9070 0x556db408e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 -- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 msgr2=0x556db40efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:23.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 0x556db40efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5d18640 1 -- 172.21.15.29:0/3210707491 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556db408ebe0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.222+0000 7f12b5517640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 0x556db40efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12b5d18640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0095f0 tx=0x7f12ac02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5aa4a84a762feb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac037040 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f12ac0327f0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a000fdf0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12ac0329f0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12be4f0740 10 client.18201 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556db40f7ef0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a000fdf0 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.223+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f12ac049040 con 0x556db3ca01b0 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 10 client.18201 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 10 client.18201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 20 client.18201 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:23.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c5cda0 0x556db40fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 4 client.18201 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556db3fe48a0 con 0x556db3c5cda0 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12b5517640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c5cda0 0x556db40fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12997fa640 10 client.18201 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12b5517640 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c5cda0 0x556db40fa6e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=419764b1ee4b39af server_cookie=d5c426492a4ede8 in_seq=0 out_seq=0 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12997fa640 1 -- 172.21.15.29:0/3210707491 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556db3fe48a0 con 0x556db3c5cda0 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12997fa640 10 client.18201 handle_command_reply: tid=0 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.224+0000 7f12be4f0740 1 client.18201 shutdown 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 msgr2=0x556db40ef650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0095f0 tx=0x7f12ac02a340 comp rx=0 tx=0).stop 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 shutdown_connections 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x556db3c5cda0 0x556db40fa6e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556db3cc9070 0x556db408e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556db3ca8390 0x556db40efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 --2- 172.21.15.29:0/3210707491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556db3ca01b0 0x556db40ef650 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12997fa640 0 client.18201 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 >> 172.21.15.29:0/3210707491 conn(0x556db3cbd460 msgr2=0x556db406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 shutdown_connections 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 1 -- 172.21.15.29:0/3210707491 wait complete. 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12be4f0740 20 client.18201 trim_cache size 0 max 16384 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 msgr2=0x7f129c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 0x7f129c0b8a20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x556db406ef40 tx=0x7f12ac02c040 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 msgr2=0x7f12b019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 0x7f12b019fc90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009fd0 tx=0x7f12a800ef10 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 shutdown_connections 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f129c0b6510 0x7f129c0b8a20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12b0108d20 0x7f12b019fc90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12b01083d0 0x7f12b019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 --2- 172.21.15.29:0/2360215858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b0102ae0 0x7f12b019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:23.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 >> 172.21.15.29:0/2360215858 conn(0x7f12b0079760 msgr2=0x7f12b0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:23.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 shutdown_connections 2024-09-21T02:01:23.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:23.225+0000 7f12b7f7c640 1 -- 172.21.15.29:0/2360215858 wait complete. 2024-09-21T02:01:23.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:23 smithi029 ceph-mon[31228]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2024-09-21T02:01:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:23 smithi190 ceph-mon[34368]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2024-09-21T02:01:24.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:23 smithi066 ceph-mon[35001]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2024-09-21T02:01:24.510 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:01:24.921 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.512+0000 7f720af02640 1 -- 172.21.15.29:0/643376500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f720411cc30 msgr2=0x7f720411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.512+0000 7f720af02640 1 --2- 172.21.15.29:0/643376500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f720411cc30 0x7f720411f140 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009900 tx=0x7f71fc02a440 comp rx=0 tx=0).stop 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.519+0000 7f720af02640 1 -- 172.21.15.29:0/643376500 shutdown_connections 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.519+0000 7f720af02640 1 --2- 172.21.15.29:0/643376500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f720411cc30 0x7f720411f140 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.519+0000 7f720af02640 1 --2- 172.21.15.29:0/643376500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720410a9e0 0x7f720410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.519+0000 7f720af02640 1 --2- 172.21.15.29:0/643376500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204111f40 0x7f720410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.519+0000 7f720af02640 1 -- 172.21.15.29:0/643376500 >> 172.21.15.29:0/643376500 conn(0x7f7204070ec0 msgr2=0x7f72040712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.520+0000 7f720af02640 1 -- 172.21.15.29:0/643376500 shutdown_connections 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.520+0000 7f720af02640 1 -- 172.21.15.29:0/643376500 wait complete. 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.521+0000 7f720af02640 1 Processor -- start 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 -- start start 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f720410a9e0 0x7f72041a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7204111f40 0x7f72041a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f72041228b0 con 0x7f720411cc30 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7204122730 con 0x7f720410a9e0 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.530+0000 7f720af02640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7204122a30 con 0x7f7204111f40 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.531+0000 7f720949f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.531+0000 7f720949f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48482/0 (socket says 172.21.15.29:48482) 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.531+0000 7f720949f640 1 -- 172.21.15.29:0/41144872 learned_addr learned my addr 172.21.15.29:0/41144872 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.533+0000 7f7203fff640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7204111f40 0x7f72041a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.533+0000 7f7208c9e640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f720410a9e0 0x7f72041a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.534+0000 7f720949f640 1 -- 172.21.15.29:0/41144872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7204111f40 msgr2=0x7f72041a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.534+0000 7f720949f640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7204111f40 0x7f72041a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.534+0000 7f720949f640 1 -- 172.21.15.29:0/41144872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f720410a9e0 msgr2=0x7f72041a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.534+0000 7f720949f640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f720410a9e0 0x7f72041a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.534+0000 7f720949f640 1 -- 172.21.15.29:0/41144872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72041af1e0 con 0x7f720411cc30 2024-09-21T02:01:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.537+0000 7f720949f640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f71fc006010 tx=0x7f71fc003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.541+0000 7f7201ffb640 1 -- 172.21.15.29:0/41144872 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc040030 con 0x7f720411cc30 2024-09-21T02:01:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.541+0000 7f7201ffb640 1 -- 172.21.15.29:0/41144872 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71fc004190 con 0x7f720411cc30 2024-09-21T02:01:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.541+0000 7f7201ffb640 1 -- 172.21.15.29:0/41144872 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0317c0 con 0x7f720411cc30 2024-09-21T02:01:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.541+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72041af470 con 0x7f720411cc30 2024-09-21T02:01:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.541+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041af840 con 0x7f720411cc30 2024-09-21T02:01:25.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.548+0000 7f7211476740 1 Processor -- start 2024-09-21T02:01:25.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.548+0000 7f7211476740 1 -- start start 2024-09-21T02:01:25.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d41120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6cb6ea0 con 0x55f3a69701b0 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6ca62d0 con 0x55f3a6999070 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7211476740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6caa0e0 con 0x55f3a6978390 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.549+0000 7f7201ffb640 1 -- 172.21.15.29:0/41144872 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f71fc043030 con 0x7f720411cc30 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.550+0000 7f7201ffb640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 0x7f71f00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.550+0000 7f720949f640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 0x7f71f00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.551+0000 7f7201ffb640 1 -- 172.21.15.29:0/41144872 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f71fc1080b0 con 0x7f720411cc30 2024-09-21T02:01:25.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.552+0000 7f720949f640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 0x7f71f00b8c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f71fc037040 tx=0x7f71fc101040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.553+0000 7f7208c9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.553+0000 7f7208c9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48488/0 (socket says 172.21.15.29:48488) 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.553+0000 7f7208c9e640 1 -- 172.21.15.29:0/4209443563 learned_addr learned my addr 172.21.15.29:0/4209443563 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 -- 172.21.15.29:0/4209443563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 msgr2=0x55f3a6d41120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d41120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 -- 172.21.15.29:0/4209443563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 msgr2=0x55f3a6d406a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:25.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d406a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 -- 172.21.15.29:0/4209443563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a6a04650 con 0x55f3a69701b0 2024-09-21T02:01:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.554+0000 7f7208c9e640 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009df0 tx=0x7f71f400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a7d06cce3585 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.555+0000 7f71faffd640 1 -- 172.21.15.29:0/4209443563 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f401a480 con 0x55f3a69701b0 2024-09-21T02:01:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.555+0000 7f71faffd640 1 -- 172.21.15.29:0/4209443563 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f401aad0 con 0x55f3a69701b0 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 -- 172.21.15.29:0/4209443563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 msgr2=0x55f3a6d40be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009df0 tx=0x7f71f400f730 comp rx=0 tx=0).stop 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 -- 172.21.15.29:0/4209443563 shutdown_connections 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d41120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69701b0 0x55f3a6d40be0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 --2- 172.21.15.29:0/4209443563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.556+0000 7f7211476740 1 -- 172.21.15.29:0/4209443563 >> 172.21.15.29:0/4209443563 conn(0x55f3a698d460 msgr2=0x55f3a6d3e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.564+0000 7f7211476740 1 -- 172.21.15.29:0/4209443563 shutdown_connections 2024-09-21T02:01:25.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.564+0000 7f7211476740 1 -- 172.21.15.29:0/4209443563 wait complete. 2024-09-21T02:01:25.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.564+0000 7f7211476740 1 -- start start 2024-09-21T02:01:25.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.564+0000 7f7211476740 1 Processor -- start 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d4c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6b07700 con 0x55f3a69579f0 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6c41160 con 0x55f3a6999070 2024-09-21T02:01:25.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f3a6d4eed0 con 0x55f3a6978390 2024-09-21T02:01:25.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.565+0000 7f7211476740 10 client.0 _ll_register_callbacks cb 0x55f3a6c5a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:25.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.568+0000 7f7203fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.568+0000 7f7203fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59628/0 (socket says 172.21.15.29:59628) 2024-09-21T02:01:25.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.568+0000 7f7203fff640 1 -- 172.21.15.29:0/2370712735 learned_addr learned my addr 172.21.15.29:0/2370712735 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:25.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.569+0000 7f7208c9e640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.570+0000 7f7203fff640 1 -- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 msgr2=0x55f3a6d4b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.570+0000 7f7203fff640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d4b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.570+0000 7f7203fff640 1 -- 172.21.15.29:0/2370712735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 msgr2=0x55f3a6d4c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.570+0000 7f7203fff640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d4c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.570+0000 7f7203fff640 1 -- 172.21.15.29:0/2370712735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a6c6ec90 con 0x55f3a6978390 2024-09-21T02:01:25.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.571+0000 7f7203fff640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005d00 tx=0x7f71ec010850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33fec56866aa4257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.571+0000 7f71e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:25.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.577+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec012890 con 0x55f3a6978390 2024-09-21T02:01:25.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.577+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3a6cb8df0 con 0x55f3a6978390 2024-09-21T02:01:25.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.577+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f71ec012a30 con 0x55f3a6978390 2024-09-21T02:01:25.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.578+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f71ec012c50 con 0x55f3a6978390 2024-09-21T02:01:25.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.578+0000 7f7211476740 10 client.27982 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:25.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.578+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f3a6d5a990 con 0x55f3a6978390 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f3a6cb8df0 con 0x55f3a6978390 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f71ec00f040 con 0x55f3a6978390 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 10 client.27982 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 10 client.27982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 20 client.27982 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a692cda0 0x55f3a6d5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 4 client.27982 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.579+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a6cb48a0 con 0x55f3a692cda0 2024-09-21T02:01:25.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.581+0000 7f7208c9e640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a692cda0 0x55f3a6d5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.584+0000 7f7208c9e640 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a692cda0 0x55f3a6d5e2e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e850bed47904bb3 server_cookie=f3f48fcd2a4b44a1 in_seq=0 out_seq=0 2024-09-21T02:01:25.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.584+0000 7f71e3fff640 10 client.27982 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f71e3fff640 1 -- 172.21.15.29:0/2370712735 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3a6cb48a0 con 0x55f3a692cda0 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f71e3fff640 10 client.27982 handle_command_reply: tid=0 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 client.27982 shutdown 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 msgr2=0x55f3a6d4e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005d00 tx=0x7f71ec010850 comp rx=0 tx=0).stop 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 shutdown_connections 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a692cda0 0x55f3a6d5e2e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d4c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d4b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 --2- 172.21.15.29:0/2370712735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6978390 0x55f3a6d4e470 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005d00 tx=0x7f71ec010850 comp rx=0 tx=0).stop 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.586+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 >> 172.21.15.29:0/2370712735 conn(0x55f3a698d460 msgr2=0x55f3a6d43830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.587+0000 7f71e3fff640 0 client.27982 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.587+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 shutdown_connections 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.587+0000 7f7211476740 1 -- 172.21.15.29:0/2370712735 wait complete. 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.587+0000 7f7211476740 20 client.27982 trim_cache size 0 max 16384 2024-09-21T02:01:25.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 Processor -- start 2024-09-21T02:01:25.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 -- start start 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6999070 0x55f3a6d45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71e400e580 con 0x55f3a69579f0 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40100f0 con 0x55f3a6d46370 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7211476740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71e400ec50 con 0x55f3a6999070 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.610+0000 7f7203fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57286/0 (socket says 172.21.15.29:57286) 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 -- 172.21.15.29:0/822634235 learned_addr learned my addr 172.21.15.29:0/822634235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:25.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7208c9e640 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6999070 0x55f3a6d45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 -- 172.21.15.29:0/822634235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6999070 msgr2=0x55f3a6d45e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6999070 0x55f3a6d45e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 -- 172.21.15.29:0/822634235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 msgr2=0x55f3a6d458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 -- 172.21.15.29:0/822634235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a6d5a7c0 con 0x55f3a6d46370 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f7203fff640 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71ec006010 tx=0x7f71ec000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60fd8ff9c9f9f65a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.611+0000 7f71fa7fc640 1 -- 172.21.15.29:0/822634235 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0259d0 con 0x55f3a6d46370 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.612+0000 7f71fa7fc640 1 -- 172.21.15.29:0/822634235 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec02b070 con 0x55f3a6d46370 2024-09-21T02:01:25.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.612+0000 7f71fa7fc640 1 -- 172.21.15.29:0/822634235 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0052f0 con 0x55f3a6d46370 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 -- 172.21.15.29:0/822634235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 msgr2=0x55f3a6d48b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71ec006010 tx=0x7f71ec000bc0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 -- 172.21.15.29:0/822634235 shutdown_connections 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6d46370 0x55f3a6d48b30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a6999070 0x55f3a6d45e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 --2- 172.21.15.29:0/822634235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a69579f0 0x55f3a6d458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 -- 172.21.15.29:0/822634235 >> 172.21.15.29:0/822634235 conn(0x55f3a698d460 msgr2=0x55f3a6d4a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.613+0000 7f7211476740 1 -- 172.21.15.29:0/822634235 shutdown_connections 2024-09-21T02:01:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.614+0000 7f7211476740 1 -- 172.21.15.29:0/822634235 wait complete. 2024-09-21T02:01:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.614+0000 7f7211476740 1 -- start start 2024-09-21T02:01:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.615+0000 7f7211476740 1 Processor -- start 2024-09-21T02:01:25.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.615+0000 7f7211476740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.616+0000 7f7211476740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.616+0000 7f7203fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.616+0000 7f7203fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59654/0 (socket says 172.21.15.29:59654) 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7208c9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7208c9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d50b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57298/0 (socket says 172.21.15.29:57298) 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7203fff640 1 -- 172.21.15.29:0/2400429347 learned_addr learned my addr 172.21.15.29:0/2400429347 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a6d46370 0x55f3a6dc7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71e400f9f0 con 0x55f3a6d46370 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71e400f320 con 0x55f3a6999070 2024-09-21T02:01:25.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.617+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71e400c560 con 0x55f3a69579f0 2024-09-21T02:01:25.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 -- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 msgr2=0x55f3a6d50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 -- 172.21.15.29:0/2400429347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a6d46370 msgr2=0x55f3a6dc7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a6d46370 0x55f3a6dc7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a6dc8220 con 0x55f3a69579f0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7211476740 10 client.0 _ll_register_callbacks cb 0x7f71ec00e370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec02b070 con 0x55f3a69579f0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f7203fff640 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71ec025750 tx=0x7f71ec012b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9c42cacf5990bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.618+0000 7f71e3fff640 10 client.27988 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.619+0000 7f71e3fff640 1 -- 172.21.15.29:0/2400429347 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0253f0 con 0x55f3a69579f0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.619+0000 7f71e3fff640 1 -- 172.21.15.29:0/2400429347 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f71ec025780 con 0x55f3a69579f0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.619+0000 7f71e3fff640 1 -- 172.21.15.29:0/2400429347 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f71ec0259e0 con 0x55f3a69579f0 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.619+0000 7f7211476740 10 client.27988 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:25.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.619+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=48}) -- 0x55f3a6dc90b0 con 0x55f3a69579f0 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f71e3fff640 1 -- 172.21.15.29:0/2400429347 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f71ec025b80 con 0x55f3a69579f0 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 10 client.27988 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 20 client.27988 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a6dc9810 0x55f3a6dcbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 4 client.27988 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:01:25.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3a6cb1cd0 con 0x55f3a6dc9810 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.620+0000 7f7208c9e640 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a6dc9810 0x55f3a6dcbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f71e3fff640 10 client.27988 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7208c9e640 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a6dc9810 0x55f3a6dcbc30 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3819e40cf72f4f9a server_cookie=52be545ed0325bef in_seq=0 out_seq=0 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f71e3fff640 1 -- 172.21.15.29:0/2400429347 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3a6cb1cd0 con 0x55f3a6dc9810 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f71e3fff640 10 client.27988 handle_command_reply: tid=0 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 client.27988 shutdown 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 msgr2=0x55f3a6d505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71ec025750 tx=0x7f71ec012b70 comp rx=0 tx=0).stop 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 shutdown_connections 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f3a6dc9810 0x55f3a6dcbc30 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f3a6d46370 0x55f3a6dc7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3a6999070 0x55f3a6d50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 --2- 172.21.15.29:0/2400429347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3a69579f0 0x55f3a6d505c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 >> 172.21.15.29:0/2400429347 conn(0x55f3a698d460 msgr2=0x55f3a6d41680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f71e3fff640 0 client.27988 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 shutdown_connections 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 1 -- 172.21.15.29:0/2400429347 wait complete. 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.621+0000 7f7211476740 20 client.27988 trim_cache size 0 max 16384 2024-09-21T02:01:25.624 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 msgr2=0x7f71f00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 0x7f71f00b8c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f71fc037040 tx=0x7f71fc101040 comp rx=0 tx=0).stop 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 msgr2=0x7f72041aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f71fc006010 tx=0x7f71fc003e90 comp rx=0 tx=0).stop 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 shutdown_connections 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71f00b6720 0x7f71f00b8c30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f720411cc30 0x7f72041aeae0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7204111f40 0x7f72041a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 --2- 172.21.15.29:0/41144872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f720410a9e0 0x7f72041a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:25.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.622+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 >> 172.21.15.29:0/41144872 conn(0x7f7204070ec0 msgr2=0x7f720411e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:25.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.623+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 shutdown_connections 2024-09-21T02:01:25.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:25.623+0000 7f720af02640 1 -- 172.21.15.29:0/41144872 wait complete. 2024-09-21T02:01:25.635 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:25 smithi029 ceph-mon[31228]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2024-09-21T02:01:25.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:25 smithi066 ceph-mon[35001]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2024-09-21T02:01:25.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:25 smithi190 ceph-mon[34368]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2024-09-21T02:01:27.476 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:01:27.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:01:27.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:27 smithi029 ceph-mon[31228]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2024-09-21T02:01:27.870 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.953+0000 7f7cbd07e640 1 -- 172.21.15.29:0/2989375360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a4f90 msgr2=0x7f7cb00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/2989375360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a4f90 0x7f7cb00a5400 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80074e0 tx=0x7f7ca802b0a0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 -- 172.21.15.29:0/2989375360 shutdown_connections 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/2989375360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a69b0 0x7f7cb00ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/2989375360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a6010 0x7f7cb00a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/2989375360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a4f90 0x7f7cb00a5400 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 -- 172.21.15.29:0/2989375360 >> 172.21.15.29:0/2989375360 conn(0x7f7cb00a0480 msgr2=0x7f7cb00a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 -- 172.21.15.29:0/2989375360 shutdown_connections 2024-09-21T02:01:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.954+0000 7f7cbd07e640 1 -- 172.21.15.29:0/2989375360 wait complete. 2024-09-21T02:01:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.955+0000 7f7cbd07e640 1 Processor -- start 2024-09-21T02:01:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 -- start start 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 0x7f7cb013d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 0x7f7cb013dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 0x7f7cb013e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 0x7f7cb013d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 0x7f7cb013d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59664/0 (socket says 172.21.15.29:59664) 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/4219823173 learned_addr learned my addr 172.21.15.29:0/4219823173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb00b08c0 con 0x7f7cb00a6010 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb00b0740 con 0x7f7cb00a69b0 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb00b0a40 con 0x7f7cb00a4f90 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.956+0000 7f7cb659c640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 0x7f7cb013dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb759e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 0x7f7cb013e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 -- 172.21.15.29:0/4219823173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 msgr2=0x7f7cb013d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 0x7f7cb013d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 -- 172.21.15.29:0/4219823173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 msgr2=0x7f7cb013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 0x7f7cb013e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb013f110 con 0x7f7cb00a6010 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb759e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 0x7f7cb013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.957+0000 7f7cb659c640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 0x7f7cb013dd40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009fd0 tx=0x7f7cac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.958+0000 7f7c9ffff640 1 -- 172.21.15.29:0/4219823173 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac015020 con 0x7f7cb00a6010 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.958+0000 7f7c9ffff640 1 -- 172.21.15.29:0/4219823173 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac011ce0 con 0x7f7cb00a6010 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.958+0000 7f7c9ffff640 1 -- 172.21.15.29:0/4219823173 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac010de0 con 0x7f7cb00a6010 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.958+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb013f400 con 0x7f7cb00a6010 2024-09-21T02:01:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.958+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb0145af0 con 0x7f7cb00a6010 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 Processor -- start 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 -- start start 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98486040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 0x55fd984866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 0x55fd98487200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd983f86f0 con 0x55fd980b52f0 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.963+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fd983e7b20 con 0x55fd980de1b0 2024-09-21T02:01:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd983eb930 con 0x55fd980bd4d0 2024-09-21T02:01:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 0x55fd984866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 0x55fd984866b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48550/0 (socket says 172.21.15.29:48550) 2024-09-21T02:01:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/1113315637 learned_addr learned my addr 172.21.15.29:0/1113315637 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7c9ffff640 1 -- 172.21.15.29:0/4219823173 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7cac018020 con 0x7f7cb00a6010 2024-09-21T02:01:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.964+0000 7f7cb759e640 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98486040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.965+0000 7f7cb659c640 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 0x55fd98487200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 -- 172.21.15.29:0/1113315637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 msgr2=0x55fd98486040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98486040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 -- 172.21.15.29:0/1113315637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 msgr2=0x55fd984866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 0x55fd984866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 -- 172.21.15.29:0/1113315637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd983ad530 con 0x55fd980bd4d0 2024-09-21T02:01:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7cb659c640 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 0x55fd98487200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0e4370 tx=0x7f7cac0ee2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a318ced6d72aedc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.966+0000 7f7c9d7fa640 1 -- 172.21.15.29:0/1113315637 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0f5070 con 0x55fd980bd4d0 2024-09-21T02:01:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.967+0000 7f7c9d7fa640 1 -- 172.21.15.29:0/1113315637 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac0f18d0 con 0x55fd980bd4d0 2024-09-21T02:01:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.967+0000 7f7c9d7fa640 1 -- 172.21.15.29:0/1113315637 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0f07e0 con 0x55fd980bd4d0 2024-09-21T02:01:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.967+0000 7f7c9ffff640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 0x7f7ca00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.967+0000 7f7cb759e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 0x7f7ca00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.967+0000 7f7c9ffff640 1 -- 172.21.15.29:0/4219823173 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7cac0e6920 con 0x7f7cb00a6010 2024-09-21T02:01:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 770.0 s, 1086.3 tps, lat 198.796 ms stddev 989.354 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cb759e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 0x7f7ca00b8b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8066c70 tx=0x7f7cb80692f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/1113315637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 msgr2=0x55fd98487200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 0x55fd98487200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0e4370 tx=0x7f7cac0ee2a0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/1113315637 shutdown_connections 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980bd4d0 0x55fd98487200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980b52f0 0x55fd984866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/1113315637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98486040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/1113315637 >> 172.21.15.29:0/1113315637 conn(0x55fd980d2300 msgr2=0x55fd98481ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/1113315637 shutdown_connections 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/1113315637 wait complete. 2024-09-21T02:01:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.968+0000 7f7cc3fe9740 1 -- start start 2024-09-21T02:01:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 Processor -- start 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd9848ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98490ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd9831fab0 con 0x55fd980bd4d0 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fd9838d730 con 0x55fd980de1b0 2024-09-21T02:01:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd98487d70 con 0x55fd980b52f0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cc3fe9740 10 client.0 _ll_register_callbacks cb 0x55fd983e16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59686/0 (socket says 172.21.15.29:59686) 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/715837242 learned_addr learned my addr 172.21.15.29:0/715837242 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cb759e640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98490ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.969+0000 7f7cb659c640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd9848ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 msgr2=0x55fd98490ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98490ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/715837242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 msgr2=0x55fd9848ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd9848ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 -- 172.21.15.29:0/715837242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd98149790 con 0x55fd980b52f0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb759e640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98490ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ca802b5e0 tx=0x7f7ca802bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842f93150e069634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7c9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8007e60 con 0x55fd980b52f0 2024-09-21T02:01:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7ca8004aa0 con 0x55fd980b52f0 2024-09-21T02:01:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac0f18d0 con 0x55fd980b52f0 2024-09-21T02:01:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7ca802be10 con 0x55fd980b52f0 2024-09-21T02:01:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cc3fe9740 10 client.27997 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.970+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55fd981c2620 con 0x55fd980b52f0 2024-09-21T02:01:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.971+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac0f18d0 con 0x55fd980b52f0 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f7ca8008040 con 0x55fd980b52f0 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 10 client.27997 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 10 client.27997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 20 client.27997 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd9809cd80 0x55fd9849d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 4 client.27997 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd983fa640 con 0x55fd9809cd80 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.972+0000 7f7cb659c640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd9809cd80 0x55fd9849d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.973+0000 7f7cb659c640 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd9809cd80 0x55fd9849d0a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34246efb37537ffc server_cookie=f3e32d533fe0a0f1 in_seq=0 out_seq=0 2024-09-21T02:01:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.973+0000 7f7c9aff5640 10 client.27997 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7c9aff5640 1 -- 172.21.15.29:0/715837242 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd983fa640 con 0x55fd9809cd80 2024-09-21T02:01:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7c9aff5640 10 client.27997 handle_command_reply: tid=0 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 client.27997 shutdown 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 msgr2=0x55fd9848f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ca802b5e0 tx=0x7f7ca802bcf0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 shutdown_connections 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd9809cd80 0x55fd9849d0a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98490ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd9848ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/715837242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd9848f8a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7c9aff5640 0 client.27997 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 >> 172.21.15.29:0/715837242 conn(0x55fd980d2300 msgr2=0x55fd98480150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 shutdown_connections 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/715837242 wait complete. 2024-09-21T02:01:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.974+0000 7f7cc3fe9740 20 client.27997 trim_cache size 0 max 16384 2024-09-21T02:01:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:27 smithi190 ceph-mon[34368]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2024-09-21T02:01:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 Processor -- start 2024-09-21T02:01:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 -- start start 2024-09-21T02:01:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd98485fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98482450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0150820 con 0x55fd980bd4d0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0152390 con 0x55fd980de1b0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0150ef0 con 0x55fd980b52f0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cb659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cb659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48568/0 (socket says 172.21.15.29:48568) 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cb659c640 1 -- 172.21.15.29:0/2892621914 learned_addr learned my addr 172.21.15.29:0/2892621914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cb759e640 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98482450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.993+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd98485fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 -- 172.21.15.29:0/2892621914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 msgr2=0x55fd98485fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd98485fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 -- 172.21.15.29:0/2892621914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 msgr2=0x55fd98482450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98482450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 -- 172.21.15.29:0/2892621914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd984998e0 con 0x55fd980bd4d0 2024-09-21T02:01:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7cb659c640 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0ac9f0 tx=0x7f7cac0ec530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64f7d5980226447 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7c9cff9640 1 -- 172.21.15.29:0/2892621914 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0f5030 con 0x55fd980bd4d0 2024-09-21T02:01:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7c9cff9640 1 -- 172.21.15.29:0/2892621914 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac0ec380 con 0x55fd980bd4d0 2024-09-21T02:01:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.994+0000 7f7c9cff9640 1 -- 172.21.15.29:0/2892621914 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0ac1d0 con 0x55fd980bd4d0 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/2892621914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 msgr2=0x55fd98486520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0ac9f0 tx=0x7f7cac0ec530 comp rx=0 tx=0).stop 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/2892621914 shutdown_connections 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98482450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd98486520 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/2892621914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd98485fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/2892621914 >> 172.21.15.29:0/2892621914 conn(0x55fd980d2300 msgr2=0x55fd9848eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/2892621914 shutdown_connections 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/2892621914 wait complete. 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 -- start start 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.995+0000 7f7cc3fe9740 1 Processor -- start 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd984974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98496800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0151c90 con 0x55fd980bd4d0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01515c0 con 0x55fd980de1b0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb014e800 con 0x55fd980b52f0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cc3fe9740 10 client.0 _ll_register_callbacks cb 0x7f7ca80071c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd984974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48570/0 (socket says 172.21.15.29:48570) 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 -- 172.21.15.29:0/3732131567 learned_addr learned my addr 172.21.15.29:0/3732131567 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb759e640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98496800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 -- 172.21.15.29:0/3732131567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 msgr2=0x55fd984974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd984974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 -- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 msgr2=0x55fd98496800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98496800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 -- 172.21.15.29:0/3732131567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd98496f70 con 0x55fd980bd4d0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd984974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7cb659c640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00bd70 tx=0x7f7cac010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107f6cef79945e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7c9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.996+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0f5030 con 0x55fd980bd4d0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7cac0b11e0 con 0x55fd980bd4d0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac0ec380 con 0x55fd980bd4d0 2024-09-21T02:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7cac0a81d0 con 0x55fd980bd4d0 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 10 client.18231 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55fd98507b00 con 0x55fd980bd4d0 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac0ec380 con 0x55fd980bd4d0 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f7cac0a8cd0 con 0x55fd980bd4d0 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 10 client.18231 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 10 client.18231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 20 client.18231 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd980721c0 0x55fd98509e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 4 client.18231 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:01:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.997+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd983f60f0 con 0x55fd980721c0 2024-09-21T02:01:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.998+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd980721c0 0x55fd98509e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.998+0000 7f7cb6d9d640 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd980721c0 0x55fd98509e50 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e26e9a340f1850 server_cookie=e0f0968fa29aacd9 in_seq=0 out_seq=0 2024-09-21T02:01:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:27.998+0000 7f7c9aff5640 10 client.18231 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7c9aff5640 1 -- 172.21.15.29:0/3732131567 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd983f60f0 con 0x55fd980721c0 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7c9aff5640 10 client.18231 handle_command_reply: tid=0 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 client.18231 shutdown 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 msgr2=0x55fd984961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00bd70 tx=0x7f7cac010760 comp rx=0 tx=0).stop 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 shutdown_connections 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55fd980721c0 0x55fd98509e50 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd980de1b0 0x55fd98496800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7c9aff5640 0 client.18231 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd980bd4d0 0x55fd984961b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 --2- 172.21.15.29:0/3732131567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd980b52f0 0x55fd984974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 >> 172.21.15.29:0/3732131567 conn(0x55fd980d2300 msgr2=0x55fd984823e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 shutdown_connections 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 1 -- 172.21.15.29:0/3732131567 wait complete. 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.006+0000 7f7cc3fe9740 20 client.18231 trim_cache size 0 max 16384 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:01:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 msgr2=0x7f7ca00b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 0x7f7ca00b8b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8066c70 tx=0x7f7cb80692f0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 msgr2=0x7f7cb013dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 0x7f7cb013dd40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009fd0 tx=0x7f7cac00b870 comp rx=0 tx=0).stop 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 shutdown_connections 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f7ca00b6650 0x7f7ca00b8b60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb00a69b0 0x7f7cb013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb00a6010 0x7f7cb013dd40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 --2- 172.21.15.29:0/4219823173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cb00a4f90 0x7f7cb013d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 >> 172.21.15.29:0/4219823173 conn(0x7f7cb00a0480 msgr2=0x7f7cb00ab540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 shutdown_connections 2024-09-21T02:01:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:28.007+0000 7f7cbd07e640 1 -- 172.21.15.29:0/4219823173 wait complete. 2024-09-21T02:01:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:27 smithi066 ceph-mon[35001]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2024-09-21T02:01:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 -- 172.21.15.29:0/703603894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 msgr2=0x7f64dc10a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 --2- 172.21.15.29:0/703603894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc10a9a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009900 tx=0x7f64d002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 -- 172.21.15.29:0/703603894 shutdown_connections 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 --2- 172.21.15.29:0/703603894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc116640 0x7f64dc118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 --2- 172.21.15.29:0/703603894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc10aee0 0x7f64dc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 --2- 172.21.15.29:0/703603894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc10a9a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 -- 172.21.15.29:0/703603894 >> 172.21.15.29:0/703603894 conn(0x7f64dc070ec0 msgr2=0x7f64dc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 -- 172.21.15.29:0/703603894 shutdown_connections 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.460+0000 7f64e2f74640 1 -- 172.21.15.29:0/703603894 wait complete. 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 Processor -- start 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 -- start start 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 0x7f64dc1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc116640 0x7f64dc1a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10c0f0 con 0x7f64dc10a590 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10bf70 con 0x7f64dc116640 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e2f74640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10c270 con 0x7f64dc10aee0 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48578/0 (socket says 172.21.15.29:48578) 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.461+0000 7f64e0d10640 1 -- 172.21.15.29:0/2523268728 learned_addr learned my addr 172.21.15.29:0/2523268728 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64dbfff640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 0x7f64dc1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e1511640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc116640 0x7f64dc1a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 -- 172.21.15.29:0/2523268728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 msgr2=0x7f64dc1a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 0x7f64dc1a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 -- 172.21.15.29:0/2523268728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc116640 msgr2=0x7f64dc1a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc116640 0x7f64dc1a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 -- 172.21.15.29:0/2523268728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc1a8c00 con 0x7f64dc10a590 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64dbfff640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 0x7f64dc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e0d10640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f64d0004770 tx=0x7f64d00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64d9ffb640 1 -- 172.21.15.29:0/2523268728 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d003f030 con 0x7f64dc10a590 2024-09-21T02:01:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64d9ffb640 1 -- 172.21.15.29:0/2523268728 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d00044c0 con 0x7f64dc10a590 2024-09-21T02:01:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc1a8e90 con 0x7f64dc10a590 2024-09-21T02:01:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.462+0000 7f64d9ffb640 1 -- 172.21.15.29:0/2523268728 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0035070 con 0x7f64dc10a590 2024-09-21T02:01:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.463+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc1b1840 con 0x7f64dc10a590 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.466+0000 7f64e94e8740 1 Processor -- start 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.466+0000 7f64e94e8740 1 -- start start 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.466+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558280b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.466+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 0x55558280b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.466+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 0x55558280beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64d9ffb640 1 -- 172.21.15.29:0/2523268728 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f64d0042030 con 0x7f64dc10a590 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e94e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555582781e60 con 0x55558243b490 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 0x55558280b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64dbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 0x55558280beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 0x55558280b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48582/0 (socket says 172.21.15.29:48582) 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e0d10640 1 -- 172.21.15.29:0/4115041967 learned_addr learned my addr 172.21.15.29:0/4115041967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555582771290 con 0x555582464350 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5555827750a0 con 0x555582443670 2024-09-21T02:01:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.467+0000 7f64e1511640 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558280b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64dbfff640 1 -- 172.21.15.29:0/4115041967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 msgr2=0x55558280b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64dbfff640 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558280b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64dbfff640 1 -- 172.21.15.29:0/4115041967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 msgr2=0x55558280b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64dbfff640 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 0x55558280b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64dbfff640 1 -- 172.21.15.29:0/4115041967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555824cf930 con 0x555582443670 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64d9ffb640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 0x7f64cc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64d9ffb640 1 -- 172.21.15.29:0/2523268728 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f64d0107670 con 0x7f64dc10a590 2024-09-21T02:01:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.469+0000 7f64dbfff640 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 0x55558280beb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06c6e0 tx=0x7f64c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93035c22b720ed76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.468+0000 7f64e1511640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 0x7f64cc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.469+0000 7f64c6ffd640 1 -- 172.21.15.29:0/4115041967 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004030 con 0x555582443670 2024-09-21T02:01:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.469+0000 7f64c6ffd640 1 -- 172.21.15.29:0/4115041967 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80054d0 con 0x555582443670 2024-09-21T02:01:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.469+0000 7f64c6ffd640 1 -- 172.21.15.29:0/4115041967 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004ce0 con 0x555582443670 2024-09-21T02:01:28.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e1511640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 0x7f64cc0b8aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005fd0 tx=0x7f64d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 msgr2=0x55558280beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 0x55558280beb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06c6e0 tx=0x7f64c80118d0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 shutdown_connections 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582443670 0x55558280beb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55558243b490 0x55558280b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 --2- 172.21.15.29:0/4115041967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558280b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 >> 172.21.15.29:0/4115041967 conn(0x555582458740 msgr2=0x555582809320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 shutdown_connections 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.470+0000 7f64e94e8740 1 -- 172.21.15.29:0/4115041967 wait complete. 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.471+0000 7f64e94e8740 1 -- start start 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.471+0000 7f64e94e8740 1 Processor -- start 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x5555828175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555825d2890 con 0x555582443670 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55558270bd40 con 0x555582464350 2024-09-21T02:01:28.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555582811ae0 con 0x55558243b490 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e94e8740 10 client.0 _ll_register_callbacks cb 0x55558276fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x5555828175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48590/0 (socket says 172.21.15.29:48590) 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64dbfff640 1 -- 172.21.15.29:0/116613355 learned_addr learned my addr 172.21.15.29:0/116613355 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.472+0000 7f64e1511640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 -- 172.21.15.29:0/116613355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 msgr2=0x5555828175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x5555828175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 -- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 msgr2=0x55558281cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 -- 172.21.15.29:0/116613355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558281d130 con 0x555582443670 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64e0d10640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x5555828175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:28.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64dbfff640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c8005110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd09a14d32817f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8028020 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c80054d0 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f64c800f750 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f64c801d070 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64e94e8740 10 client.18243 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.473+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555582836d30 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80054d0 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f64c80245a0 con 0x555582443670 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 10 client.18243 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 10 client.18243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 20 client.18243 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555582422cd0 0x55558283a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 4 client.18243 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555582783db0 con 0x555582422cd0 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64e0d10640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555582422cd0 0x55558283a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.474+0000 7f64bbfff640 10 client.18243 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64e0d10640 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555582422cd0 0x55558283a150 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6850d2258313be2 server_cookie=7f95f8f21fbfd8b7 in_seq=0 out_seq=0 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64bbfff640 1 -- 172.21.15.29:0/116613355 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555582783db0 con 0x555582422cd0 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64bbfff640 10 client.18243 handle_command_reply: tid=0 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64e94e8740 1 client.18243 shutdown 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 msgr2=0x555582817bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.475+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c8005110 comp rx=0 tx=0).stop 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 shutdown_connections 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555582422cd0 0x55558283a150 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582817bb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 --2- 172.21.15.29:0/116613355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x5555828175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 >> 172.21.15.29:0/116613355 conn(0x555582458740 msgr2=0x555582807850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64bbfff640 0 client.18243 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 shutdown_connections 2024-09-21T02:01:28.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 1 -- 172.21.15.29:0/116613355 wait complete. 2024-09-21T02:01:28.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.476+0000 7f64e94e8740 20 client.18243 trim_cache size 0 max 16384 2024-09-21T02:01:28.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.510+0000 7f64e94e8740 1 Processor -- start 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 -- start start 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x555582807850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582807d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b58f0 con 0x555582443670 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b7460 con 0x555582464350 2024-09-21T02:01:28.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64e94e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b5fc0 con 0x55558243b490 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582807d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.514+0000 7f64dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582807d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48592/0 (socket says 172.21.15.29:48592) 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64dbfff640 1 -- 172.21.15.29:0/307920929 learned_addr learned my addr 172.21.15.29:0/307920929 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x555582807850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 -- 172.21.15.29:0/307920929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 msgr2=0x55558281aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 -- 172.21.15.29:0/307920929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 msgr2=0x555582807d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582807d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.515+0000 7f64e0d10640 1 -- 172.21.15.29:0/307920929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558281da30 con 0x55558243b490 2024-09-21T02:01:28.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.518+0000 7f64e0d10640 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x555582807850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64d00caad0 tx=0x7f64d010d0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76fd2166ed451f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.522+0000 7f64c67fc640 1 -- 172.21.15.29:0/307920929 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0049040 con 0x55558243b490 2024-09-21T02:01:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.522+0000 7f64c67fc640 1 -- 172.21.15.29:0/307920929 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d00cd370 con 0x55558243b490 2024-09-21T02:01:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.522+0000 7f64c67fc640 1 -- 172.21.15.29:0/307920929 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00d6d40 con 0x55558243b490 2024-09-21T02:01:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.524+0000 7f64e94e8740 1 -- 172.21.15.29:0/307920929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 msgr2=0x555582807850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.524+0000 7f64e94e8740 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x555582807850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64d00caad0 tx=0x7f64d010d0e0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.526+0000 7f64e94e8740 1 -- 172.21.15.29:0/307920929 shutdown_connections 2024-09-21T02:01:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.526+0000 7f64e94e8740 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555582464350 0x55558281aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.526+0000 7f64e94e8740 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x555582807d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.526+0000 7f64e94e8740 1 --2- 172.21.15.29:0/307920929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55558243b490 0x555582807850 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.526+0000 7f64e94e8740 1 -- 172.21.15.29:0/307920929 >> 172.21.15.29:0/307920929 conn(0x555582458740 msgr2=0x555582815c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.531+0000 7f64e94e8740 1 -- 172.21.15.29:0/307920929 shutdown_connections 2024-09-21T02:01:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.532+0000 7f64e94e8740 1 -- 172.21.15.29:0/307920929 wait complete. 2024-09-21T02:01:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.533+0000 7f64e94e8740 1 -- start start 2024-09-21T02:01:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.533+0000 7f64e94e8740 1 Processor -- start 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x55558284bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582464350 0x555582832fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b6d60 con 0x555582443670 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b6690 con 0x55558243b490 2024-09-21T02:01:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1b38d0 con 0x555582464350 2024-09-21T02:01:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e94e8740 10 client.0 _ll_register_callbacks cb 0x7f64d010af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.534+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57416/0 (socket says 172.21.15.29:57416) 2024-09-21T02:01:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 -- 172.21.15.29:0/25842030 learned_addr learned my addr 172.21.15.29:0/25842030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 -- 172.21.15.29:0/25842030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582464350 msgr2=0x555582832fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582464350 0x555582832fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 -- 172.21.15.29:0/25842030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 msgr2=0x55558284bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x55558284bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.535+0000 7f64e0d10640 1 -- 172.21.15.29:0/25842030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558284c1a0 con 0x55558243b490 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.536+0000 7f64e0d10640 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64d00d7390 tx=0x7f64d00cd760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff6801be0d0c520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.537+0000 7f64bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.537+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0049040 con 0x55558243b490 2024-09-21T02:01:28.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.537+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d00cd370 con 0x55558243b490 2024-09-21T02:01:28.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.537+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f64d00d6f40 con 0x55558243b490 2024-09-21T02:01:28.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.538+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f64d00cd1b0 con 0x55558243b490 2024-09-21T02:01:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.538+0000 7f64e94e8740 10 client.28067 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.538+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5555828a4cc0 con 0x55558243b490 2024-09-21T02:01:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.539+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d00f3030 con 0x55558243b490 2024-09-21T02:01:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.539+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f64d01070f0 con 0x55558243b490 2024-09-21T02:01:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.539+0000 7f64e94e8740 10 client.28067 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64e94e8740 10 client.28067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64e94e8740 20 client.28067 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555823f8080 0x5555828a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64e94e8740 4 client.28067 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:01:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f64d00cd370 con 0x5555823f8080 2024-09-21T02:01:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.540+0000 7f64dbfff640 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555823f8080 0x5555828a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.548+0000 7f64dbfff640 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555823f8080 0x5555828a7490 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a69e623b3893ff server_cookie=60ef33166c4ab208 in_seq=0 out_seq=0 2024-09-21T02:01:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.548+0000 7f64bbfff640 10 client.28067 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.550+0000 7f64bbfff640 1 -- 172.21.15.29:0/25842030 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f64d00cd370 con 0x5555823f8080 2024-09-21T02:01:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.550+0000 7f64bbfff640 10 client.28067 handle_command_reply: tid=0 2024-09-21T02:01:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.550+0000 7f64e94e8740 1 client.28067 shutdown 2024-09-21T02:01:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.552+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 msgr2=0x55558284b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.552+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64d00d7390 tx=0x7f64d00cd760 comp rx=0 tx=0).stop 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 shutdown_connections 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5555823f8080 0x5555828a7490 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555582464350 0x555582832fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555582443670 0x55558284bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 --2- 172.21.15.29:0/25842030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55558243b490 0x55558284b6d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 >> 172.21.15.29:0/25842030 conn(0x555582458740 msgr2=0x55558280c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64bbfff640 0 client.28067 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.554+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 shutdown_connections 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.555+0000 7f64e94e8740 1 -- 172.21.15.29:0/25842030 wait complete. 2024-09-21T02:01:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.555+0000 7f64e94e8740 20 client.28067 trim_cache size 0 max 16384 2024-09-21T02:01:28.556 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:01:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T02:01:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "5313c304-821b-4232-9273-31da1c2b71bf", 2024-09-21T02:01:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T02:01:28.556 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:01:28.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.556+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 msgr2=0x7f64cc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.556+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 0x7f64cc0b8aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005fd0 tx=0x7f64d4009500 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 msgr2=0x7f64dc1a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f64d0004770 tx=0x7f64d00040c0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 shutdown_connections 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f64cc0b6590 0x7f64cc0b8aa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64dc116640 0x7f64dc1a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64dc10aee0 0x7f64dc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 --2- 172.21.15.29:0/2523268728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64dc10a590 0x7f64dc1a5230 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 >> 172.21.15.29:0/2523268728 conn(0x7f64dc070ec0 msgr2=0x7f64dc073040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 shutdown_connections 2024-09-21T02:01:28.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:28.557+0000 7f64e2f74640 1 -- 172.21.15.29:0/2523268728 wait complete. 2024-09-21T02:01:29.270 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:01:29.670 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:29.843 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 30 MiB/s wr, 1.23k op/s 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:29 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 30 MiB/s wr, 1.23k op/s 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:29.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:29 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 30 MiB/s wr, 1.23k op/s 2024-09-21T02:01:30.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T02:01:30.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.051 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:29 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:01:30.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.289+0000 7fec030e3640 1 -- 172.21.15.29:0/36025485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc108d20 msgr2=0x7febfc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.289+0000 7fec030e3640 1 --2- 172.21.15.29:0/36025485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc108d20 0x7febfc10f550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7febf4009be0 tx=0x7febf402a6c0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.289+0000 7fec030e3640 1 -- 172.21.15.29:0/36025485 shutdown_connections 2024-09-21T02:01:30.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.289+0000 7fec030e3640 1 --2- 172.21.15.29:0/36025485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc108d20 0x7febfc10f550 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.289+0000 7fec030e3640 1 --2- 172.21.15.29:0/36025485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 0x7febfc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.290+0000 7fec030e3640 1 --2- 172.21.15.29:0/36025485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc102ae0 0x7febfc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.290+0000 7fec030e3640 1 -- 172.21.15.29:0/36025485 >> 172.21.15.29:0/36025485 conn(0x7febfc0797b0 msgr2=0x7febfc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.290+0000 7fec030e3640 1 -- 172.21.15.29:0/36025485 shutdown_connections 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.290+0000 7fec030e3640 1 -- 172.21.15.29:0/36025485 wait complete. 2024-09-21T02:01:30.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.291+0000 7fec030e3640 1 Processor -- start 2024-09-21T02:01:30.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.291+0000 7fec030e3640 1 -- start start 2024-09-21T02:01:30.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.292+0000 7fec030e3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 0x7febfc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.292+0000 7fec030e3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 0x7febfc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec030e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 0x7febfc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 0x7febfc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 0x7febfc1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59788/0 (socket says 172.21.15.29:59788) 2024-09-21T02:01:30.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec00e7f640 1 -- 172.21.15.29:0/1384276797 learned_addr learned my addr 172.21.15.29:0/1384276797 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:30.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec01680640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 0x7febfc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1129e0 con 0x7febfc1083d0 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febfc112860 con 0x7febfc108d20 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7febfbfff640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 0x7febfc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.293+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7febfc112b60 con 0x7febfc102ae0 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 -- 172.21.15.29:0/1384276797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 msgr2=0x7febfc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 0x7febfc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 -- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 msgr2=0x7febfc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 0x7febfc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc19f360 con 0x7febfc108d20 2024-09-21T02:01:30.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec01680640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 0x7febfc19ec30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febf402abe0 tx=0x7febf4003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7febf9ffb640 1 -- 172.21.15.29:0/1384276797 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4044070 con 0x7febfc108d20 2024-09-21T02:01:30.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7febf9ffb640 1 -- 172.21.15.29:0/1384276797 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf40318f0 con 0x7febfc108d20 2024-09-21T02:01:30.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc19f5f0 con 0x7febfc108d20 2024-09-21T02:01:30.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.294+0000 7febf9ffb640 1 -- 172.21.15.29:0/1384276797 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4036040 con 0x7febfc108d20 2024-09-21T02:01:30.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.295+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc06b820 con 0x7febfc108d20 2024-09-21T02:01:30.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.298+0000 7febf9ffb640 1 -- 172.21.15.29:0/1384276797 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7febf4047070 con 0x7febfc108d20 2024-09-21T02:01:30.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7febf9ffb640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 0x7febec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7febf9ffb640 1 -- 172.21.15.29:0/1384276797 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7febf410b480 con 0x7febfc108d20 2024-09-21T02:01:30.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7fec00e7f640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 0x7febec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7fec09657740 1 Processor -- start 2024-09-21T02:01:30.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7fec09657740 1 -- start start 2024-09-21T02:01:30.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.300+0000 7fec00e7f640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 0x7febec0b8aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febf0009c90 tx=0x7febf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e05cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a264d7be50 con 0x55a264a3d390 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a264d6b280 con 0x55a264a351b0 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec09657740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a264d6f090 con 0x55a264a5e070 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec01680640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e05cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:59796/0 (socket says 172.21.15.29:59796) 2024-09-21T02:01:30.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7fec00e7f640 1 -- 172.21.15.29:0/2946585201 learned_addr learned my addr 172.21.15.29:0/2946585201 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.301+0000 7febfbfff640 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.302+0000 7fec00e7f640 1 -- 172.21.15.29:0/2946585201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 msgr2=0x55a264e057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.302+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.302+0000 7fec00e7f640 1 -- 172.21.15.29:0/2946585201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 msgr2=0x55a264e05cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.302+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e05cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.302+0000 7fec00e7f640 1 -- 172.21.15.29:0/2946585201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a264ac9650 con 0x55a264a5e070 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.303+0000 7febfbfff640 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.303+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febf0009fc0 tx=0x7febf0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847d0d17c9db22bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.303+0000 7febe6ffd640 1 -- 172.21.15.29:0/2946585201 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf002b070 con 0x55a264a5e070 2024-09-21T02:01:30.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.304+0000 7febe6ffd640 1 -- 172.21.15.29:0/2946585201 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf0026ad0 con 0x55a264a5e070 2024-09-21T02:01:30.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.304+0000 7febe6ffd640 1 -- 172.21.15.29:0/2946585201 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000ea20 con 0x55a264a5e070 2024-09-21T02:01:30.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.304+0000 7fec09657740 1 -- 172.21.15.29:0/2946585201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 msgr2=0x55a264e05270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.304+0000 7fec09657740 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febf0009fc0 tx=0x7febf0014000 comp rx=0 tx=0).stop 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 -- 172.21.15.29:0/2946585201 shutdown_connections 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e05cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 --2- 172.21.15.29:0/2946585201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e05270 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 -- 172.21.15.29:0/2946585201 >> 172.21.15.29:0/2946585201 conn(0x55a264a52460 msgr2=0x55a264e03160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 -- 172.21.15.29:0/2946585201 shutdown_connections 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 -- 172.21.15.29:0/2946585201 wait complete. 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 -- start start 2024-09-21T02:01:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.305+0000 7fec09657740 1 Processor -- start 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e10970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57426/0 (socket says 172.21.15.29:57426) 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a264bcc700 con 0x55a264a3d390 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a264d06110 con 0x55a264a351b0 2024-09-21T02:01:30.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec09657740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a264e13a50 con 0x55a264a5e070 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec00e7f640 1 -- 172.21.15.29:0/1584949561 learned_addr learned my addr 172.21.15.29:0/1584949561 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7febfbfff640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.306+0000 7fec01680640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7fec09657740 10 client.0 _ll_register_callbacks cb 0x55a264d1f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 -- 172.21.15.29:0/1584949561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 msgr2=0x55a264e1cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e1cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 -- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 msgr2=0x55a264e10970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e10970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 -- 172.21.15.29:0/1584949561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a264d33c40 con 0x55a264a3d390 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.307+0000 7febfbfff640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e10eb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7febfc105e70 tx=0x7febe800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc2261233c58b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.308+0000 7febdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8027060 con 0x55a264a3d390 2024-09-21T02:01:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7febe800e740 con 0x55a264a3d390 2024-09-21T02:01:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf0026ad0 con 0x55a264a3d390 2024-09-21T02:01:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7febe800e940 con 0x55a264a3d390 2024-09-21T02:01:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7fec09657740 10 client.18258 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.310+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a264e1f7d0 con 0x55a264a3d390 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf0026ad0 con 0x55a264a3d390 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7febe8021070 con 0x55a264a3d390 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 10 client.18258 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 20 client.18258 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a1c9f0 0x55a264e22db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 4 client.18258 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.311+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a264d7dda0 con 0x55a264a1c9f0 2024-09-21T02:01:30.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.312+0000 7fec00e7f640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a1c9f0 0x55a264e22db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.312+0000 7fec00e7f640 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a1c9f0 0x55a264e22db0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7fae41a7169334 server_cookie=8f349f9152dff2ff in_seq=0 out_seq=0 2024-09-21T02:01:30.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.312+0000 7febdbfff640 10 client.18258 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.313+0000 7febdbfff640 1 -- 172.21.15.29:0/1584949561 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a264d7dda0 con 0x55a264a1c9f0 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.313+0000 7febdbfff640 10 client.18258 handle_command_reply: tid=0 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.313+0000 7fec09657740 1 client.18258 shutdown 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.313+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 msgr2=0x55a264e10eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.313+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e10eb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7febfc105e70 tx=0x7febe800fa10 comp rx=0 tx=0).stop 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 shutdown_connections 2024-09-21T02:01:30.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a1c9f0 0x55a264e22db0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a5e070 0x55a264e1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7febdbfff640 0 client.18258 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a3d390 0x55a264e10eb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 --2- 172.21.15.29:0/1584949561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a351b0 0x55a264e10970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 >> 172.21.15.29:0/1584949561 conn(0x55a264a52460 msgr2=0x55a264e08400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 shutdown_connections 2024-09-21T02:01:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 1 -- 172.21.15.29:0/1584949561 wait complete. 2024-09-21T02:01:30.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.314+0000 7fec09657740 20 client.18258 trim_cache size 0 max 16384 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.340+0000 7fec09657740 1 Processor -- start 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.340+0000 7fec09657740 1 -- start start 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a3d390 0x55a264e02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ad1a0 con 0x55a264a351b0 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1aed10 con 0x55a264a1c9f0 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec09657740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ad870 con 0x55a264a3d390 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7febfbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec00e7f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57434/0 (socket says 172.21.15.29:57434) 2024-09-21T02:01:30.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7febfbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e02710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48652/0 (socket says 172.21.15.29:48652) 2024-09-21T02:01:30.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec00e7f640 1 -- 172.21.15.29:0/2509655772 learned_addr learned my addr 172.21.15.29:0/2509655772 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:30.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec01680640 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a3d390 0x55a264e02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.341+0000 7fec00e7f640 1 -- 172.21.15.29:0/2509655772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a3d390 msgr2=0x55a264e02c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a3d390 0x55a264e02c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7fec00e7f640 1 -- 172.21.15.29:0/2509655772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 msgr2=0x55a264e02710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e02710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7fec00e7f640 1 -- 172.21.15.29:0/2509655772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a264e0f8d0 con 0x55a264a1c9f0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7fec00e7f640 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7febf0029040 tx=0x7febf0009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24c4dd295e3fa6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7febe67fc640 1 -- 172.21.15.29:0/2509655772 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf002b040 con 0x55a264a1c9f0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7febe67fc640 1 -- 172.21.15.29:0/2509655772 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf002c040 con 0x55a264a1c9f0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.342+0000 7febe67fc640 1 -- 172.21.15.29:0/2509655772 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004640 con 0x55a264a1c9f0 2024-09-21T02:01:30.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- 172.21.15.29:0/2509655772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 msgr2=0x55a264e10e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7febf0029040 tx=0x7febf0009ab0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- 172.21.15.29:0/2509655772 shutdown_connections 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a3d390 0x55a264e02c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 --2- 172.21.15.29:0/2509655772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a1c9f0 0x55a264e10e30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- 172.21.15.29:0/2509655772 >> 172.21.15.29:0/2509655772 conn(0x55a264a52460 msgr2=0x55a264e10720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- 172.21.15.29:0/2509655772 shutdown_connections 2024-09-21T02:01:30.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- 172.21.15.29:0/2509655772 wait complete. 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.343+0000 7fec09657740 1 -- start start 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.344+0000 7fec09657740 1 Processor -- start 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.344+0000 7fec09657740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a1c9f0 0x55a264e196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.344+0000 7fec09657740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e14c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec09657740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec09657740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ae610 con 0x55a264a351b0 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec09657740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1adf40 con 0x55a264a3d390 2024-09-21T02:01:30.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec09657740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ab180 con 0x55a264a1c9f0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec09657740 10 client.0 _ll_register_callbacks cb 0x7febe8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:57440/0 (socket says 172.21.15.29:57440) 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 -- 172.21.15.29:0/4152951801 learned_addr learned my addr 172.21.15.29:0/4152951801 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec00e7f640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a1c9f0 0x55a264e196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7febfbfff640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e14c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 -- 172.21.15.29:0/4152951801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a1c9f0 msgr2=0x55a264e196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a1c9f0 0x55a264e196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 -- 172.21.15.29:0/4152951801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 msgr2=0x55a264e14c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e14c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 -- 172.21.15.29:0/4152951801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a264e8c7b0 con 0x55a264a3d390 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7fec01680640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7febf4108ea0 tx=0x7febf4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ea531312407fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:30.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.345+0000 7febdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf404e040 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7febf40da240 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf002c040 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7febf40d60a0 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 10 client.28085 fetch_fsmap learned FSMap version 48 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a264e8d8d0 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf002c040 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7febf40d08b0 con 0x55a264a3d390 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 10 client.28085 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 20 client.28085 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a5e070 0x55a264e900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 4 client.28085 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.346+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a264d79850 con 0x55a264a5e070 2024-09-21T02:01:30.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7fec00e7f640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a5e070 0x55a264e900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7febdbfff640 10 client.28085 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7fec00e7f640 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a5e070 0x55a264e900f0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5c940ab0863711 server_cookie=1e8b579a221bc325 in_seq=0 out_seq=0 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7febdbfff640 1 -- 172.21.15.29:0/4152951801 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55a264d79850 con 0x55a264a5e070 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7febdbfff640 10 client.28085 handle_command_reply: tid=0 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.347+0000 7fec09657740 1 client.28085 shutdown 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 msgr2=0x55a264e15320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7febf4108ea0 tx=0x7febf4009500 comp rx=0 tx=0).stop 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 shutdown_connections 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a264a5e070 0x55a264e900f0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7febdbfff640 0 client.28085 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:30.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a264a3d390 0x55a264e15320 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a264a351b0 0x55a264e14c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 --2- 172.21.15.29:0/4152951801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a264a1c9f0 0x55a264e196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 >> 172.21.15.29:0/4152951801 conn(0x55a264a52460 msgr2=0x55a264e062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 shutdown_connections 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 1 -- 172.21.15.29:0/4152951801 wait complete. 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.348+0000 7fec09657740 20 client.28085 trim_cache size 0 max 16384 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:01:30.353 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "5313c304-821b-4232-9273-31da1c2b71bf", 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.354 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.356 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.357 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:01:30.358 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.359 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.360 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.361 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:01:30.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:01:30.363 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:01:30.363 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:01:30.363 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:01:30.363 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:01:30.363 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.349+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 msgr2=0x7febec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.349+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 0x7febec0b8aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febf0009c90 tx=0x7febf0005f90 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 msgr2=0x7febfc19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 0x7febfc19ec30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febf402abe0 tx=0x7febf4003ba0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 shutdown_connections 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7febec0b6590 0x7febec0b8aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febfc108d20 0x7febfc19ec30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febfc1083d0 0x7febfc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 --2- 172.21.15.29:0/1384276797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febfc102ae0 0x7febfc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 >> 172.21.15.29:0/1384276797 conn(0x7febfc0797b0 msgr2=0x7febfc104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 shutdown_connections 2024-09-21T02:01:30.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:01:30.350+0000 7fec030e3640 1 -- 172.21.15.29:0/1384276797 wait complete. 2024-09-21T02:01:31.169 INFO:tasks.cephfs.filesystem:scrub status for tag:5313c304-821b-4232-9273-31da1c2b71bf - {'path': '/', 'tag': '5313c304-821b-4232-9273-31da1c2b71bf', 'options': 'recursive,force'} 2024-09-21T02:01:31.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:31 smithi029 ceph-mon[31228]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T02:01:31.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:31 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:01:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:31 smithi190 ceph-mon[34368]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T02:01:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:31 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:01:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:31 smithi066 ceph-mon[35001]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T02:01:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:31 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:01:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 775.0 s, 1914.0 tps, lat 52.961 ms stddev 66.557 2024-09-21T02:01:33.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:33 smithi029 ceph-mon[31228]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 18 MiB/s wr, 900 op/s 2024-09-21T02:01:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:33 smithi190 ceph-mon[34368]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 18 MiB/s wr, 900 op/s 2024-09-21T02:01:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:33 smithi066 ceph-mon[35001]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 18 MiB/s wr, 900 op/s 2024-09-21T02:01:35.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:35 smithi029 ceph-mon[31228]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-21T02:01:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:35 smithi190 ceph-mon[34368]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-21T02:01:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:35 smithi066 ceph-mon[35001]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-21T02:01:37.848 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:37 smithi029 ceph-mon[31228]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-09-21T02:01:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 780.0 s, 1440.6 tps, lat 68.757 ms stddev 81.819 2024-09-21T02:01:37.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:37 smithi190 ceph-mon[34368]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-09-21T02:01:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:37 smithi066 ceph-mon[35001]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-09-21T02:01:39.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:39 smithi190 ceph-mon[34368]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 647 op/s 2024-09-21T02:01:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:39 smithi066 ceph-mon[35001]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 647 op/s 2024-09-21T02:01:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:39 smithi029 ceph-mon[31228]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 647 op/s 2024-09-21T02:01:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:01:40.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:01:40.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:01:40.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:40 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:01:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:01:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:01:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:01:41.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:40 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:01:41.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:01:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:01:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:01:41.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:40 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.146+0000 7fd745027640 1 -- 172.21.15.29:0/2901738249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740101a90 msgr2=0x7fd740101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.146+0000 7fd745027640 1 --2- 172.21.15.29:0/2901738249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740101a90 0x7fd740101ea0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 -- 172.21.15.29:0/2901738249 shutdown_connections 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 --2- 172.21.15.29:0/2901738249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740109480 0x7fd74010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 --2- 172.21.15.29:0/2901738249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740106b20 0x7fd740108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 --2- 172.21.15.29:0/2901738249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740101a90 0x7fd740101ea0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 -- 172.21.15.29:0/2901738249 >> 172.21.15.29:0/2901738249 conn(0x7fd7400fd690 msgr2=0x7fd7400ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 -- 172.21.15.29:0/2901738249 shutdown_connections 2024-09-21T02:01:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.147+0000 7fd745027640 1 -- 172.21.15.29:0/2901738249 wait complete. 2024-09-21T02:01:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 Processor -- start 2024-09-21T02:01:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 -- start start 2024-09-21T02:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740106b20 0x7fd740074870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006d3f0 con 0x7fd740109480 2024-09-21T02:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006d270 con 0x7fd740101a90 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd745027640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006d570 con 0x7fd740106b20 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.148+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37552/0 (socket says 172.21.15.29:37552) 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 -- 172.21.15.29:0/894108647 learned_addr learned my addr 172.21.15.29:0/894108647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45600/0 (socket says 172.21.15.29:45600) 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73e59c640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740106b20 0x7fd740074870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 -- 172.21.15.29:0/894108647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740106b20 msgr2=0x7fd740074870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740106b20 0x7fd740074870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 -- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 msgr2=0x7fd740072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 -- 172.21.15.29:0/894108647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7400733f0 con 0x7fd740101a90 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73f59e640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd730009850 tx=0x7fd730003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.149+0000 7fd727fff640 1 -- 172.21.15.29:0/894108647 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73003f070 con 0x7fd740101a90 2024-09-21T02:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.150+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd740073680 con 0x7fd740101a90 2024-09-21T02:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.150+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd740073bc0 con 0x7fd740101a90 2024-09-21T02:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.150+0000 7fd727fff640 1 -- 172.21.15.29:0/894108647 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd730031d60 con 0x7fd740101a90 2024-09-21T02:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.150+0000 7fd727fff640 1 -- 172.21.15.29:0/894108647 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730049060 con 0x7fd740101a90 2024-09-21T02:01:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.152+0000 7fd727fff640 1 -- 172.21.15.29:0/894108647 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd730042070 con 0x7fd740101a90 2024-09-21T02:01:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 Processor -- start 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 -- start start 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ab4421d6f0 con 0x55ab43f031b0 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ab4420cb20 con 0x55ab43ee24d0 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd74c06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ab44210930 con 0x55ab43eda2f0 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45612/0 (socket says 172.21.15.29:45612) 2024-09-21T02:01:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 -- 172.21.15.29:0/3065459219 learned_addr learned my addr 172.21.15.29:0/3065459219 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 -- 172.21.15.29:0/3065459219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 msgr2=0x55ab442ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 -- 172.21.15.29:0/3065459219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 msgr2=0x55ab442ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73f59e640 1 -- 172.21.15.29:0/3065459219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab441d2530 con 0x55ab43f031b0 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd727fff640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 0x7fd7280bce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.153+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd738010750 tx=0x7fd738010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de93ddf9015809c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 0x7fd7280bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd727fff640 1 -- 172.21.15.29:0/894108647 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd730107c70 con 0x7fd740101a90 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd7257fa640 1 -- 172.21.15.29:0/3065459219 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800ead0 con 0x55ab43f031b0 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd7257fa640 1 -- 172.21.15.29:0/3065459219 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd738010e60 con 0x55ab43f031b0 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.154+0000 7fd7257fa640 1 -- 172.21.15.29:0/3065459219 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738011640 con 0x55ab43f031b0 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 -- 172.21.15.29:0/3065459219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 msgr2=0x55ab442aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd73ed9d640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 0x7fd7280bce30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd730004650 tx=0x7fd73000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd738010750 tx=0x7fd738010c10 comp rx=0 tx=0).stop 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 -- 172.21.15.29:0/3065459219 shutdown_connections 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3065459219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442aae50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.155+0000 7fd74c06a740 1 -- 172.21.15.29:0/3065459219 >> 172.21.15.29:0/3065459219 conn(0x55ab43ef7300 msgr2=0x55ab442a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.157+0000 7fd74c06a740 1 -- 172.21.15.29:0/3065459219 shutdown_connections 2024-09-21T02:01:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 -- 172.21.15.29:0/3065459219 wait complete. 2024-09-21T02:01:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 -- start start 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 Processor -- start 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 0x55ab442b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.158+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd74c06a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ab44144ab0 con 0x55ab43f031b0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd74c06a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ab441b2730 con 0x55ab43eda2f0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd74c06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ab442acb30 con 0x55ab43ee24d0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd74c06a740 10 client.0 _ll_register_callbacks cb 0x55ab442066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:37560/0 (socket says 172.21.15.29:37560) 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 -- 172.21.15.29:0/3347911266 learned_addr learned my addr 172.21.15.29:0/3347911266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 0x55ab442b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 -- 172.21.15.29:0/3347911266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 msgr2=0x55ab442b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 0x55ab442b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 -- 172.21.15.29:0/3347911266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 msgr2=0x55ab442b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 -- 172.21.15.29:0/3347911266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab43f6e790 con 0x55ab43eda2f0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 0x55ab442b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd73f59e640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7380127a0 tx=0x7fd738005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0f3878e650753 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.159+0000 7fd722ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738025040 con 0x55ab43eda2f0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd738004a40 con 0x55ab43eda2f0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd738010e60 con 0x55ab43eda2f0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd738004c40 con 0x55ab43eda2f0 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 10 client.28097 fetch_fsmap learned FSMap version 49 2024-09-21T02:01:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ab43fe7620 con 0x55ab43eda2f0 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd738010e60 con 0x55ab43eda2f0 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fd73800f040 con 0x55ab43eda2f0 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 10 client.28097 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 10 client.28097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 20 client.28097 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43ec1d80 0x55ab442c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 4 client.28097 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.160+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab4421f640 con 0x55ab43ec1d80 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.161+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43ec1d80 0x55ab442c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.161+0000 7fd722ff5640 10 client.28097 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.161+0000 7fd73e59c640 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43ec1d80 0x55ab442c1b70 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46bbf902a2fcc10 server_cookie=19dabc8ce7b703ee in_seq=0 out_seq=0 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd722ff5640 1 -- 172.21.15.29:0/3347911266 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab4421f640 con 0x55ab43ec1d80 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd722ff5640 10 client.28097 handle_command_reply: tid=0 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 client.28097 shutdown 2024-09-21T02:01:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 msgr2=0x55ab442b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7380127a0 tx=0x7fd738005a90 comp rx=0 tx=0).stop 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 shutdown_connections 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43ec1d80 0x55ab442c1b70 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43ee24d0 0x55ab442b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 --2- 172.21.15.29:0/3347911266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43eda2f0 0x55ab442b45e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 >> 172.21.15.29:0/3347911266 conn(0x55ab43ef7300 msgr2=0x55ab442a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd722ff5640 0 client.28097 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.162+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 shutdown_connections 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.163+0000 7fd74c06a740 1 -- 172.21.15.29:0/3347911266 wait complete. 2024-09-21T02:01:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.163+0000 7fd74c06a740 20 client.28097 trim_cache size 0 max 16384 2024-09-21T02:01:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.187+0000 7fd74c06a740 1 Processor -- start 2024-09-21T02:01:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.187+0000 7fd74c06a740 1 -- start start 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:55082/0 (socket says 172.21.15.29:55082) 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.188+0000 7fd73f59e640 1 -- 172.21.15.29:0/542813953 learned_addr learned my addr 172.21.15.29:0/542813953 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.189+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b1490 con 0x55ab43f031b0 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.189+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b2fd0 con 0x55ab43ee24d0 2024-09-21T02:01:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.189+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b1b60 con 0x55ab43eda2f0 2024-09-21T02:01:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 -- 172.21.15.29:0/542813953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 msgr2=0x55ab442a6d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442a6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 -- 172.21.15.29:0/542813953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 msgr2=0x55ab442a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.193+0000 7fd73f59e640 1 -- 172.21.15.29:0/542813953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab442b5100 con 0x55ab43eda2f0 2024-09-21T02:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.200+0000 7fd73f59e640 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd73800ef60 tx=0x7fd738009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71541d6ccd8d623 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.202+0000 7fd724ff9640 1 -- 172.21.15.29:0/542813953 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7380049a0 con 0x55ab43eda2f0 2024-09-21T02:01:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.202+0000 7fd724ff9640 1 -- 172.21.15.29:0/542813953 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73801c9d0 con 0x55ab43eda2f0 2024-09-21T02:01:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.202+0000 7fd724ff9640 1 -- 172.21.15.29:0/542813953 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73802c6e0 con 0x55ab43eda2f0 2024-09-21T02:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.204+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 msgr2=0x55ab442b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.204+0000 7fd74c06a740 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd73800ef60 tx=0x7fd738009600 comp rx=0 tx=0).stop 2024-09-21T02:01:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.210+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 shutdown_connections 2024-09-21T02:01:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.210+0000 7fd74c06a740 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43f031b0 0x55ab442a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.210+0000 7fd74c06a740 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43ee24d0 0x55ab442a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.210+0000 7fd74c06a740 1 --2- 172.21.15.29:0/542813953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442b3200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.210+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 >> 172.21.15.29:0/542813953 conn(0x55ab43ef7300 msgr2=0x55ab442b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.213+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 shutdown_connections 2024-09-21T02:01:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.213+0000 7fd74c06a740 1 -- 172.21.15.29:0/542813953 wait complete. 2024-09-21T02:01:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.213+0000 7fd74c06a740 1 -- start start 2024-09-21T02:01:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.213+0000 7fd74c06a740 1 Processor -- start 2024-09-21T02:01:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.214+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 0x55ab442baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:55088/0 (socket says 172.21.15.29:55088) 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd73f59e640 1 -- 172.21.15.29:0/199542684 learned_addr learned my addr 172.21.15.29:0/199542684 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43f031b0 0x55ab442bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b2900 con 0x55ab43ee24d0 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b2230 con 0x55ab43f031b0 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401af470 con 0x55ab43eda2f0 2024-09-21T02:01:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.215+0000 7fd74c06a740 10 client.0 _ll_register_callbacks cb 0x7fd738016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 0x55ab442baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 -- 172.21.15.29:0/199542684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 msgr2=0x55ab442bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 -- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43f031b0 msgr2=0x55ab442bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43f031b0 0x55ab442bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.220+0000 7fd73e59c640 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab442bbad0 con 0x55ab43ee24d0 2024-09-21T02:01:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.221+0000 7fd722ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.221+0000 7fd73e59c640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 0x55ab442baea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd734009f80 tx=0x7fd734012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed7c9289652f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.221+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734005220 con 0x55ab43ee24d0 2024-09-21T02:01:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.221+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fd7340053c0 con 0x55ab43ee24d0 2024-09-21T02:01:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.222+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73801c9d0 con 0x55ab43ee24d0 2024-09-21T02:01:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.222+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd7340055c0 con 0x55ab43ee24d0 2024-09-21T02:01:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.222+0000 7fd74c06a740 10 client.18282 fetch_fsmap learned FSMap version 49 2024-09-21T02:01:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.222+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ab4432c530 con 0x55ab43ee24d0 2024-09-21T02:01:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.222+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73801c9d0 con 0x55ab43ee24d0 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fd73401e9f0 con 0x55ab43ee24d0 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 10 client.18282 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 10 client.18282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 20 client.18282 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43e971c0 0x55ab4432ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 4 client.18282 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab4421b0f0 con 0x55ab43e971c0 2024-09-21T02:01:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.223+0000 7fd73f59e640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43e971c0 0x55ab4432ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.224+0000 7fd73f59e640 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43e971c0 0x55ab4432ee70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37279fa5b826d3dc server_cookie=8dc88cfa9c1f4ce3 in_seq=0 out_seq=0 2024-09-21T02:01:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.224+0000 7fd722ff5640 10 client.18282 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd722ff5640 1 -- 172.21.15.29:0/199542684 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab4421b0f0 con 0x55ab43e971c0 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd722ff5640 10 client.18282 handle_command_reply: tid=0 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 client.18282 shutdown 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 msgr2=0x55ab442baea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 0x55ab442baea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd734009f80 tx=0x7fd734012c70 comp rx=0 tx=0).stop 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 shutdown_connections 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55ab43e971c0 0x55ab4432ee70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab43f031b0 0x55ab442bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab43ee24d0 0x55ab442baea0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 --2- 172.21.15.29:0/199542684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab43eda2f0 0x55ab442bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 >> 172.21.15.29:0/199542684 conn(0x55ab43ef7300 msgr2=0x55ab442b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd722ff5640 0 client.18282 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 shutdown_connections 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 1 -- 172.21.15.29:0/199542684 wait complete. 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.242+0000 7fd74c06a740 20 client.18282 trim_cache size 0 max 16384 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:01:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 msgr2=0x7fd7280bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 0x7fd7280bce30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd730004650 tx=0x7fd73000b040 comp rx=0 tx=0).stop 2024-09-21T02:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 msgr2=0x7fd740074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd730009850 tx=0x7fd730003f80 comp rx=0 tx=0).stop 2024-09-21T02:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 shutdown_connections 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fd7280ba920 0x7fd7280bce30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd740109480 0x7fd740072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740106b20 0x7fd740074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 --2- 172.21.15.29:0/894108647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd740101a90 0x7fd740074270 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 >> 172.21.15.29:0/894108647 conn(0x7fd7400fd690 msgr2=0x7fd7401078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 shutdown_connections 2024-09-21T02:01:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:41.243+0000 7fd745027640 1 -- 172.21.15.29:0/894108647 wait complete. 2024-09-21T02:01:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:41 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:01:41.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:41 smithi190 ceph-mon[34368]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-09-21T02:01:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:41 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:01:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:41 smithi066 ceph-mon[35001]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-09-21T02:01:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:41 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:01:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:41 smithi029 ceph-mon[31228]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-09-21T02:01:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 785.0 s, 1733.4 tps, lat 57.480 ms stddev 74.387 2024-09-21T02:01:43.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:43 smithi190 ceph-mon[34368]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 672 op/s 2024-09-21T02:01:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:43 smithi066 ceph-mon[35001]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 672 op/s 2024-09-21T02:01:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:43 smithi029 ceph-mon[31228]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 672 op/s 2024-09-21T02:01:45.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:45 smithi190 ceph-mon[34368]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 615 op/s 2024-09-21T02:01:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:45 smithi066 ceph-mon[35001]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 615 op/s 2024-09-21T02:01:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:45 smithi029 ceph-mon[31228]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 615 op/s 2024-09-21T02:01:46.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:46 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:47.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:46 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:47.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:46 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 790.0 s, 1805.0 tps, lat 55.892 ms stddev 68.918 2024-09-21T02:01:47.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:47 smithi190 ceph-mon[34368]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 673 op/s 2024-09-21T02:01:47.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:47.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:47.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:01:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:47 smithi066 ceph-mon[35001]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 673 op/s 2024-09-21T02:01:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:01:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:47 smithi029 ceph-mon[31228]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 673 op/s 2024-09-21T02:01:48.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:01:49.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:49 smithi190 ceph-mon[34368]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-09-21T02:01:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:49 smithi066 ceph-mon[35001]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-09-21T02:01:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:49 smithi029 ceph-mon[31228]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-09-21T02:01:51.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:51 smithi190 ceph-mon[34368]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 700 op/s 2024-09-21T02:01:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:51 smithi066 ceph-mon[35001]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 700 op/s 2024-09-21T02:01:52.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:51 smithi029 ceph-mon[31228]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 700 op/s 2024-09-21T02:01:52.973 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 795.0 s, 1949.8 tps, lat 51.454 ms stddev 61.985 2024-09-21T02:01:53.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:53 smithi190 ceph-mon[34368]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 MiB/s wr, 633 op/s 2024-09-21T02:01:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:53 smithi066 ceph-mon[35001]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 MiB/s wr, 633 op/s 2024-09-21T02:01:54.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:53 smithi029 ceph-mon[31228]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 MiB/s wr, 633 op/s 2024-09-21T02:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.385+0000 7f8375fbe640 1 -- 172.21.15.29:0/3824632886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 msgr2=0x7f8368098a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.385+0000 7f8375fbe640 1 --2- 172.21.15.29:0/3824632886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368098a30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8364009900 tx=0x7f836402a410 comp rx=0 tx=0).stop 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.386+0000 7f8375fbe640 1 -- 172.21.15.29:0/3824632886 shutdown_connections 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.386+0000 7f8375fbe640 1 --2- 172.21.15.29:0/3824632886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f836809df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.386+0000 7f8375fbe640 1 --2- 172.21.15.29:0/3824632886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f836809b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.386+0000 7f8375fbe640 1 --2- 172.21.15.29:0/3824632886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368098a30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.386+0000 7f8375fbe640 1 -- 172.21.15.29:0/3824632886 >> 172.21.15.29:0/3824632886 conn(0x7f8368090590 msgr2=0x7f8368092a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.387+0000 7f8375fbe640 1 -- 172.21.15.29:0/3824632886 shutdown_connections 2024-09-21T02:01:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.387+0000 7f8375fbe640 1 -- 172.21.15.29:0/3824632886 wait complete. 2024-09-21T02:01:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.388+0000 7f8375fbe640 1 Processor -- start 2024-09-21T02:01:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.388+0000 7f8375fbe640 1 -- start start 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368096070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f8368094930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f8368094e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83680a1960 con 0x7f8368005210 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83680a17e0 con 0x7f836809ba80 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8375fbe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f83680a1ae0 con 0x7f8368098f70 2024-09-21T02:01:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f83757bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f8368094e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f8374fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368096070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f83757bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f8368094e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50850/0 (socket says 172.21.15.29:50850) 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f83757bd640 1 -- 172.21.15.29:0/2225836575 learned_addr learned my addr 172.21.15.29:0/2225836575 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.389+0000 7f836ffff640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f8368094930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 -- 172.21.15.29:0/2225836575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 msgr2=0x7f8368094e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f8368094e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 -- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 msgr2=0x7f8368096070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368096070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 -- 172.21.15.29:0/2225836575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8368130260 con 0x7f8368098f70 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836ffff640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f8368094930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e9f0 tx=0x7f835c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836dffb640 1 -- 172.21.15.29:0/2225836575 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c00ccc0 con 0x7f8368098f70 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836dffb640 1 -- 172.21.15.29:0/2225836575 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c0108e0 con 0x7f8368098f70 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83680954d0 con 0x7f8368098f70 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f836dffb640 1 -- 172.21.15.29:0/2225836575 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c018840 con 0x7f8368098f70 2024-09-21T02:01:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.390+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836813ae60 con 0x7f8368098f70 2024-09-21T02:01:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.393+0000 7f837e266740 1 Processor -- start 2024-09-21T02:01:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.393+0000 7f837e266740 1 -- start start 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced30d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced30d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced30e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ced27f6f0 con 0x556cecf3c2f0 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ced26eb20 con 0x556cecf651b0 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f837e266740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556ced272930 con 0x556cecf444d0 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f8374fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced30d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f8374fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced30d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36290/0 (socket says 172.21.15.29:36290) 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f8374fbc640 1 -- 172.21.15.29:0/3193660287 learned_addr learned my addr 172.21.15.29:0/3193660287 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced30d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f836dffb640 1 -- 172.21.15.29:0/2225836575 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f835c0189e0 con 0x7f8368098f70 2024-09-21T02:01:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 -- 172.21.15.29:0/3193660287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 msgr2=0x556ced30e200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced30e200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 -- 172.21.15.29:0/3193660287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 msgr2=0x556ced30d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced30d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.394+0000 7f83757bd640 1 -- 172.21.15.29:0/3193660287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ced234530 con 0x556cecf651b0 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.395+0000 7f83757bd640 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced30d040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8370000b60 tx=0x7f8370075a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2d9c965a51af3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.395+0000 7f835affd640 1 -- 172.21.15.29:0/3193660287 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837007f900 con 0x556cecf651b0 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.395+0000 7f835affd640 1 -- 172.21.15.29:0/3193660287 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8370084030 con 0x556cecf651b0 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.395+0000 7f835affd640 1 -- 172.21.15.29:0/3193660287 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370080e20 con 0x556cecf651b0 2024-09-21T02:01:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.395+0000 7f836dffb640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 0x7f83600b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f836dffb640 1 -- 172.21.15.29:0/2225836575 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f835c0e7530 con 0x7f8368098f70 2024-09-21T02:01:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f8374fbc640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 0x7f83600b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 -- 172.21.15.29:0/3193660287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 msgr2=0x556ced30d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced30d040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8370000b60 tx=0x7f8370075a80 comp rx=0 tx=0).stop 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 -- 172.21.15.29:0/3193660287 shutdown_connections 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced30e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced30d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 --2- 172.21.15.29:0/3193660287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced30d040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 -- 172.21.15.29:0/3193660287 >> 172.21.15.29:0/3193660287 conn(0x556cecf59300 msgr2=0x556ced308ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 -- 172.21.15.29:0/3193660287 shutdown_connections 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f837e266740 1 -- 172.21.15.29:0/3193660287 wait complete. 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.396+0000 7f8374fbc640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 0x7f83600b8aa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8364005f70 tx=0x7f836400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.397+0000 7f837e266740 1 -- start start 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.397+0000 7f837e266740 1 Processor -- start 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced316f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced317ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ced1a6ab0 con 0x556cecf3c2f0 2024-09-21T02:01:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ced214730 con 0x556cecf651b0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556ced30ed20 con 0x556cecf444d0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36304/0 (socket says 172.21.15.29:36304) 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 -- 172.21.15.29:0/3974734615 learned_addr learned my addr 172.21.15.29:0/3974734615 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f836ffff640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced316f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f83757bd640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced317ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f837e266740 10 client.0 _ll_register_callbacks cb 0x556ced2686b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 -- 172.21.15.29:0/3974734615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 msgr2=0x556ced316f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced316f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 -- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 msgr2=0x556ced317ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced317ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f8374fbc640 1 -- 172.21.15.29:0/3974734615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cecfd0790 con 0x556cecf3c2f0 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.398+0000 7f836ffff640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced316f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.399+0000 7f8374fbc640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8364031a20 tx=0x7f836402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d533e9bfb1a23caf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.399+0000 7f834ffff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:01:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.399+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364035040 con 0x556cecf3c2f0 2024-09-21T02:01:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.399+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8364005030 con 0x556cecf3c2f0 2024-09-21T02:01:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.400+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8370084030 con 0x556cecf3c2f0 2024-09-21T02:01:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.402+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8364005230 con 0x556cecf3c2f0 2024-09-21T02:01:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.402+0000 7f837e266740 10 client.18294 fetch_fsmap learned FSMap version 49 2024-09-21T02:01:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.402+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556ced049620 con 0x556cecf3c2f0 2024-09-21T02:01:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.403+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83640053d0 con 0x556cecf3c2f0 2024-09-21T02:01:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8364003940 con 0x556cecf3c2f0 2024-09-21T02:01:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 10 client.18294 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 10 client.18294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 20 client.18294 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf23d80 0x556ced323f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 4 client.18294 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8370084030 con 0x556cecf23d80 2024-09-21T02:01:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f836ffff640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf23d80 0x556ced323f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f836ffff640 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf23d80 0x556ced323f90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d058a24994cf83f server_cookie=9cc8c7e91058b25d in_seq=0 out_seq=0 2024-09-21T02:01:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.404+0000 7f834ffff640 10 client.18294 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f834ffff640 1 -- 172.21.15.29:0/3974734615 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8370084030 con 0x556cecf23d80 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f834ffff640 10 client.18294 handle_command_reply: tid=0 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f837e266740 1 client.18294 shutdown 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 msgr2=0x556ced3168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8364031a20 tx=0x7f836402a410 comp rx=0 tx=0).stop 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.405+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 shutdown_connections 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf23d80 0x556ced323f90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf651b0 0x556ced317ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f834ffff640 0 client.18294 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced316f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 --2- 172.21.15.29:0/3974734615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf3c2f0 0x556ced3168a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 >> 172.21.15.29:0/3974734615 conn(0x556cecf59300 msgr2=0x556ced307150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 shutdown_connections 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 1 -- 172.21.15.29:0/3974734615 wait complete. 2024-09-21T02:01:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.406+0000 7f837e266740 20 client.18294 trim_cache size 0 max 16384 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.424+0000 7f837e266740 1 Processor -- start 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.424+0000 7f837e266740 1 -- start start 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf23d80 0x556ced315aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf444d0 0x556ced307e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8368143c30 con 0x556cecf444d0 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8368145770 con 0x556cecf3c2f0 2024-09-21T02:01:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f837e266740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8368144300 con 0x556cecf23d80 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f83757bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf444d0 0x556ced307e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50880/0 (socket says 172.21.15.29:50880) 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 -- 172.21.15.29:0/1623060035 learned_addr learned my addr 172.21.15.29:0/1623060035 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f8374fbc640 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf23d80 0x556ced315aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 -- 172.21.15.29:0/1623060035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf23d80 msgr2=0x556ced315aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf23d80 0x556ced315aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.425+0000 7f836ffff640 1 -- 172.21.15.29:0/1623060035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf444d0 msgr2=0x556ced307e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f836ffff640 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf444d0 0x556ced307e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f836ffff640 1 -- 172.21.15.29:0/1623060035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ced315890 con 0x556cecf3c2f0 2024-09-21T02:01:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f836ffff640 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f835c000c00 tx=0x7f835c0e4eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfcac2be72473c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f835a7fc640 1 -- 172.21.15.29:0/1623060035 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c0b2150 con 0x556cecf3c2f0 2024-09-21T02:01:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f835a7fc640 1 -- 172.21.15.29:0/1623060035 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c0b7090 con 0x556cecf3c2f0 2024-09-21T02:01:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.426+0000 7f835a7fc640 1 -- 172.21.15.29:0/1623060035 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c0e7310 con 0x556cecf3c2f0 2024-09-21T02:01:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- 172.21.15.29:0/1623060035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 msgr2=0x556ced3078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f835c000c00 tx=0x7f835c0e4eb0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- 172.21.15.29:0/1623060035 shutdown_connections 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf444d0 0x556ced307e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced3078c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 --2- 172.21.15.29:0/1623060035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf23d80 0x556ced315aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- 172.21.15.29:0/1623060035 >> 172.21.15.29:0/1623060035 conn(0x556cecf59300 msgr2=0x556ced308410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- 172.21.15.29:0/1623060035 shutdown_connections 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- 172.21.15.29:0/1623060035 wait complete. 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.427+0000 7f837e266740 1 -- start start 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f837e266740 1 Processor -- start 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f837e266740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf23d80 0x556ced31e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f837e266740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f837e266740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced31ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f836ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f836ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50892/0 (socket says 172.21.15.29:50892) 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.428+0000 7f836ffff640 1 -- 172.21.15.29:0/4156485118 learned_addr learned my addr 172.21.15.29:0/4156485118 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83681450a0 con 0x556cecf23d80 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83681449d0 con 0x556cecf3c2f0 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8368141c10 con 0x556cecf444d0 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f837e266740 10 client.0 _ll_register_callbacks cb 0x7f8364009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 -- 172.21.15.29:0/4156485118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 msgr2=0x556ced31ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced31ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 -- 172.21.15.29:0/4156485118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf23d80 msgr2=0x556ced31e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf23d80 0x556ced31e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ced31ba70 con 0x556cecf3c2f0 2024-09-21T02:01:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.429+0000 7f836ffff640 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f835c0e4fb0 tx=0x7f835c0e5300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d7fa5a618b1414 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.431+0000 7f834ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:01:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.431+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c0c7070 con 0x556cecf3c2f0 2024-09-21T02:01:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.431+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f835c010d90 con 0x556cecf3c2f0 2024-09-21T02:01:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.432+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c0b7090 con 0x556cecf3c2f0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.432+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f835c0b2150 con 0x556cecf3c2f0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.432+0000 7f837e266740 10 client.28115 fetch_fsmap learned FSMap version 49 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.432+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556ced38eeb0 con 0x556cecf3c2f0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c0cf240 con 0x556cecf3c2f0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f835c0a90c0 con 0x556cecf3c2f0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 10 client.28115 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 10 client.28115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 20 client.28115 populate_metadata read hostname 'smithi029' 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf651b0 0x556ced3917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 4 client.28115 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f835c0b7090 con 0x556cecf651b0 2024-09-21T02:01:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.433+0000 7f8374fbc640 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf651b0 0x556ced3917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.434+0000 7f8374fbc640 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf651b0 0x556ced3917d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baed29de64fdc2bf server_cookie=8d674498b7f63a2b in_seq=0 out_seq=0 2024-09-21T02:01:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.434+0000 7f834ffff640 10 client.28115 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f834ffff640 1 -- 172.21.15.29:0/4156485118 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f835c0b7090 con 0x556cecf651b0 2024-09-21T02:01:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f834ffff640 10 client.28115 handle_command_reply: tid=0 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 client.28115 shutdown 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 msgr2=0x556ced31a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f835c0e4fb0 tx=0x7f835c0e5300 comp rx=0 tx=0).stop 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 shutdown_connections 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x556cecf651b0 0x556ced3917d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556cecf444d0 0x556ced31ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.441+0000 7f834ffff640 0 client.28115 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cecf3c2f0 0x556ced31a780 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 1 --2- 172.21.15.29:0/4156485118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cecf23d80 0x556ced31e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 >> 172.21.15.29:0/4156485118 conn(0x556cecf59300 msgr2=0x556ced308410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 shutdown_connections 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 1 -- 172.21.15.29:0/4156485118 wait complete. 2024-09-21T02:01:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.442+0000 7f837e266740 20 client.28115 trim_cache size 0 max 16384 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 msgr2=0x7f83600b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 0x7f83600b8aa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8364005f70 tx=0x7f836400b040 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 msgr2=0x7f8368094930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f8368094930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e9f0 tx=0x7f835c00edf0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 shutdown_connections 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f83600b6590 0x7f83600b8aa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f836809ba80 0x7f8368094e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8368098f70 0x7f8368094930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 --2- 172.21.15.29:0/2225836575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8368005210 0x7f8368096070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 >> 172.21.15.29:0/2225836575 conn(0x7f8368090590 msgr2=0x7f836809c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 shutdown_connections 2024-09-21T02:01:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:01:54.443+0000 7f8375fbe640 1 -- 172.21.15.29:0/2225836575 wait complete. 2024-09-21T02:01:55.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:55 smithi190 ceph-mon[34368]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.7 MiB/s wr, 609 op/s 2024-09-21T02:01:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:55 smithi066 ceph-mon[35001]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.7 MiB/s wr, 609 op/s 2024-09-21T02:01:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:55 smithi029 ceph-mon[31228]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.7 MiB/s wr, 609 op/s 2024-09-21T02:01:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 800.0 s, 1422.0 tps, lat 69.227 ms stddev 78.412 2024-09-21T02:01:57.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:57 smithi190 ceph-mon[34368]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 MiB/s wr, 672 op/s 2024-09-21T02:01:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:57 smithi066 ceph-mon[35001]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 MiB/s wr, 672 op/s 2024-09-21T02:01:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:57 smithi029 ceph-mon[31228]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 MiB/s wr, 672 op/s 2024-09-21T02:01:59.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:01:59 smithi190 ceph-mon[34368]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-21T02:02:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:01:59 smithi066 ceph-mon[35001]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-21T02:02:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:01:59 smithi029 ceph-mon[31228]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-21T02:02:01.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:02:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:01.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:01 smithi190 ceph-mon[34368]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-09-21T02:02:01.985 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:01 smithi029 ceph-mon[31228]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-09-21T02:02:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:01 smithi066 ceph-mon[35001]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f05fe7fc640 1 -- 172.21.15.29:0/3417003722 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8040bd0 con 0x7f060010dbf0 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 -- 172.21.15.29:0/3417003722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 msgr2=0x7f060010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 --2- 172.21.15.29:0/3417003722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060010e000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f05f801acb0 tx=0x7f05f803b900 comp rx=0 tx=0).stop 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 -- 172.21.15.29:0/3417003722 shutdown_connections 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 --2- 172.21.15.29:0/3417003722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f060011cba0 0x7f060011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 --2- 172.21.15.29:0/3417003722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060010e000 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 --2- 172.21.15.29:0/3417003722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0600111cd0 0x7f060010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.442+0000 7f060493e640 1 -- 172.21.15.29:0/3417003722 >> 172.21.15.29:0/3417003722 conn(0x7f060006e9f0 msgr2=0x7f060006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.445+0000 7f060493e640 1 -- 172.21.15.29:0/3417003722 shutdown_connections 2024-09-21T02:02:02.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.447+0000 7f060493e640 1 -- 172.21.15.29:0/3417003722 wait complete. 2024-09-21T02:02:02.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.448+0000 7f060493e640 1 Processor -- start 2024-09-21T02:02:02.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.448+0000 7f060493e640 1 -- start start 2024-09-21T02:02:02.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0600111cd0 0x7f0600118710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060011cba0 0x7f0600118c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0600114bf0 con 0x7f060011cba0 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0600114a70 con 0x7f060010dbf0 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f060493e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0600114d70 con 0x7f0600111cd0 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f05ff7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f05ff7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38640/0 (socket says 172.21.15.29:38640) 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f05ff7fe640 1 -- 172.21.15.29:0/2081680284 learned_addr learned my addr 172.21.15.29:0/2081680284 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f05feffd640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0600111cd0 0x7f0600118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.449+0000 7f05fffff640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060011cba0 0x7f0600118c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 -- 172.21.15.29:0/2081680284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0600111cd0 msgr2=0x7f0600118710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0600111cd0 0x7f0600118710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 -- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060011cba0 msgr2=0x7f0600118c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060011cba0 0x7f0600118c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 -- 172.21.15.29:0/2081680284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06001194b0 con 0x7f060010dbf0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.450+0000 7f05ff7fe640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c970 tx=0x7f05f000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.451+0000 7f05fcff9640 1 -- 172.21.15.29:0/2081680284 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0004220 con 0x7f060010dbf0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.451+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06001accc0 con 0x7f060010dbf0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.451+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001ad220 con 0x7f060010dbf0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.451+0000 7f05fcff9640 1 -- 172.21.15.29:0/2081680284 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f00043c0 con 0x7f060010dbf0 2024-09-21T02:02:02.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.452+0000 7f05fcff9640 1 -- 172.21.15.29:0/2081680284 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f001d060 con 0x7f060010dbf0 2024-09-21T02:02:02.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.454+0000 7f060c114740 1 Processor -- start 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 -- start start 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f05fcff9640 1 -- 172.21.15.29:0/2081680284 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f05f00079b0 con 0x7f060010dbf0 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd56950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 0x5571dbd56e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5571dbccce50 con 0x5571db98e390 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5571dbcbc280 con 0x5571db9af070 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f060c114740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5571dbcc0090 con 0x5571db9861b0 2024-09-21T02:02:02.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f05fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f05ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd56950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.455+0000 7f05fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38642/0 (socket says 172.21.15.29:38642) 2024-09-21T02:02:02.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.456+0000 7f05fffff640 1 -- 172.21.15.29:0/264366046 learned_addr learned my addr 172.21.15.29:0/264366046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:02.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.456+0000 7f05feffd640 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 0x5571dbd56e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 -- 172.21.15.29:0/264366046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 msgr2=0x5571dbd56950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd56950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 -- 172.21.15.29:0/264366046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 msgr2=0x5571dbd56e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 0x5571dbd56e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 -- 172.21.15.29:0/264366046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571dba1a650 con 0x5571db9af070 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05feffd640 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 0x5571dbd56e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fffff640 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bba0 tx=0x7f05f400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994189cb56776012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fcff9640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 0x7f05e80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05e5ffb640 1 -- 172.21.15.29:0/264366046 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4015070 con 0x5571db9af070 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05ff7fe640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 0x7f05e80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.457+0000 7f05fcff9640 1 -- 172.21.15.29:0/2081680284 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f05f00e7240 con 0x7f060010dbf0 2024-09-21T02:02:02.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.458+0000 7f05e5ffb640 1 -- 172.21.15.29:0/264366046 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f40054d0 con 0x5571db9af070 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.458+0000 7f05e5ffb640 1 -- 172.21.15.29:0/264366046 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011440 con 0x5571db9af070 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- 172.21.15.29:0/264366046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 msgr2=0x5571dbd56410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f05ff7fe640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 0x7f05e80b8ae0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5571dbd55090 tx=0x7f05f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bba0 tx=0x7f05f400f730 comp rx=0 tx=0).stop 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- 172.21.15.29:0/264366046 shutdown_connections 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db98e390 0x5571dbd56e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd56950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 --2- 172.21.15.29:0/264366046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd56410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- 172.21.15.29:0/264366046 >> 172.21.15.29:0/264366046 conn(0x5571db9a3460 msgr2=0x5571dbd54300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- 172.21.15.29:0/264366046 shutdown_connections 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- 172.21.15.29:0/264366046 wait complete. 2024-09-21T02:02:02.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.459+0000 7f060c114740 1 -- start start 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 Processor -- start 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd61b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd62040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f05fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd61b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5571dbb1d700 con 0x5571db9861b0 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f05fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd61b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33710/0 (socket says 172.21.15.29:33710) 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f05fffff640 1 -- 172.21.15.29:0/3837086126 learned_addr learned my addr 172.21.15.29:0/3837086126 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5571dbc57110 con 0x5571db9af070 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5571dbd64bf0 con 0x5571db98e390 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.460+0000 7f05fffff640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f060c114740 10 client.0 _ll_register_callbacks cb 0x5571dbc70990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:02.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05feffd640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd62040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 -- 172.21.15.29:0/3837086126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 msgr2=0x5571dbd62040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd62040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 -- 172.21.15.29:0/3837086126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 msgr2=0x5571dbd61b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd61b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 -- 172.21.15.29:0/3837086126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571dbc84c40 con 0x5571db9af070 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05fffff640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd6df70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000c00 tx=0x7f05f4004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660c45c3c740e5ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.461+0000 7f05deffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:02.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.462+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4015070 con 0x5571db9af070 2024-09-21T02:02:02.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.462+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f05f4005210 con 0x5571db9af070 2024-09-21T02:02:02.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.462+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f40054d0 con 0x5571db9af070 2024-09-21T02:02:02.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.462+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f05f4011440 con 0x5571db9af070 2024-09-21T02:02:02.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.463+0000 7f060c114740 10 client.28127 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:02.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.463+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5571dbd70940 con 0x5571db9af070 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.463+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f402f070 con 0x5571db9af070 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.463+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f05f4032080 con 0x5571db9af070 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 10 client.28127 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 10 client.28127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 20 client.28127 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db96d9f0 0x5571dbd73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 4 client.28127 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f05f40054d0 con 0x5571db96d9f0 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f05feffd640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db96d9f0 0x5571dbd73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f05deffd640 10 client.28127 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:02.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.464+0000 7f05feffd640 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db96d9f0 0x5571dbd73d00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a82d3ecdc4448d3 server_cookie=f4fe5440f6a56b3a in_seq=0 out_seq=0 2024-09-21T02:02:02.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f05deffd640 1 -- 172.21.15.29:0/3837086126 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f05f40054d0 con 0x5571db96d9f0 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f05deffd640 10 client.28127 handle_command_reply: tid=0 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 client.28127 shutdown 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 msgr2=0x5571dbd6df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd6df70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000c00 tx=0x7f05f4004b30 comp rx=0 tx=0).stop 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 shutdown_connections 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db96d9f0 0x5571dbd73d00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f05deffd640 0 client.28127 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db9af070 0x5571dbd6df70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd62040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 --2- 172.21.15.29:0/3837086126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd61b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 >> 172.21.15.29:0/3837086126 conn(0x5571db9a3460 msgr2=0x5571dbd595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 shutdown_connections 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 1 -- 172.21.15.29:0/3837086126 wait complete. 2024-09-21T02:02:02.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.466+0000 7f060c114740 20 client.28127 trim_cache size 0 max 16384 2024-09-21T02:02:02.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.488+0000 7f060c114740 1 Processor -- start 2024-09-21T02:02:02.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.488+0000 7f060c114740 1 -- start start 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd57b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd5ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00e420 con 0x5571db9861b0 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f05fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f05fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38674/0 (socket says 172.21.15.29:38674) 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f05fffff640 1 -- 172.21.15.29:0/557284672 learned_addr learned my addr 172.21.15.29:0/557284672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00ff60 con 0x5571db96d9f0 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00eaf0 con 0x5571db98e390 2024-09-21T02:02:02.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.489+0000 7f05fffff640 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd5ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 -- 172.21.15.29:0/557284672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 msgr2=0x5571dbd5ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd5ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 -- 172.21.15.29:0/557284672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 msgr2=0x5571dbd57b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd57b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 -- 172.21.15.29:0/557284672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571dbd54ec0 con 0x5571db96d9f0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05fffff640 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05f4004c40 tx=0x7f05f4005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba49bca894dd8e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05e57fa640 1 -- 172.21.15.29:0/557284672 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4015070 con 0x5571db96d9f0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05e57fa640 1 -- 172.21.15.29:0/557284672 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f4037040 con 0x5571db96d9f0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.490+0000 7f05e57fa640 1 -- 172.21.15.29:0/557284672 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011a80 con 0x5571db96d9f0 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 msgr2=0x5571dbd59d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05f4004c40 tx=0x7f05f4005d10 comp rx=0 tx=0).stop 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 shutdown_connections 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db98e390 0x5571dbd5ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db9861b0 0x5571dbd57b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 --2- 172.21.15.29:0/557284672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db96d9f0 0x5571dbd59d20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 >> 172.21.15.29:0/557284672 conn(0x5571db9a3460 msgr2=0x5571dbd57440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 shutdown_connections 2024-09-21T02:02:02.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.491+0000 7f060c114740 1 -- 172.21.15.29:0/557284672 wait complete. 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.492+0000 7f060c114740 1 -- start start 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.492+0000 7f060c114740 1 Processor -- start 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db96d9f0 0x5571dbd6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db98e390 0x5571dbd67fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00f890 con 0x5571db96d9f0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00f1c0 con 0x5571db98e390 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00c400 con 0x5571db9861b0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f060c114740 10 client.0 _ll_register_callbacks cb 0x7f05f400b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db96d9f0 0x5571dbd6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44394/0 (socket says 172.21.15.29:44394) 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 -- 172.21.15.29:0/2926777121 learned_addr learned my addr 172.21.15.29:0/2926777121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05fffff640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db98e390 0x5571dbd67fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 -- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db98e390 msgr2=0x5571dbd67fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db98e390 0x5571dbd67fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 -- 172.21.15.29:0/2926777121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db96d9f0 msgr2=0x5571dbd6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db96d9f0 0x5571dbd6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.493+0000 7f05feffd640 1 -- 172.21.15.29:0/2926777121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571dbd68660 con 0x5571db9861b0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.494+0000 7f05feffd640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f060006c910 tx=0x7f05f8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab11fe6b28670898 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:02.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.494+0000 7f05deffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:02.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8045030 con 0x5571db9861b0 2024-09-21T02:02:02.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f05f8003dc0 con 0x5571db9861b0 2024-09-21T02:02:02.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571dbd623f0 con 0x5571db9861b0 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f05f80407d0 con 0x5571db9861b0 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f060c114740 10 client.28084 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5571dbdde4b0 con 0x5571db9861b0 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.495+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5571dbd623f0 con 0x5571db9861b0 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f05f805c020 con 0x5571db9861b0 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 10 client.28084 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 10 client.28084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 20 client.28084 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db9af070 0x5571dbde0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:02.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 4 client.28084 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571dbcceda0 con 0x5571db9af070 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f05fffff640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db9af070 0x5571dbde0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f05deffd640 10 client.28084 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.496+0000 7f05fffff640 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db9af070 0x5571dbde0c50 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9439042a859f575e server_cookie=9569f6a769db67b9 in_seq=0 out_seq=0 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f05deffd640 1 -- 172.21.15.29:0/2926777121 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x5571dbcceda0 con 0x5571db9af070 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f05deffd640 10 client.28084 handle_command_reply: tid=0 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 client.28084 shutdown 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 msgr2=0x5571dbd67a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f060006c910 tx=0x7f05f8003b90 comp rx=0 tx=0).stop 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 shutdown_connections 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5571db9af070 0x5571dbde0c50 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571db98e390 0x5571dbd67fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571db9861b0 0x5571dbd67a80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 --2- 172.21.15.29:0/2926777121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571db96d9f0 0x5571dbd6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 >> 172.21.15.29:0/2926777121 conn(0x5571db9a3460 msgr2=0x5571dbd57cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f05deffd640 0 client.28084 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 shutdown_connections 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 1 -- 172.21.15.29:0/2926777121 wait complete. 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.497+0000 7f060c114740 20 client.28084 trim_cache size 0 max 16384 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:02:02.501 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.503 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.504 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:02:02.505 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.507 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.508 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.509 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:02.510 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:02:02.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 msgr2=0x7f05e80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 0x7f05e80b8ae0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5571dbd55090 tx=0x7f05f0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 msgr2=0x7f060011c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:02.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c970 tx=0x7f05f000ce30 comp rx=0 tx=0).stop 2024-09-21T02:02:02.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 shutdown_connections 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f05e80b65d0 0x7f05e80b8ae0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060011cba0 0x7f0600118c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0600111cd0 0x7f0600118710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 --2- 172.21.15.29:0/2081680284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f060010dbf0 0x7f060011c430 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 >> 172.21.15.29:0/2081680284 conn(0x7f060006e9f0 msgr2=0x7f060011d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 shutdown_connections 2024-09-21T02:02:02.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:02.498+0000 7f060493e640 1 -- 172.21.15.29:0/2081680284 wait complete. 2024-09-21T02:02:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 805.0 s, 2230.2 tps, lat 45.275 ms stddev 53.630 2024-09-21T02:02:03.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:02:03.474 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:03.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:03 smithi029 ceph-mon[31228]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 671 op/s 2024-09-21T02:02:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:03.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:03 smithi190 ceph-mon[34368]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 671 op/s 2024-09-21T02:02:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:03 smithi066 ceph-mon[35001]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 671 op/s 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 -- 172.21.15.29:0/1376739893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f01081b0 msgr2=0x7fe1f01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 --2- 172.21.15.29:0/1376739893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f01081b0 0x7fe1f01085c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1e001a910 tx=0x7fe1e003b470 comp rx=0 tx=0).stop 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 -- 172.21.15.29:0/1376739893 shutdown_connections 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 --2- 172.21.15.29:0/1376739893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0108b00 0x7fe1f010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 --2- 172.21.15.29:0/1376739893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f01081b0 0x7fe1f01085c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 --2- 172.21.15.29:0/1376739893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0102870 0x7fe1f0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 -- 172.21.15.29:0/1376739893 >> 172.21.15.29:0/1376739893 conn(0x7fe1f00fe2c0 msgr2=0x7fe1f0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 -- 172.21.15.29:0/1376739893 shutdown_connections 2024-09-21T02:02:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.551+0000 7fe1effff640 1 -- 172.21.15.29:0/1376739893 wait complete. 2024-09-21T02:02:04.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.552+0000 7fe1effff640 1 Processor -- start 2024-09-21T02:02:04.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.552+0000 7fe1effff640 1 -- start start 2024-09-21T02:02:04.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0102870 0x7fe1f0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f0108b00 0x7fe1f00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f006e650 con 0x7fe1f0108b00 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f006e4d0 con 0x7fe1f0102870 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1effff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f006e7d0 con 0x7fe1f01081b0 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1ee7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0102870 0x7fe1f0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1ee7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44408/0 (socket says 172.21.15.29:44408) 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1ee7fc640 1 -- 172.21.15.29:0/300182390 learned_addr learned my addr 172.21.15.29:0/300182390 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.553+0000 7fe1ef7fe640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f0108b00 0x7fe1f00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 -- 172.21.15.29:0/300182390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0102870 msgr2=0x7fe1f0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0102870 0x7fe1f0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 -- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f0108b00 msgr2=0x7fe1f00707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f0108b00 0x7fe1f00707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 -- 172.21.15.29:0/300182390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f0071010 con 0x7fe1f01081b0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.554+0000 7fe1ee7fc640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e0004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.555+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0046070 con 0x7fe1f01081b0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.555+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e0011070 con 0x7fe1f01081b0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.555+0000 7fe1effff640 1 -- 172.21.15.29:0/300182390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f019bd90 con 0x7fe1f01081b0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.555+0000 7fe1effff640 1 -- 172.21.15.29:0/300182390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f019c2f0 con 0x7fe1f01081b0 2024-09-21T02:02:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.555+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000cb50 con 0x7fe1f01081b0 2024-09-21T02:02:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.557+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f0102d00 con 0x7fe1f01081b0 2024-09-21T02:02:04.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.560+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe1e0040920 con 0x7fe1f01081b0 2024-09-21T02:02:04.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.561+0000 7fe1d7fff640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 0x7fe1d80b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:04.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.561+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe1e01086b0 con 0x7fe1f01081b0 2024-09-21T02:02:04.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.562+0000 7fe1eeffd640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 0x7fe1d80b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:04.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.562+0000 7fe1eeffd640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 0x7fe1d80b8c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0074c60 tx=0x7fe1e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:04.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.563+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1f0071010 con 0x7fe1f01081b0 2024-09-21T02:02:04.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.899+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f019cdd0 con 0x7fe1f01081b0 2024-09-21T02:02:04.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.900+0000 7fe1d7fff640 1 -- 172.21.15.29:0/300182390 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11970 (secure 0 0 0) 0x7fe1e00cf000 con 0x7fe1f01081b0 2024-09-21T02:02:04.900 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:02:04.901 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":49,"btime":"2024-09-21T02:01:30:578913+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:01:30.250452+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T02:02:04.901 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 49 2024-09-21T02:02:04.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 msgr2=0x7fe1d80b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 0x7fe1d80b8c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0074c60 tx=0x7fe1e4008040 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 msgr2=0x7fe1f00701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e0004aa0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 shutdown_connections 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe1d80b6710 0x7fe1d80b8c20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f0108b00 0x7fe1f00707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f01081b0 0x7fe1f00701e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 --2- 172.21.15.29:0/300182390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f0102870 0x7fe1f0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 >> 172.21.15.29:0/300182390 conn(0x7fe1f00fe2c0 msgr2=0x7fe1f01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 shutdown_connections 2024-09-21T02:02:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:04.902+0000 7fe1d5ffb640 1 -- 172.21.15.29:0/300182390 wait complete. 2024-09-21T02:02:05.788 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:02:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:05 smithi190 ceph-mon[34368]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 667 op/s 2024-09-21T02:02:05.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:05 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/300182390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:05 smithi066 ceph-mon[35001]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 667 op/s 2024-09-21T02:02:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:05 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/300182390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:05 smithi029 ceph-mon[31228]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 667 op/s 2024-09-21T02:02:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:05 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/300182390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:06.179 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:06.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 -- 172.21.15.29:0/2769426628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 msgr2=0x7f1ad8115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/2769426628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad8115fd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009900 tx=0x7f1ad402a410 comp rx=0 tx=0).stop 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 -- 172.21.15.29:0/2769426628 shutdown_connections 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/2769426628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad8118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/2769426628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad8115fd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/2769426628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 -- 172.21.15.29:0/2769426628 >> 172.21.15.29:0/2769426628 conn(0x7f1ad8070ec0 msgr2=0x7f1ad80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 -- 172.21.15.29:0/2769426628 shutdown_connections 2024-09-21T02:02:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.834+0000 7f1adf5b4640 1 -- 172.21.15.29:0/2769426628 wait complete. 2024-09-21T02:02:06.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.836+0000 7f1adf5b4640 1 Processor -- start 2024-09-21T02:02:06.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.836+0000 7f1adf5b4640 1 -- start start 2024-09-21T02:02:06.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad81a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad81a3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad81a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1add350640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad81a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811c2a0 con 0x7f1ad810aee0 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811c120 con 0x7f1ad810a590 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1adf5b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811c420 con 0x7f1ad8116640 2024-09-21T02:02:06.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.837+0000 7f1add350640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad81a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38722/0 (socket says 172.21.15.29:38722) 2024-09-21T02:02:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.838+0000 7f1add350640 1 -- 172.21.15.29:0/502051366 learned_addr learned my addr 172.21.15.29:0/502051366 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.838+0000 7f1addb51640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad81a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad81a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1addb51640 1 -- 172.21.15.29:0/502051366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 msgr2=0x7f1ad81a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1addb51640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad81a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1addb51640 1 -- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 msgr2=0x7f1ad81a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1addb51640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad81a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.839+0000 7f1addb51640 1 -- 172.21.15.29:0/502051366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad81a8c20 con 0x7f1ad8116640 2024-09-21T02:02:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.842+0000 7f1addb51640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad81a84f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800e970 tx=0x7f1ac800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.843+0000 7f1ace7fc640 1 -- 172.21.15.29:0/502051366 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800ccc0 con 0x7f1ad8116640 2024-09-21T02:02:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.843+0000 7f1ace7fc640 1 -- 172.21.15.29:0/502051366 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8005020 con 0x7f1ad8116640 2024-09-21T02:02:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.843+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad81a8f10 con 0x7f1ad8116640 2024-09-21T02:02:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.843+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81af670 con 0x7f1ad8116640 2024-09-21T02:02:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.843+0000 7f1ace7fc640 1 -- 172.21.15.29:0/502051366 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800ccc0 con 0x7f1ad8116640 2024-09-21T02:02:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.855+0000 7f1ae5b28740 1 Processor -- start 2024-09-21T02:02:06.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.859+0000 7f1ace7fc640 1 -- 172.21.15.29:0/502051366 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1ac80047a0 con 0x7f1ad8116640 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.860+0000 7f1ace7fc640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 0x7f1ac40b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.861+0000 7f1ace7fc640 1 -- 172.21.15.29:0/502051366 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac800cac0 con 0x7f1ad8116640 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.861+0000 7f1ae5b28740 1 -- start start 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.861+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f5eb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55853f563ea0 con 0x55853f225390 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55853f5532d0 con 0x55853f246070 2024-09-21T02:02:06.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55853f5570e0 con 0x55853f21d1b0 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1add350640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 0x7f1ac40b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1add350640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38730/0 (socket says 172.21.15.29:38730) 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1add350640 1 -- 172.21.15.29:0/2947462341 learned_addr learned my addr 172.21.15.29:0/2947462341 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.862+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f5eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.863+0000 7f1add350640 1 -- 172.21.15.29:0/2947462341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 msgr2=0x55853f5eb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.863+0000 7f1add350640 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f5eb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.863+0000 7f1add350640 1 -- 172.21.15.29:0/2947462341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 msgr2=0x55853f5fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.863+0000 7f1add350640 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.863+0000 7f1add350640 1 -- 172.21.15.29:0/2947462341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853f2b1650 con 0x55853f246070 2024-09-21T02:02:06.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.864+0000 7f1add350640 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00114e0 tx=0x7f1ad00119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c8bbf6e438d41e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.864+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 0x7f1ac40b8e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009230 tx=0x7f1ad4032b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.867+0000 7f1ac37fe640 1 -- 172.21.15.29:0/2947462341 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001f070 con 0x55853f246070 2024-09-21T02:02:06.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.867+0000 7f1ac37fe640 1 -- 172.21.15.29:0/2947462341 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad0012c20 con 0x55853f246070 2024-09-21T02:02:06.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 -- 172.21.15.29:0/2947462341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 msgr2=0x55853f5eb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00114e0 tx=0x7f1ad00119a0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 -- 172.21.15.29:0/2947462341 shutdown_connections 2024-09-21T02:02:06.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f5eb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/2947462341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f5eb100 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.868+0000 7f1ae5b28740 1 -- 172.21.15.29:0/2947462341 >> 172.21.15.29:0/2947462341 conn(0x55853f23a460 msgr2=0x55853f5ed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.869+0000 7f1ae5b28740 1 -- 172.21.15.29:0/2947462341 shutdown_connections 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.869+0000 7f1ae5b28740 1 -- 172.21.15.29:0/2947462341 wait complete. 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.869+0000 7f1ae5b28740 1 -- start start 2024-09-21T02:02:06.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.869+0000 7f1ae5b28740 1 Processor -- start 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.871+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.871+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f602450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.871+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f607470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.871+0000 7f1add350640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.871+0000 7f1add350640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44440/0 (socket says 172.21.15.29:44440) 2024-09-21T02:02:06.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.872+0000 7f1add350640 1 -- 172.21.15.29:0/682515950 learned_addr learned my addr 172.21.15.29:0/682515950 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:06.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.872+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55853f3b4700 con 0x55853f225390 2024-09-21T02:02:06.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.872+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55853f4ee160 con 0x55853f246070 2024-09-21T02:02:06.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.872+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55853f5fe5a0 con 0x55853f21d1b0 2024-09-21T02:02:06.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.872+0000 7f1ae5b28740 10 client.0 _ll_register_callbacks cb 0x55853f5079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:06.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.874+0000 7f1add350640 1 -- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 msgr2=0x55853f607470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.874+0000 7f1add350640 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f607470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.874+0000 7f1add350640 1 -- 172.21.15.29:0/682515950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 msgr2=0x55853f602450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.874+0000 7f1add350640 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f602450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.874+0000 7f1add350640 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853f51bc90 con 0x55853f21d1b0 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.875+0000 7f1add350640 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000a010 tx=0x7f1ad0009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1a880e7f4d18e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.875+0000 7f1ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:06.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.875+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001f040 con 0x55853f21d1b0 2024-09-21T02:02:06.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.875+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ad0003b70 con 0x55853f21d1b0 2024-09-21T02:02:06.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.876+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55853f565df0 con 0x55853f21d1b0 2024-09-21T02:02:06.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.877+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1ad0003d70 con 0x55853f21d1b0 2024-09-21T02:02:06.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.877+0000 7f1ae5b28740 10 client.28099 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:06.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.877+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55853f610450 con 0x55853f21d1b0 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.877+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55853f565df0 con 0x55853f21d1b0 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.877+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1ad00052a0 con 0x55853f21d1b0 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 10 client.28099 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 10 client.28099 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 20 client.28099 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f2049f0 0x55853f613ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 4 client.28099 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853f5618a0 con 0x55853f2049f0 2024-09-21T02:02:06.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.878+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f2049f0 0x55853f613ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.879+0000 7f1ac0ff9640 10 client.28099 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:06.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.879+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f2049f0 0x55853f613ce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc469aad8d93538f server_cookie=4bcfb64c3454ea54 in_seq=0 out_seq=0 2024-09-21T02:02:06.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.881+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/682515950 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55853f5618a0 con 0x55853f2049f0 2024-09-21T02:02:06.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.881+0000 7f1ac0ff9640 10 client.28099 handle_command_reply: tid=0 2024-09-21T02:02:06.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.881+0000 7f1ae5b28740 1 client.28099 shutdown 2024-09-21T02:02:06.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.882+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 msgr2=0x55853f601e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.882+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000a010 tx=0x7f1ad0009dc0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 shutdown_connections 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f2049f0 0x55853f613ce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f246070 0x55853f607470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ac0ff9640 0 client.28099 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f602450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/682515950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f601e70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.883+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 >> 172.21.15.29:0/682515950 conn(0x55853f23a460 msgr2=0x55853f5ec5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.884+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 shutdown_connections 2024-09-21T02:02:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.884+0000 7f1ae5b28740 1 -- 172.21.15.29:0/682515950 wait complete. 2024-09-21T02:02:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.884+0000 7f1ae5b28740 20 client.28099 trim_cache size 0 max 16384 2024-09-21T02:02:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.903+0000 7f1ae5b28740 1 Processor -- start 2024-09-21T02:02:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.903+0000 7f1ae5b28740 1 -- start start 2024-09-21T02:02:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f2049f0 0x55853f5e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f21d1b0 0x55853f5eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00e460 con 0x55853f225390 2024-09-21T02:02:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00ffa0 con 0x55853f21d1b0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00eb30 con 0x55853f2049f0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33802/0 (socket says 172.21.15.29:33802) 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 -- 172.21.15.29:0/1031071329 learned_addr learned my addr 172.21.15.29:0/1031071329 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1add350640 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f2049f0 0x55853f5e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f21d1b0 0x55853f5eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 -- 172.21.15.29:0/1031071329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f2049f0 msgr2=0x55853f5e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f2049f0 0x55853f5e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 -- 172.21.15.29:0/1031071329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f21d1b0 msgr2=0x55853f5eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f21d1b0 0x55853f5eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.904+0000 7f1addb51640 1 -- 172.21.15.29:0/1031071329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853f5edf10 con 0x55853f225390 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.905+0000 7f1addb51640 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80aa290 tx=0x7f1ac80b6740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d1b714ab52253f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.905+0000 7f1ac2ffd640 1 -- 172.21.15.29:0/1031071329 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80a9a10 con 0x55853f225390 2024-09-21T02:02:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.905+0000 7f1ac2ffd640 1 -- 172.21.15.29:0/1031071329 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8013e50 con 0x55853f225390 2024-09-21T02:02:06.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.905+0000 7f1ac2ffd640 1 -- 172.21.15.29:0/1031071329 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80a9bb0 con 0x55853f225390 2024-09-21T02:02:06.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 -- 172.21.15.29:0/1031071329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 msgr2=0x55853f5eb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80aa290 tx=0x7f1ac80b6740 comp rx=0 tx=0).stop 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 -- 172.21.15.29:0/1031071329 shutdown_connections 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f5eb5e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f21d1b0 0x55853f5eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/1031071329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f2049f0 0x55853f5e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 -- 172.21.15.29:0/1031071329 >> 172.21.15.29:0/1031071329 conn(0x55853f23a460 msgr2=0x55853f5fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 -- 172.21.15.29:0/1031071329 shutdown_connections 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.906+0000 7f1ae5b28740 1 -- 172.21.15.29:0/1031071329 wait complete. 2024-09-21T02:02:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.907+0000 7f1ae5b28740 1 -- start start 2024-09-21T02:02:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.908+0000 7f1ae5b28740 1 Processor -- start 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f2049f0 0x55853f6093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f6099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00f8d0 con 0x55853f225390 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00f200 con 0x55853f2049f0 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc00c440 con 0x55853f21d1b0 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1ae5b28740 10 client.0 _ll_register_callbacks cb 0x7f1ad001d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33812/0 (socket says 172.21.15.29:33812) 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 -- 172.21.15.29:0/475612168 learned_addr learned my addr 172.21.15.29:0/475612168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1add350640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f2049f0 0x55853f6093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1adcb4f640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f6099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 -- 172.21.15.29:0/475612168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 msgr2=0x55853f6099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f6099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 -- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f2049f0 msgr2=0x55853f6093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f2049f0 0x55853f6093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.909+0000 7f1addb51640 1 -- 172.21.15.29:0/475612168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853f60de30 con 0x55853f225390 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1addb51640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800e940 tx=0x7f1ac80a96a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264143967474378b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8098070 con 0x55853f225390 2024-09-21T02:02:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac8013e50 con 0x55853f225390 2024-09-21T02:02:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1ac80d0ba0 con 0x55853f225390 2024-09-21T02:02:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1ac801c040 con 0x55853f225390 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ae5b28740 10 client.18327 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55853f67e190 con 0x55853f225390 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.910+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8013e50 con 0x55853f225390 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac80adbb0 con 0x55853f225390 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 10 client.18327 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 10 client.18327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 20 client.18327 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f246070 0x55853f6808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 4 client.18327 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55853f55ecd0 con 0x55853f246070 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.911+0000 7f1add350640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f246070 0x55853f6808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1add350640 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f246070 0x55853f6808f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f96011c9bee35453 server_cookie=801e20a5736c2da7 in_seq=0 out_seq=0 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ac0ff9640 10 client.18327 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ac0ff9640 1 -- 172.21.15.29:0/475612168 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55853f55ecd0 con 0x55853f246070 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ac0ff9640 10 client.18327 handle_command_reply: tid=0 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 client.18327 shutdown 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 msgr2=0x55853f60d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800e940 tx=0x7f1ac80a96a0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 shutdown_connections 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55853f246070 0x55853f6808f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55853f225390 0x55853f60d7c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853f21d1b0 0x55853f6099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 --2- 172.21.15.29:0/475612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55853f2049f0 0x55853f6093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 >> 172.21.15.29:0/475612168 conn(0x55853f23a460 msgr2=0x55853f5fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ac0ff9640 0 client.18327 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 shutdown_connections 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.912+0000 7f1ae5b28740 1 -- 172.21.15.29:0/475612168 wait complete. 2024-09-21T02:02:06.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.913+0000 7f1ae5b28740 20 client.18327 trim_cache size 0 max 16384 2024-09-21T02:02:06.916 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:06.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.914+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 msgr2=0x7f1ac40b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.914+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 0x7f1ac40b8e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009230 tx=0x7f1ad4032b60 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.914+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 msgr2=0x7f1ad81a84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.914+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad81a84f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800e970 tx=0x7f1ac800ee30 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 shutdown_connections 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1ac40b6900 0x7f1ac40b8e10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8116640 0x7f1ad81a84f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad810aee0 0x7f1ad81a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 --2- 172.21.15.29:0/502051366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad810a590 0x7f1ad81a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 >> 172.21.15.29:0/502051366 conn(0x7f1ad8070ec0 msgr2=0x7f1ad80730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 shutdown_connections 2024-09-21T02:02:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:06.915+0000 7f1adf5b4640 1 -- 172.21.15.29:0/502051366 wait complete. 2024-09-21T02:02:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 -- 172.21.15.29:0/3222436144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec1025a0 msgr2=0x7f8bec102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/3222436144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec1025a0 0x7f8bec102a10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009900 tx=0x7f8bdc02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 -- 172.21.15.29:0/3222436144 shutdown_connections 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/3222436144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103fc0 0x7f8bec10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/3222436144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec103620 0x7f8bec103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/3222436144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec1025a0 0x7f8bec102a10 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 -- 172.21.15.29:0/3222436144 >> 172.21.15.29:0/3222436144 conn(0x7f8bec0fdee0 msgr2=0x7f8bec100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 -- 172.21.15.29:0/3222436144 shutdown_connections 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.577+0000 7f8bf2fad640 1 -- 172.21.15.29:0/3222436144 wait complete. 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 Processor -- start 2024-09-21T02:02:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 -- start start 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 0x7f8bec199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103620 0x7f8bec19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 0x7f8bec19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10e1f0 con 0x7f8bec103fc0 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10e070 con 0x7f8bec1025a0 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf2fad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10e370 con 0x7f8bec103620 2024-09-21T02:02:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf154a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 0x7f8bec19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 0x7f8bec199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 0x7f8bec199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38788/0 (socket says 172.21.15.29:38788) 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.578+0000 7f8bf0d49640 1 -- 172.21.15.29:0/1230990294 learned_addr learned my addr 172.21.15.29:0/1230990294 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103620 0x7f8bec19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 -- 172.21.15.29:0/1230990294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103620 msgr2=0x7f8bec19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103620 0x7f8bec19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 -- 172.21.15.29:0/1230990294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 msgr2=0x7f8bec199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 0x7f8bec199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 -- 172.21.15.29:0/1230990294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec19b970 con 0x7f8bec103fc0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf154a640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 0x7f8bec19b150 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8be400d680 tx=0x7f8be400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8be9ffb640 1 -- 172.21.15.29:0/1230990294 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4017070 con 0x7f8bec103fc0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8be9ffb640 1 -- 172.21.15.29:0/1230990294 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be4009c70 con 0x7f8bec103fc0 2024-09-21T02:02:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec19bc60 con 0x7f8bec103fc0 2024-09-21T02:02:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8be9ffb640 1 -- 172.21.15.29:0/1230990294 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4003b40 con 0x7f8bec103fc0 2024-09-21T02:02:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.579+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a2350 con 0x7f8bec103fc0 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 Processor -- start 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8be9ffb640 1 -- 172.21.15.29:0/1230990294 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8be4003ce0 con 0x7f8bec103fc0 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 -- start start 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570221b0 0x5604573ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 0x5604573cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56045733c6f0 con 0x560456ff92f0 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56045732bb20 con 0x5604570014d0 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.582+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56045732f930 con 0x5604570221b0 2024-09-21T02:02:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33852/0 (socket says 172.21.15.29:33852) 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 -- 172.21.15.29:0/566406679 learned_addr learned my addr 172.21.15.29:0/566406679 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bebfff640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570221b0 0x5604573ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bebfff640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 0x5604573cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8be9ffb640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 0x7f8bd80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 -- 172.21.15.29:0/566406679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570221b0 msgr2=0x5604573ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570221b0 0x5604573ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 -- 172.21.15.29:0/566406679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 msgr2=0x5604573cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 0x5604573cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 -- 172.21.15.29:0/566406679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604572f1530 con 0x560456ff92f0 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bebfff640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 0x5604573cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf154a640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 0x7f8bd80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8be9ffb640 1 -- 172.21.15.29:0/1230990294 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8be40e69c0 con 0x7f8bec103fc0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009a20 tx=0x7f8bdc0318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9da45fdeaed07d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8be2ffd640 1 -- 172.21.15.29:0/566406679 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc035030 con 0x560456ff92f0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8be2ffd640 1 -- 172.21.15.29:0/566406679 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc040070 con 0x560456ff92f0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.583+0000 7f8be2ffd640 1 -- 172.21.15.29:0/566406679 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc003cc0 con 0x560456ff92f0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/566406679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 msgr2=0x5604573ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf154a640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 0x7f8bd80b8aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8be40acba0 tx=0x7f8be40a14c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009a20 tx=0x7f8bdc0318f0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/566406679 shutdown_connections 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570014d0 0x5604573cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573ce8d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/566406679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570221b0 0x5604573ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/566406679 >> 172.21.15.29:0/566406679 conn(0x560457016300 msgr2=0x5604573c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/566406679 shutdown_connections 2024-09-21T02:02:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.584+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/566406679 wait complete. 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 -- start start 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 Processor -- start 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560456ff92f0 0x5604573d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560457263ab0 con 0x5604570014d0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5604572d1730 con 0x5604570221b0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5604573cfee0 con 0x560456ff92f0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf9ffb740 10 client.0 _ll_register_callbacks cb 0x5604573256b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf0d49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33860/0 (socket says 172.21.15.29:33860) 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bf0d49640 1 -- 172.21.15.29:0/1959359345 learned_addr learned my addr 172.21.15.29:0/1959359345 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560456ff92f0 0x5604573d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.585+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 -- 172.21.15.29:0/1959359345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560456ff92f0 msgr2=0x5604573d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560456ff92f0 0x5604573d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 -- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 msgr2=0x5604573d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 -- 172.21.15.29:0/1959359345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045708d790 con 0x5604570014d0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0096c0 tx=0x7f8bdc009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefce5607e7f7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc035040 con 0x5604570014d0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8bdc045470 con 0x5604570014d0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc040070 con 0x5604570014d0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc045670 con 0x5604570014d0 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf9ffb740 10 client.18339 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560457106620 con 0x5604570014d0 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc040070 con 0x5604570014d0 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.586+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8bdc03c890 con 0x5604570014d0 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 10 client.18339 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 20 client.18339 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fe0d80 0x5604573e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 4 client.18339 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56045733e640 con 0x560456fe0d80 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fe0d80 0x5604573e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bcbfff640 10 client.18339 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.587+0000 7f8bebfff640 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fe0d80 0x5604573e4cb0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd5dbde5b56c4cc server_cookie=bdafdb30d2023eed in_seq=0 out_seq=0 2024-09-21T02:02:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bcbfff640 1 -- 172.21.15.29:0/1959359345 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56045733e640 con 0x560456fe0d80 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bcbfff640 10 client.18339 handle_command_reply: tid=0 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 client.18339 shutdown 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 msgr2=0x5604573d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0096c0 tx=0x7f8bdc009230 comp rx=0 tx=0).stop 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 shutdown_connections 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fe0d80 0x5604573e4cb0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570014d0 0x5604573d80d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/1959359345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560456ff92f0 0x5604573d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bcbfff640 0 client.18339 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 >> 172.21.15.29:0/1959359345 conn(0x560457016300 msgr2=0x5604573c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 shutdown_connections 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/1959359345 wait complete. 2024-09-21T02:02:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.588+0000 7f8bf9ffb740 20 client.18339 trim_cache size 0 max 16384 2024-09-21T02:02:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 Processor -- start 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 -- start start 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560456ff92f0 0x5604573c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1a8c30 con 0x5604570221b0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1aa770 con 0x560456ff92f0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1a9300 con 0x5604570014d0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33864/0 (socket says 172.21.15.29:33864) 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bebfff640 1 -- 172.21.15.29:0/3726025157 learned_addr learned my addr 172.21.15.29:0/3726025157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560456ff92f0 0x5604573c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.603+0000 7f8bf0d49640 1 -- 172.21.15.29:0/3726025157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560456ff92f0 msgr2=0x5604573c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560456ff92f0 0x5604573c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bf0d49640 1 -- 172.21.15.29:0/3726025157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 msgr2=0x5604573d5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bf0d49640 1 -- 172.21.15.29:0/3726025157 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604573e5020 con 0x5604570014d0 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dfab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc03cc60 tx=0x7f8bdc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305b1e7b1af07574 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8be27fc640 1 -- 172.21.15.29:0/3726025157 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc035040 con 0x5604570014d0 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8be27fc640 1 -- 172.21.15.29:0/3726025157 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc040040 con 0x5604570014d0 2024-09-21T02:02:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.604+0000 7f8be27fc640 1 -- 172.21.15.29:0/3726025157 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc04fa30 con 0x5604570014d0 2024-09-21T02:02:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3726025157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 msgr2=0x5604573dfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dfab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc03cc60 tx=0x7f8bdc0056d0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3726025157 shutdown_connections 2024-09-21T02:02:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604570221b0 0x5604573d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dfab0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3726025157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560456ff92f0 0x5604573c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3726025157 >> 172.21.15.29:0/3726025157 conn(0x560457016300 msgr2=0x5604573c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3726025157 shutdown_connections 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3726025157 wait complete. 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- start start 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 Processor -- start 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1aa0a0 con 0x560456ff92f0 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1a99d0 con 0x5604570221b0 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1a6c10 con 0x5604570014d0 2024-09-21T02:02:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.605+0000 7f8bf9ffb740 10 client.0 _ll_register_callbacks cb 0x7f8bd400ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33866/0 (socket says 172.21.15.29:33866) 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 -- 172.21.15.29:0/3355860857 learned_addr learned my addr 172.21.15.29:0/3355860857 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 -- 172.21.15.29:0/3355860857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 msgr2=0x5604573dce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 -- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 msgr2=0x5604573dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 -- 172.21.15.29:0/3355860857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604573de150 con 0x560456ff92f0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bebfff640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005c40 tx=0x7f8bd400a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6140f03e57f31b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4003970 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc040040 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.606+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f8bd4003b10 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8bd4003d30 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 10 client.18345 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5604573df530 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc040040 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8bd4028070 con 0x560456ff92f0 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 10 client.18345 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 10 client.18345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 20 client.18345 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fb61c0 0x560457451650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 4 client.18345 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:02:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56045733a0f0 con 0x560456fb61c0 2024-09-21T02:02:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.607+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fb61c0 0x560457451650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.608+0000 7f8bcbfff640 10 client.18345 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.608+0000 7f8bf0d49640 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fb61c0 0x560457451650 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a3ee73f26a0551 server_cookie=538f81e496f4d5dc in_seq=0 out_seq=0 2024-09-21T02:02:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.675+0000 7f8bcbfff640 1 -- 172.21.15.29:0/3355860857 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56045733a0f0 con 0x560456fb61c0 2024-09-21T02:02:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.675+0000 7f8bcbfff640 10 client.18345 handle_command_reply: tid=0 2024-09-21T02:02:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.676+0000 7f8bf9ffb740 1 client.18345 shutdown 2024-09-21T02:02:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.676+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 msgr2=0x5604573dc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.676+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005c40 tx=0x7f8bd400a590 comp rx=0 tx=0).stop 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 shutdown_connections 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560456fb61c0 0x560457451650 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5604570221b0 0x5604573dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5604570014d0 0x5604573dce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 --2- 172.21.15.29:0/3355860857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560456ff92f0 0x5604573dc7c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.679+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 >> 172.21.15.29:0/3355860857 conn(0x560457016300 msgr2=0x5604573d5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.685+0000 7f8bcbfff640 0 client.18345 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.685+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 shutdown_connections 2024-09-21T02:02:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.685+0000 7f8bf9ffb740 1 -- 172.21.15.29:0/3355860857 wait complete. 2024-09-21T02:02:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.687+0000 7f8bf9ffb740 20 client.18345 trim_cache size 0 max 16384 2024-09-21T02:02:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:02:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:02:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:02:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:02:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.698+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 msgr2=0x7f8bd80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.698+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 0x7f8bd80b8aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8be40acba0 tx=0x7f8be40a14c0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.698+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 msgr2=0x7f8bec19b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.698+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 0x7f8bec19b150 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8be400d680 tx=0x7f8be400db40 comp rx=0 tx=0).stop 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 shutdown_connections 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f8bd80b6590 0x7f8bd80b8aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bec103fc0 0x7f8bec19b150 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bec103620 0x7f8bec19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 --2- 172.21.15.29:0/1230990294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bec1025a0 0x7f8bec199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.699+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 >> 172.21.15.29:0/1230990294 conn(0x7f8bec0fdee0 msgr2=0x7f8bec0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.701+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 shutdown_connections 2024-09-21T02:02:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:07.701+0000 7f8bf2fad640 1 -- 172.21.15.29:0/1230990294 wait complete. 2024-09-21T02:02:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 810.0 s, 1747.4 tps, lat 57.556 ms stddev 67.419 2024-09-21T02:02:07.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:07 smithi190 ceph-mon[34368]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 MiB/s wr, 746 op/s 2024-09-21T02:02:08.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:07 smithi066 ceph-mon[35001]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 MiB/s wr, 746 op/s 2024-09-21T02:02:08.098 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:07 smithi029 ceph-mon[31228]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 MiB/s wr, 746 op/s 2024-09-21T02:02:08.684 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:02:09.104 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:09.832 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:09 smithi029 ceph-mon[31228]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-09-21T02:02:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 -- 172.21.15.29:0/820116037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1841083a0 msgr2=0x7fc1841087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 --2- 172.21.15.29:0/820116037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1841083a0 0x7fc1841087b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc174009900 tx=0x7fc17402a410 comp rx=0 tx=0).stop 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 -- 172.21.15.29:0/820116037 shutdown_connections 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 --2- 172.21.15.29:0/820116037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184108cf0 0x7fc18410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 --2- 172.21.15.29:0/820116037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1841083a0 0x7fc1841087b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 --2- 172.21.15.29:0/820116037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc184102a60 0x7fc184102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 -- 172.21.15.29:0/820116037 >> 172.21.15.29:0/820116037 conn(0x7fc184079730 msgr2=0x7fc184079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 -- 172.21.15.29:0/820116037 shutdown_connections 2024-09-21T02:02:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.963+0000 7fc18c43a640 1 -- 172.21.15.29:0/820116037 wait complete. 2024-09-21T02:02:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 Processor -- start 2024-09-21T02:02:09.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 -- start start 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 0x7fc18419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc184108cf0 0x7fc18419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841129b0 con 0x7fc184108cf0 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc184112830 con 0x7fc1841083a0 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18c43a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc184112b30 con 0x7fc184102a60 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc1899d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18a9d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc184108cf0 0x7fc18419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc1899d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38846/0 (socket says 172.21.15.29:38846) 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc1899d5640 1 -- 172.21.15.29:0/1329493111 learned_addr learned my addr 172.21.15.29:0/1329493111 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.964+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 0x7fc18419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 -- 172.21.15.29:0/1329493111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 msgr2=0x7fc18419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:09.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 0x7fc18419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 -- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc184108cf0 msgr2=0x7fc18419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc184108cf0 0x7fc18419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 -- 172.21.15.29:0/1329493111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1841a65b0 con 0x7fc1841083a0 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 0x7fc18419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.965+0000 7fc1899d5640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc174005ab0 tx=0x7fc174003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.966+0000 7fc17b7fe640 1 -- 172.21.15.29:0/1329493111 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17403f070 con 0x7fc1841083a0 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.966+0000 7fc17b7fe640 1 -- 172.21.15.29:0/1329493111 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc174031b50 con 0x7fc1841083a0 2024-09-21T02:02:09.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.966+0000 7fc17b7fe640 1 -- 172.21.15.29:0/1329493111 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174035030 con 0x7fc1841083a0 2024-09-21T02:02:09.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.966+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc184199110 con 0x7fc1841083a0 2024-09-21T02:02:09.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.966+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841995a0 con 0x7fc1841083a0 2024-09-21T02:02:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 Processor -- start 2024-09-21T02:02:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc17b7fe640 1 -- 172.21.15.29:0/1329493111 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc174042070 con 0x7fc1841083a0 2024-09-21T02:02:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 -- start start 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb6173b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb616b0ea0 con 0x55cb61372390 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb616a02d0 con 0x55cb61393070 2024-09-21T02:02:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc1929ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb616a40e0 con 0x55cb6136a1b0 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc18a9d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc18a9d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38854/0 (socket says 172.21.15.29:38854) 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc18a9d7640 1 -- 172.21.15.29:0/2090145440 learned_addr learned my addr 172.21.15.29:0/2090145440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb6173b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.971+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44536/0 (socket says 172.21.15.29:44536) 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 -- 172.21.15.29:0/2090145440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 msgr2=0x55cb6173abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 -- 172.21.15.29:0/2090145440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 msgr2=0x55cb6173b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb6173b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 -- 172.21.15.29:0/2090145440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb613fe650 con 0x55cb61393070 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc17b7fe640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 0x7fc1700bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009f50 tx=0x7fc17c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce30230bbbdc67cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 0x7fc1700bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.972+0000 7fc178d9a640 1 -- 172.21.15.29:0/2090145440 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004d20 con 0x55cb61393070 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.973+0000 7fc17b7fe640 1 -- 172.21.15.29:0/1329493111 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1741073f0 con 0x7fc1841083a0 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.973+0000 7fc178d9a640 1 -- 172.21.15.29:0/2090145440 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c004050 con 0x55cb61393070 2024-09-21T02:02:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.973+0000 7fc178d9a640 1 -- 172.21.15.29:0/2090145440 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005280 con 0x55cb61393070 2024-09-21T02:02:09.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 -- 172.21.15.29:0/2090145440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 msgr2=0x55cb6173a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009f50 tx=0x7fc17c0118d0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 -- 172.21.15.29:0/2090145440 shutdown_connections 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb6173b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb6173abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2090145440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61393070 0x55cb6173a6a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc1929ae740 1 -- 172.21.15.29:0/2090145440 >> 172.21.15.29:0/2090145440 conn(0x55cb61387460 msgr2=0x55cb61738590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.974+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 0x7fc1700bcfb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc180009fd0 tx=0x7fc180005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.975+0000 7fc1929ae740 1 -- 172.21.15.29:0/2090145440 shutdown_connections 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.975+0000 7fc1929ae740 1 -- 172.21.15.29:0/2090145440 wait complete. 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.975+0000 7fc1929ae740 1 -- start start 2024-09-21T02:02:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.975+0000 7fc1929ae740 1 Processor -- start 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb61746300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61393070 0x55cb61752170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb61501700 con 0x55cb61372390 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb6163b160 con 0x55cb6136a1b0 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb61748e80 con 0x55cb61393070 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1929ae740 10 client.0 _ll_register_callbacks cb 0x55cb616549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38866/0 (socket says 172.21.15.29:38866) 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 -- 172.21.15.29:0/2143195746 learned_addr learned my addr 172.21.15.29:0/2143195746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc1899d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb61746300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61393070 0x55cb61752170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 -- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61393070 msgr2=0x55cb61752170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61393070 0x55cb61752170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 -- 172.21.15.29:0/2143195746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 msgr2=0x55cb61746300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb61746300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 -- 172.21.15.29:0/2143195746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb61668c90 con 0x55cb6136a1b0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc180012e00 tx=0x7fc180004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de2185cf5d45ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.976+0000 7fc16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180011940 con 0x55cb6136a1b0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc180011ae0 con 0x55cb6136a1b0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c004050 con 0x55cb6136a1b0 2024-09-21T02:02:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc180011d20 con 0x55cb6136a1b0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 10 client.28178 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cb61754c50 con 0x55cb6136a1b0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c004050 con 0x55cb6136a1b0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc180027070 con 0x55cb6136a1b0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 10 client.28178 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 10 client.28178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 20 client.28178 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb613519f0 0x55cb61758100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 4 client.28178 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.977+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb616b2df0 con 0x55cb613519f0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.978+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb613519f0 0x55cb61758100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.978+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb613519f0 0x55cb61758100 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6edf77c8e6e63513 server_cookie=31afba80c93ca28e in_seq=0 out_seq=0 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.978+0000 7fc16dffb640 10 client.28178 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:02:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc16dffb640 1 -- 172.21.15.29:0/2143195746 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb616b2df0 con 0x55cb613519f0 2024-09-21T02:02:09.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc16dffb640 10 client.28178 handle_command_reply: tid=0 2024-09-21T02:02:09.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 client.28178 shutdown 2024-09-21T02:02:09.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:09 smithi190 ceph-mon[34368]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-09-21T02:02:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 msgr2=0x55cb61745dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc180012e00 tx=0x7fc180004730 comp rx=0 tx=0).stop 2024-09-21T02:02:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 shutdown_connections 2024-09-21T02:02:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb613519f0 0x55cb61758100 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61393070 0x55cb61752170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61372390 0x55cb61746300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2143195746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb61745dc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.979+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 >> 172.21.15.29:0/2143195746 conn(0x55cb61387460 msgr2=0x55cb6173d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.982+0000 7fc16dffb640 0 client.28178 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.982+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 shutdown_connections 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.982+0000 7fc1929ae740 1 -- 172.21.15.29:0/2143195746 wait complete. 2024-09-21T02:02:09.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:09.983+0000 7fc1929ae740 20 client.28178 trim_cache size 0 max 16384 2024-09-21T02:02:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 Processor -- start 2024-09-21T02:02:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 -- start start 2024-09-21T02:02:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb61746280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61372390 0x55cb61752070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb61742fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841ad1b0 con 0x55cb61393070 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841aecf0 con 0x55cb61372390 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.003+0000 7fc1929ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841ad880 con 0x55cb6136a1b0 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb61746280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb61746280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:44550/0 (socket says 172.21.15.29:44550) 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a1d6640 1 -- 172.21.15.29:0/1871953020 learned_addr learned my addr 172.21.15.29:0/1871953020 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb61742fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 -- 172.21.15.29:0/1871953020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 msgr2=0x55cb61746280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb61746280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 -- 172.21.15.29:0/1871953020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61372390 msgr2=0x55cb61752070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:02:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61372390 0x55cb61752070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.004+0000 7fc18a9d7640 1 -- 172.21.15.29:0/1871953020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb61746e60 con 0x55cb61393070 2024-09-21T02:02:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.005+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb61742fb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc17c004750 tx=0x7fc17c009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21e9fe588af5c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:10.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.008+0000 7fc16ffff640 1 -- 172.21.15.29:0/1871953020 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004050 con 0x55cb61393070 2024-09-21T02:02:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.010+0000 7fc16ffff640 1 -- 172.21.15.29:0/1871953020 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c004780 con 0x55cb61393070 2024-09-21T02:02:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.010+0000 7fc16ffff640 1 -- 172.21.15.29:0/1871953020 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005250 con 0x55cb61393070 2024-09-21T02:02:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.011+0000 7fc1929ae740 1 -- 172.21.15.29:0/1871953020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 msgr2=0x55cb61742fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.011+0000 7fc1929ae740 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb61742fb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc17c004750 tx=0x7fc17c009cc0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.016+0000 7fc1929ae740 1 -- 172.21.15.29:0/1871953020 shutdown_connections 2024-09-21T02:02:10.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.016+0000 7fc1929ae740 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb61742fb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.016+0000 7fc1929ae740 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb61372390 0x55cb61752070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.016+0000 7fc1929ae740 1 --2- 172.21.15.29:0/1871953020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb6136a1b0 0x55cb61746280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.016+0000 7fc1929ae740 1 -- 172.21.15.29:0/1871953020 >> 172.21.15.29:0/1871953020 conn(0x55cb61387460 msgr2=0x55cb61745b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:10.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.020+0000 7fc1929ae740 1 -- 172.21.15.29:0/1871953020 shutdown_connections 2024-09-21T02:02:10.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.020+0000 7fc1929ae740 1 -- 172.21.15.29:0/1871953020 wait complete. 2024-09-21T02:02:10.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.021+0000 7fc1929ae740 1 -- start start 2024-09-21T02:02:10.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.021+0000 7fc1929ae740 1 Processor -- start 2024-09-21T02:02:10.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb6174a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 0x55cb6174a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc1929ae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb6174aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb6174a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc18a1d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb6174a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:38876/0 (socket says 172.21.15.29:38876) 2024-09-21T02:02:10.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc1899d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 0x55cb6174a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc18a1d6640 1 -- 172.21.15.29:0/2743267663 learned_addr learned my addr 172.21.15.29:0/2743267663 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.022+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841ae620 con 0x55cb61393070 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.023+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841adf50 con 0x55cb6136a1b0 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.023+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841ab190 con 0x55cb61372390 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.023+0000 7fc1929ae740 10 client.0 _ll_register_callbacks cb 0x7fc180009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:10.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.023+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb6174aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 -- 172.21.15.29:0/2743267663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 msgr2=0x55cb6174a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb6174a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 -- 172.21.15.29:0/2743267663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 msgr2=0x55cb6174aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb6174aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb617c1b00 con 0x55cb61372390 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc18a9d7640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb6174aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc1899d5640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 0x55cb6174a5b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc174104f50 tx=0x7fc174031f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a4b42b75ceace0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:10.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.024+0000 7fc16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:10.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.028+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174049040 con 0x55cb61372390 2024-09-21T02:02:10.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.028+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc1740c98b0 con 0x55cb61372390 2024-09-21T02:02:10.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.030+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c004780 con 0x55cb61372390 2024-09-21T02:02:10.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.031+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc1740c94e0 con 0x55cb61372390 2024-09-21T02:02:10.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.031+0000 7fc1929ae740 10 client.28141 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:10.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.031+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cb617c2d50 con 0x55cb61372390 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1740cd150 con 0x55cb61372390 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc17404c060 con 0x55cb61372390 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 10 client.28141 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 10 client.28141 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 20 client.28141 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb61326da0 0x55cb617c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 4 client.28141 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc17c004780 con 0x55cb61326da0 2024-09-21T02:02:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.034+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb61326da0 0x55cb617c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.036+0000 7fc18a1d6640 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb61326da0 0x55cb617c5630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c5292d9f1056650 server_cookie=a7bd886379971083 in_seq=0 out_seq=0 2024-09-21T02:02:10.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.036+0000 7fc16dffb640 10 client.28141 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:02:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.036+0000 7fc16dffb640 1 -- 172.21.15.29:0/2743267663 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc17c004780 con 0x55cb61326da0 2024-09-21T02:02:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.036+0000 7fc16dffb640 10 client.28141 handle_command_reply: tid=0 2024-09-21T02:02:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.036+0000 7fc1929ae740 1 client.28141 shutdown 2024-09-21T02:02:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 msgr2=0x55cb6174a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 0x55cb6174a5b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc174104f50 tx=0x7fc174031f50 comp rx=0 tx=0).stop 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 shutdown_connections 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb61393070 0x55cb6174aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x55cb61326da0 0x55cb617c5630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb61372390 0x55cb6174a5b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 --2- 172.21.15.29:0/2743267663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb6136a1b0 0x55cb6174a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 >> 172.21.15.29:0/2743267663 conn(0x55cb61387460 msgr2=0x55cb6173b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc16dffb640 0 client.28141 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 shutdown_connections 2024-09-21T02:02:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.037+0000 7fc1929ae740 1 -- 172.21.15.29:0/2743267663 wait complete. 2024-09-21T02:02:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.043+0000 7fc1929ae740 20 client.28141 trim_cache size 0 max 16384 2024-09-21T02:02:10.043 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.044+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 msgr2=0x7fc1700bcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.044+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 0x7fc1700bcfb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc180009fd0 tx=0x7fc180005f90 comp rx=0 tx=0).stop 2024-09-21T02:02:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.044+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 msgr2=0x7fc18419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.044+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc174005ab0 tx=0x7fc174003b90 comp rx=0 tx=0).stop 2024-09-21T02:02:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 shutdown_connections 2024-09-21T02:02:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc1700baaa0 0x7fc1700bcfb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc184108cf0 0x7fc18419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1841083a0 0x7fc18419b860 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 --2- 172.21.15.29:0/1329493111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc184102a60 0x7fc18419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 >> 172.21.15.29:0/1329493111 conn(0x7fc184079730 msgr2=0x7fc184104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 shutdown_connections 2024-09-21T02:02:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:10.046+0000 7fc18c43a640 1 -- 172.21.15.29:0/1329493111 wait complete. 2024-09-21T02:02:10.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:09 smithi066 ceph-mon[35001]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-09-21T02:02:11.392 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:02:11.774 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:11.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:11 smithi190 ceph-mon[34368]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 688 op/s 2024-09-21T02:02:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:11 smithi066 ceph-mon[35001]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 688 op/s 2024-09-21T02:02:12.052 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:11 smithi029 ceph-mon[31228]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 688 op/s 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 -- 172.21.15.29:0/1126445628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24111cd0 msgr2=0x7fee2410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 --2- 172.21.15.29:0/1126445628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24111cd0 0x7fee2410d6b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fee14009ca0 tx=0x7fee1402a8d0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 -- 172.21.15.29:0/1126445628 shutdown_connections 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 --2- 172.21.15.29:0/1126445628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2411cba0 0x7fee2411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 --2- 172.21.15.29:0/1126445628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee2410dbf0 0x7fee2410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 --2- 172.21.15.29:0/1126445628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24111cd0 0x7fee2410d6b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.373+0000 7fee29d68640 1 -- 172.21.15.29:0/1126445628 >> 172.21.15.29:0/1126445628 conn(0x7fee2406e9f0 msgr2=0x7fee2406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.377+0000 7fee29d68640 1 -- 172.21.15.29:0/1126445628 shutdown_connections 2024-09-21T02:02:12.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.377+0000 7fee29d68640 1 -- 172.21.15.29:0/1126445628 wait complete. 2024-09-21T02:02:12.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 Processor -- start 2024-09-21T02:02:12.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 -- start start 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2410dbf0 0x7fee241a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee24111cd0 0x7fee241a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee24122720 con 0x7fee24111cd0 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee241225a0 con 0x7fee2410dbf0 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.378+0000 7fee29d68640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee241228a0 con 0x7fee2411cba0 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48200/0 (socket says 172.21.15.29:48200) 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 -- 172.21.15.29:0/2413929080 learned_addr learned my addr 172.21.15.29:0/2413929080 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:12.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee23fff640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee24111cd0 0x7fee241a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 -- 172.21.15.29:0/2413929080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2410dbf0 msgr2=0x7fee241a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2410dbf0 0x7fee241a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 -- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee24111cd0 msgr2=0x7fee241a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee24111cd0 0x7fee241a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.379+0000 7fee29567640 1 -- 172.21.15.29:0/2413929080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee241af0e0 con 0x7fee2411cba0 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.380+0000 7fee29567640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00d750 tx=0x7fee1c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.380+0000 7fee21ffb640 1 -- 172.21.15.29:0/2413929080 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c017070 con 0x7fee2411cba0 2024-09-21T02:02:12.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.380+0000 7fee29d68640 1 -- 172.21.15.29:0/2413929080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241af3d0 con 0x7fee2411cba0 2024-09-21T02:02:12.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.380+0000 7fee29d68640 1 -- 172.21.15.29:0/2413929080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241b5b30 con 0x7fee2411cba0 2024-09-21T02:02:12.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.384+0000 7fee3153e740 1 Processor -- start 2024-09-21T02:02:12.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.384+0000 7fee21ffb640 1 -- 172.21.15.29:0/2413929080 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c0054d0 con 0x7fee2411cba0 2024-09-21T02:02:12.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.384+0000 7fee21ffb640 1 -- 172.21.15.29:0/2413929080 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004b90 con 0x7fee2411cba0 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.384+0000 7fee3153e740 1 -- start start 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cd44645ea0 con 0x55cd442ff1b0 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cd446352d0 con 0x55cd44328070 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee3153e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd446390e0 con 0x55cd44307390 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41466/0 (socket says 172.21.15.29:41466) 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee23fff640 1 -- 172.21.15.29:0/990016492 learned_addr learned my addr 172.21.15.29:0/990016492 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.385+0000 7fee23fff640 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 -- 172.21.15.29:0/990016492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 msgr2=0x55cd446d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 -- 172.21.15.29:0/990016492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 msgr2=0x55cd446cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 -- 172.21.15.29:0/990016492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd44393650 con 0x55cd44328070 2024-09-21T02:02:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.386+0000 7fee23fff640 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee0c014930 tx=0x7fee0c00c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874a19c9869fe710 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.387+0000 7fee29d68640 1 -- 172.21.15.29:0/990016492 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c004030 con 0x55cd44328070 2024-09-21T02:02:12.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.387+0000 7fee29d68640 1 -- 172.21.15.29:0/990016492 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0c00fc10 con 0x55cd44328070 2024-09-21T02:02:12.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.387+0000 7fee29d68640 1 -- 172.21.15.29:0/990016492 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0035f0 con 0x55cd44328070 2024-09-21T02:02:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.389+0000 7fee21ffb640 1 -- 172.21.15.29:0/2413929080 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fee1c004d30 con 0x7fee2411cba0 2024-09-21T02:02:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.390+0000 7fee3153e740 1 -- 172.21.15.29:0/990016492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 msgr2=0x55cd446d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.390+0000 7fee3153e740 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee0c014930 tx=0x7fee0c00c430 comp rx=0 tx=0).stop 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.390+0000 7fee21ffb640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 0x7fee100b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee21ffb640 1 -- 172.21.15.29:0/2413929080 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee1c0e6d60 con 0x7fee2411cba0 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 -- 172.21.15.29:0/990016492 shutdown_connections 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 --2- 172.21.15.29:0/990016492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44328070 0x55cd446d2540 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 -- 172.21.15.29:0/990016492 >> 172.21.15.29:0/990016492 conn(0x55cd4431c460 msgr2=0x55cd446cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.391+0000 7fee3153e740 1 -- 172.21.15.29:0/990016492 shutdown_connections 2024-09-21T02:02:12.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.392+0000 7fee3153e740 1 -- 172.21.15.29:0/990016492 wait complete. 2024-09-21T02:02:12.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.392+0000 7fee3153e740 1 -- start start 2024-09-21T02:02:12.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.392+0000 7fee3153e740 1 Processor -- start 2024-09-21T02:02:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.394+0000 7fee23fff640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 0x7fee100b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cd44496700 con 0x55cd44328070 2024-09-21T02:02:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cd445d0160 con 0x55cd442ff1b0 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd446eefa0 con 0x55cd44307390 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee3153e740 10 client.0 _ll_register_callbacks cb 0x55cd445e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee23fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee23fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48210/0 (socket says 172.21.15.29:48210) 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee23fff640 1 -- 172.21.15.29:0/3522778979 learned_addr learned my addr 172.21.15.29:0/3522778979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.395+0000 7fee28d66640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 -- 172.21.15.29:0/3522778979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 msgr2=0x55cd446db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446db0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 -- 172.21.15.29:0/3522778979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 msgr2=0x55cd446dbb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446dbb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 -- 172.21.15.29:0/3522778979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd445fdc90 con 0x55cd44307390 2024-09-21T02:02:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee23fff640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00ce40 tx=0x7fee0c00ffc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704a2e806b3eee12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee03fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.396+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c01f440 con 0x55cd44307390 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.397+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee0c00fc10 con 0x55cd44307390 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.397+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee0c01f5e0 con 0x55cd44307390 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.397+0000 7fee23fff640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 0x7fee100b8b20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55cd446dc130 tx=0x7fee0c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.398+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fee0c01f960 con 0x55cd44307390 2024-09-21T02:02:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.399+0000 7fee3153e740 10 client.28153 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.399+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cd446fac10 con 0x55cd44307390 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.403+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0c00fc10 con 0x55cd44307390 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.403+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fee0c03c650 con 0x55cd44307390 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 10 client.28153 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 10 client.28153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 20 client.28153 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442e69f0 0x55cd446fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 4 client.28153 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.404+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd44647df0 con 0x55cd442e69f0 2024-09-21T02:02:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.405+0000 7fee28d66640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442e69f0 0x55cd446fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.405+0000 7fee28d66640 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442e69f0 0x55cd446fe480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=849914a7a9d40279 server_cookie=3a9d60d07a096d31 in_seq=0 out_seq=0 2024-09-21T02:02:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.405+0000 7fee03fff640 10 client.28153 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:02:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.406+0000 7fee03fff640 1 -- 172.21.15.29:0/3522778979 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd44647df0 con 0x55cd442e69f0 2024-09-21T02:02:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.406+0000 7fee03fff640 10 client.28153 handle_command_reply: tid=0 2024-09-21T02:02:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.406+0000 7fee3153e740 1 client.28153 shutdown 2024-09-21T02:02:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 msgr2=0x55cd446db600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00ce40 tx=0x7fee0c00ffc0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 shutdown_connections 2024-09-21T02:02:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442e69f0 0x55cd446fe480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446dbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446db600 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 --2- 172.21.15.29:0/3522778979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446db0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 >> 172.21.15.29:0/3522778979 conn(0x55cd4431c460 msgr2=0x55cd446d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.407+0000 7fee03fff640 0 client.28153 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.408+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 shutdown_connections 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.408+0000 7fee3153e740 1 -- 172.21.15.29:0/3522778979 wait complete. 2024-09-21T02:02:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.408+0000 7fee3153e740 20 client.28153 trim_cache size 0 max 16384 2024-09-21T02:02:12.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 Processor -- start 2024-09-21T02:02:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 -- start start 2024-09-21T02:02:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44307390 0x55cd446d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 0x55cd446d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee2411a6c0 con 0x55cd442ff1b0 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee241b7440 con 0x55cd44307390 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee3153e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee241b5fd0 con 0x55cd44328070 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee28d66640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee28d66640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59014/0 (socket says 172.21.15.29:59014) 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.435+0000 7fee28d66640 1 -- 172.21.15.29:0/3273673779 learned_addr learned my addr 172.21.15.29:0/3273673779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee23fff640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44307390 0x55cd446d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee29567640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 0x55cd446d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 -- 172.21.15.29:0/3273673779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 msgr2=0x55cd446d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 0x55cd446d82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 -- 172.21.15.29:0/3273673779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44307390 msgr2=0x55cd446d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44307390 0x55cd446d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 -- 172.21.15.29:0/3273673779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd446dc160 con 0x55cd442ff1b0 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee29567640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 0x55cd446d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.436+0000 7fee28d66640 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fee24110950 tx=0x7fee1402f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03c59e873770abc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.437+0000 7fee1a7fc640 1 -- 172.21.15.29:0/3273673779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14031660 con 0x55cd442ff1b0 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.437+0000 7fee1a7fc640 1 -- 172.21.15.29:0/3273673779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee14036040 con 0x55cd442ff1b0 2024-09-21T02:02:12.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.437+0000 7fee1a7fc640 1 -- 172.21.15.29:0/3273673779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140042a0 con 0x55cd442ff1b0 2024-09-21T02:02:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.438+0000 7fee3153e740 1 -- 172.21.15.29:0/3273673779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 msgr2=0x55cd446dd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.438+0000 7fee3153e740 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fee24110950 tx=0x7fee1402f320 comp rx=0 tx=0).stop 2024-09-21T02:02:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 -- 172.21.15.29:0/3273673779 shutdown_connections 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44328070 0x55cd446d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd44307390 0x55cd446d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 --2- 172.21.15.29:0/3273673779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd442ff1b0 0x55cd446dd0b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 -- 172.21.15.29:0/3273673779 >> 172.21.15.29:0/3273673779 conn(0x55cd4431c460 msgr2=0x55cd446f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 -- 172.21.15.29:0/3273673779 shutdown_connections 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 -- 172.21.15.29:0/3273673779 wait complete. 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 -- start start 2024-09-21T02:02:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.439+0000 7fee3153e740 1 Processor -- start 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee241b6d70 con 0x55cd44328070 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee241b66a0 con 0x55cd442ff1b0 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410e000 con 0x55cd44307390 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41500/0 (socket says 172.21.15.29:41500) 2024-09-21T02:02:12.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 -- 172.21.15.29:0/3120383610 learned_addr learned my addr 172.21.15.29:0/3120383610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee23fff640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee3153e740 10 client.0 _ll_register_callbacks cb 0x7fee0c00ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee29567640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 -- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 msgr2=0x55cd446efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 -- 172.21.15.29:0/3120383610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 msgr2=0x55cd446f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee28d66640 1 -- 172.21.15.29:0/3120383610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd446f0650 con 0x55cd442ff1b0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.440+0000 7fee29567640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee28d66640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee14005e30 tx=0x7fee14003a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be112fa7bc5d8eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee03fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14031a30 con 0x55cd442ff1b0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee14036040 con 0x55cd442ff1b0 2024-09-21T02:02:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee14031bd0 con 0x55cd442ff1b0 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fee14031dd0 con 0x55cd442ff1b0 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee3153e740 10 client.28196 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.441+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cd44768a20 con 0x55cd442ff1b0 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee14036040 con 0x55cd442ff1b0 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fee14030c60 con 0x55cd442ff1b0 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 10 client.28196 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 10 client.28196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 20 client.28196 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442bbda0 0x55cd4476b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 4 client.28196 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd446438a0 con 0x55cd442bbda0 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.442+0000 7fee23fff640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442bbda0 0x55cd4476b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.443+0000 7fee03fff640 10 client.28196 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:02:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.444+0000 7fee23fff640 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442bbda0 0x55cd4476b110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34fb9e3292b04bbe server_cookie=9e8078550eb8f758 in_seq=0 out_seq=0 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee03fff640 1 -- 172.21.15.29:0/3120383610 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd446438a0 con 0x55cd442bbda0 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee03fff640 10 client.28196 handle_command_reply: tid=0 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 client.28196 shutdown 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 msgr2=0x55cd446f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee14005e30 tx=0x7fee14003a20 comp rx=0 tx=0).stop 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 shutdown_connections 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x55cd442bbda0 0x55cd4476b110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd44328070 0x55cd446f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee03fff640 0 client.28196 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd44307390 0x55cd446efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 --2- 172.21.15.29:0/3120383610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cd442ff1b0 0x55cd446f4720 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 >> 172.21.15.29:0/3120383610 conn(0x55cd4431c460 msgr2=0x55cd446f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 shutdown_connections 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 1 -- 172.21.15.29:0/3120383610 wait complete. 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.445+0000 7fee3153e740 20 client.28196 trim_cache size 0 max 16384 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 msgr2=0x7fee100b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 0x7fee100b8b20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55cd446dc130 tx=0x7fee0c021000 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 msgr2=0x7fee241ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00d750 tx=0x7fee1c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 shutdown_connections 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee100b6610 0x7fee100b8b20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2411cba0 0x7fee241ae9b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee24111cd0 0x7fee241a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 --2- 172.21.15.29:0/2413929080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2410dbf0 0x7fee241a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.446+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 >> 172.21.15.29:0/2413929080 conn(0x7fee2406e9f0 msgr2=0x7fee2410fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.447+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 shutdown_connections 2024-09-21T02:02:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:12.447+0000 7fee1b7fe640 1 -- 172.21.15.29:0/2413929080 wait complete. 2024-09-21T02:02:12.969 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 815.0 s, 1819.6 tps, lat 54.609 ms stddev 69.017 2024-09-21T02:02:13.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:02:13.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:13 smithi190 ceph-mon[34368]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 740 op/s 2024-09-21T02:02:14.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:13 smithi066 ceph-mon[35001]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 740 op/s 2024-09-21T02:02:14.089 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:14.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:13 smithi029 ceph-mon[31228]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 740 op/s 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 -- 172.21.15.29:0/3887544231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410ad10 msgr2=0x7f0624116fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 --2- 172.21.15.29:0/3887544231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410ad10 0x7f0624116fc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f061c009be0 tx=0x7f061c02a6f0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 -- 172.21.15.29:0/3887544231 shutdown_connections 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 --2- 172.21.15.29:0/3887544231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410ad10 0x7f0624116fc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 --2- 172.21.15.29:0/3887544231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f062410a3c0 0x7f062410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 --2- 172.21.15.29:0/3887544231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410f150 0x7f062410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 -- 172.21.15.29:0/3887544231 >> 172.21.15.29:0/3887544231 conn(0x7f06240712f0 msgr2=0x7f0624073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 -- 172.21.15.29:0/3887544231 shutdown_connections 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.691+0000 7f0628d86640 1 -- 172.21.15.29:0/3887544231 wait complete. 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 Processor -- start 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 -- start start 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 0x7f0624114dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f062410ad10 0x7f06241110b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 0x7f06241117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f062411a0a0 con 0x7f062410a3c0 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0624119f20 con 0x7f062410f150 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.693+0000 7f0628d86640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f062411a220 con 0x7f062410ad10 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 0x7f0624114dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 0x7f06241117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 0x7f0624114dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59040/0 (socket says 172.21.15.29:59040) 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f062259c640 1 -- 172.21.15.29:0/1457208846 learned_addr learned my addr 172.21.15.29:0/1457208846 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 -- 172.21.15.29:0/1457208846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f062410ad10 msgr2=0x7f06241110b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f062410ad10 0x7f06241110b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 -- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 msgr2=0x7f0624114dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 0x7f0624114dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.694+0000 7f0622d9d640 1 -- 172.21.15.29:0/1457208846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06241a4b00 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f0622d9d640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 0x7f06241117d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f061c02ace0 tx=0x7f061c030f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f06137fe640 1 -- 172.21.15.29:0/1457208846 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c041070 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f06137fe640 1 -- 172.21.15.29:0/1457208846 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c003e20 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f06137fe640 1 -- 172.21.15.29:0/1457208846 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c036070 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06241a4d90 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.695+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06241a51f0 con 0x7f062410f150 2024-09-21T02:02:14.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.699+0000 7f06137fe640 1 -- 172.21.15.29:0/1457208846 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f061c044070 con 0x7f062410f150 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.699+0000 7f062f2fa740 1 Processor -- start 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.700+0000 7f062f2fa740 1 -- start start 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 0x56157fb22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56157fa97ea0 con 0x56157f7511b0 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56157fa872d0 con 0x56157f759390 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062f2fa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56157fa8b0e0 con 0x56157f77a070 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59054/0 (socket says 172.21.15.29:59054) 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 -- 172.21.15.29:0/1390831375 learned_addr learned my addr 172.21.15.29:0/1390831375 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f0621d9b640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f0621d9b640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 0x56157fb22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f06137fe640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 0x7f060c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 -- 172.21.15.29:0/1390831375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 msgr2=0x56157fb216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 -- 172.21.15.29:0/1390831375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 msgr2=0x56157fb22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 0x56157fb22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f062259c640 1 -- 172.21.15.29:0/1390831375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157f7e5650 con 0x56157f7511b0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.701+0000 7f0621d9b640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 0x56157fb22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f0622d9d640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 0x7f060c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f06137fe640 1 -- 172.21.15.29:0/1457208846 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f061c107860 con 0x7f062410f150 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f062259c640 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f061800a8e0 tx=0x7f061800ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5fef04210ce022 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f0610ff9640 1 -- 172.21.15.29:0/1390831375 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061801a020 con 0x56157f7511b0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f0622d9d640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 0x7f060c0b8a90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c0c2340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f0610ff9640 1 -- 172.21.15.29:0/1390831375 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0618009e50 con 0x56157f7511b0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.702+0000 7f0610ff9640 1 -- 172.21.15.29:0/1390831375 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618015760 con 0x56157f7511b0 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.703+0000 7f062f2fa740 1 -- 172.21.15.29:0/1390831375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 msgr2=0x56157fb21be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.703+0000 7f062f2fa740 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f061800a8e0 tx=0x7f061800ac00 comp rx=0 tx=0).stop 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 -- 172.21.15.29:0/1390831375 shutdown_connections 2024-09-21T02:02:14.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f759390 0x56157fb22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb21be0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 --2- 172.21.15.29:0/1390831375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 -- 172.21.15.29:0/1390831375 >> 172.21.15.29:0/1390831375 conn(0x56157f76e460 msgr2=0x56157fb1f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 -- 172.21.15.29:0/1390831375 shutdown_connections 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 -- 172.21.15.29:0/1390831375 wait complete. 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.704+0000 7f062f2fa740 1 -- start start 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 Processor -- start 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59058/0 (socket says 172.21.15.29:59058) 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062259c640 1 -- 172.21.15.29:0/3512312800 learned_addr learned my addr 172.21.15.29:0/3512312800 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.795 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:14.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56157f8e8700 con 0x56157f759390 2024-09-21T02:02:14.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56157fa22160 con 0x56157f7511b0 2024-09-21T02:02:14.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56157fb2fe80 con 0x56157f77a070 2024-09-21T02:02:14.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.705+0000 7f062f2fa740 10 client.0 _ll_register_callbacks cb 0x56157fa3b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 -- 172.21.15.29:0/3512312800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 msgr2=0x56157fb2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 -- 172.21.15.29:0/3512312800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 msgr2=0x56157fb2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157fa4fc90 con 0x56157f7511b0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f062259c640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb2cce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06140058c0 tx=0x7f0614010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b052e23764f27e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061400f7b0 con 0x56157f7511b0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f061400f950 con 0x56157f7511b0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.706+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0618009e50 con 0x56157f7511b0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.707+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f061400fb50 con 0x56157f7511b0 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.707+0000 7f062f2fa740 10 client.28208 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:14.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.707+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56157fb3bb40 con 0x56157f7511b0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.707+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0618009e50 con 0x56157f7511b0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.707+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f061401e430 con 0x56157f7511b0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 10 client.28208 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 10 client.28208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 20 client.28208 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f7389f0 0x56157fb3f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 4 client.28208 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56157fa99df0 con 0x56157f7389f0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.708+0000 7f062259c640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f7389f0 0x56157fb3f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.709+0000 7f062259c640 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f7389f0 0x56157fb3f010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13c40a9e881a833e server_cookie=701b8156fb24a363 in_seq=0 out_seq=0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.709+0000 7f0609ffb640 10 client.28208 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.709+0000 7f0609ffb640 1 -- 172.21.15.29:0/3512312800 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56157fa99df0 con 0x56157f7389f0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.709+0000 7f0609ffb640 10 client.28208 handle_command_reply: tid=0 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.709+0000 7f062f2fa740 1 client.28208 shutdown 2024-09-21T02:02:14.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 msgr2=0x56157fb2cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb2cce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06140058c0 tx=0x7f0614010b30 comp rx=0 tx=0).stop 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 shutdown_connections 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f7389f0 0x56157fb3f010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f77a070 0x56157fb2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f759390 0x56157fb2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f0609ffb640 0 client.28208 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3512312800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb2cce0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 >> 172.21.15.29:0/3512312800 conn(0x56157f76e460 msgr2=0x56157fb24830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 shutdown_connections 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 1 -- 172.21.15.29:0/3512312800 wait complete. 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.710+0000 7f062f2fa740 20 client.28208 trim_cache size 0 max 16384 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.730+0000 7f062f2fa740 1 Processor -- start 2024-09-21T02:02:14.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.730+0000 7f062f2fa740 1 -- start start 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb25160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f77a070 0x56157fb29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06241c69d0 con 0x56157f7511b0 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06241c8510 con 0x56157f77a070 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062f2fa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f06241c70a0 con 0x56157f759390 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48270/0 (socket says 172.21.15.29:48270) 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 -- 172.21.15.29:0/3970475489 learned_addr learned my addr 172.21.15.29:0/3970475489 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f77a070 0x56157fb29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 -- 172.21.15.29:0/3970475489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f77a070 msgr2=0x56157fb29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f77a070 0x56157fb29ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 -- 172.21.15.29:0/3970475489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 msgr2=0x56157fb25160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb25160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.731+0000 7f062259c640 1 -- 172.21.15.29:0/3970475489 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157fb1ff00 con 0x56157f759390 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.732+0000 7f0621d9b640 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb25160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.732+0000 7f062259c640 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f061800c010 tx=0x7f0618016f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88836258b17a1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.732+0000 7f060bfff640 1 -- 172.21.15.29:0/3970475489 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061801a040 con 0x56157f759390 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.732+0000 7f060bfff640 1 -- 172.21.15.29:0/3970475489 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061800a490 con 0x56157f759390 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.732+0000 7f060bfff640 1 -- 172.21.15.29:0/3970475489 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618003b80 con 0x56157f759390 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.733+0000 7f062f2fa740 1 -- 172.21.15.29:0/3970475489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 msgr2=0x56157fb22de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.733+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f061800c010 tx=0x7f0618016f30 comp rx=0 tx=0).stop 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.733+0000 7f062f2fa740 1 -- 172.21.15.29:0/3970475489 shutdown_connections 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.733+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f77a070 0x56157fb29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb22de0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 --2- 172.21.15.29:0/3970475489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f7511b0 0x56157fb25160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 -- 172.21.15.29:0/3970475489 >> 172.21.15.29:0/3970475489 conn(0x56157f76e460 msgr2=0x56157fb226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 -- 172.21.15.29:0/3970475489 shutdown_connections 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 -- 172.21.15.29:0/3970475489 wait complete. 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 -- start start 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.734+0000 7f062f2fa740 1 Processor -- start 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f77a070 0x56157fb331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06241c7e40 con 0x56157f77a070 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06241c7770 con 0x56157f7511b0 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f062410a7d0 con 0x56157f759390 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062f2fa740 10 client.0 _ll_register_callbacks cb 0x7f0614009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f77a070 0x56157fb331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48284/0 (socket says 172.21.15.29:48284) 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 -- 172.21.15.29:0/141869728 learned_addr learned my addr 172.21.15.29:0/141869728 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f0621d9b640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 -- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 msgr2=0x56157fb357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 -- 172.21.15.29:0/141869728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f77a070 msgr2=0x56157fb331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f77a070 0x56157fb331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.735+0000 7f062259c640 1 -- 172.21.15.29:0/141869728 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157fb337f0 con 0x56157f759390 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f062259c640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06180055b0 tx=0x7f06180055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fea550b2343e767 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f0609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618040060 con 0x56157f759390 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0618003a10 con 0x56157f759390 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061800a490 con 0x56157f759390 2024-09-21T02:02:14.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f061801a040 con 0x56157f759390 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f062f2fa740 10 client.28171 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56157fba9780 con 0x56157f759390 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.736+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061800a490 con 0x56157f759390 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0618016930 con 0x56157f759390 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 10 client.28171 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 10 client.28171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 20 client.28171 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f70dda0 0x56157fbabcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 4 client.28171 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56157fa958a0 con 0x56157f70dda0 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.737+0000 7f0621d9b640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f70dda0 0x56157fbabcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.738+0000 7f0609ffb640 10 client.28171 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.738+0000 7f0621d9b640 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f70dda0 0x56157fbabcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c74d1f6793606757 server_cookie=1de8c69642e8cb56 in_seq=0 out_seq=0 2024-09-21T02:02:14.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f0609ffb640 1 -- 172.21.15.29:0/141869728 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56157fa958a0 con 0x56157f70dda0 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f0609ffb640 10 client.28171 handle_command_reply: tid=0 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 client.28171 shutdown 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 msgr2=0x56157fb32c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06180055b0 tx=0x7f06180055e0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 shutdown_connections 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56157f70dda0 0x56157fbabcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56157f77a070 0x56157fb331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56157f759390 0x56157fb32c70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f0609ffb640 0 client.28171 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 --2- 172.21.15.29:0/141869728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56157f7511b0 0x56157fb357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 >> 172.21.15.29:0/141869728 conn(0x56157f76e460 msgr2=0x56157fb24f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 shutdown_connections 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.739+0000 7f062f2fa740 1 -- 172.21.15.29:0/141869728 wait complete. 2024-09-21T02:02:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f062f2fa740 20 client.28171 trim_cache size 0 max 16384 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 msgr2=0x7f060c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 0x7f060c0b8a90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c0c2340 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 msgr2=0x7f06241117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 0x7f06241117d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f061c02ace0 tx=0x7f061c030f00 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 shutdown_connections 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f060c0b6580 0x7f060c0b8a90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f062410f150 0x7f06241117d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f062410ad10 0x7f06241110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 --2- 172.21.15.29:0/1457208846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f062410a3c0 0x7f0624114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 >> 172.21.15.29:0/1457208846 conn(0x7f06240712f0 msgr2=0x7f062410dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 shutdown_connections 2024-09-21T02:02:14.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:14.740+0000 7f0628d86640 1 -- 172.21.15.29:0/1457208846 wait complete. 2024-09-21T02:02:15.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:15 smithi190 ceph-mon[34368]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 603 op/s 2024-09-21T02:02:16.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:15 smithi066 ceph-mon[35001]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 603 op/s 2024-09-21T02:02:16.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:15 smithi029 ceph-mon[31228]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 603 op/s 2024-09-21T02:02:16.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:02:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:17.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.219+0000 7ff8e49d0640 1 -- 172.21.15.29:0/3081188755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e011cba0 msgr2=0x7ff8e011efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.219+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/3081188755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e011cba0 0x7ff8e011efc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009be0 tx=0x7ff8d802a6e0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.222+0000 7ff8e49d0640 1 -- 172.21.15.29:0/3081188755 shutdown_connections 2024-09-21T02:02:17.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.222+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/3081188755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e011cba0 0x7ff8e011efc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.222+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/3081188755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.222+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/3081188755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e0111cd0 0x7ff8e010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.222+0000 7ff8e49d0640 1 -- 172.21.15.29:0/3081188755 >> 172.21.15.29:0/3081188755 conn(0x7ff8e006e9f0 msgr2=0x7ff8e006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.224+0000 7ff8e49d0640 1 -- 172.21.15.29:0/3081188755 shutdown_connections 2024-09-21T02:02:17.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.224+0000 7ff8e49d0640 1 -- 172.21.15.29:0/3081188755 wait complete. 2024-09-21T02:02:17.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.225+0000 7ff8e49d0640 1 Processor -- start 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 -- start start 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e0111cd0 0x7ff8e0118740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e011cba0 0x7ff8e0118d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114c40 con 0x7ff8e011cba0 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114ac0 con 0x7ff8e010dbf0 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.226+0000 7ff8e49d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114dc0 con 0x7ff8e0111cd0 2024-09-21T02:02:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41566/0 (socket says 172.21.15.29:41566) 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 -- 172.21.15.29:0/2025220992 learned_addr learned my addr 172.21.15.29:0/2025220992 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 -- 172.21.15.29:0/2025220992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e0111cd0 msgr2=0x7ff8e0118740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e0111cd0 0x7ff8e0118740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 -- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e011cba0 msgr2=0x7ff8e0118d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e011cba0 0x7ff8e0118d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 -- 172.21.15.29:0/2025220992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e0119520 con 0x7ff8e010dbf0 2024-09-21T02:02:17.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.227+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400b450 tx=0x7ff8d400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.233+0000 7ff8c7fff640 1 -- 172.21.15.29:0/2025220992 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4015020 con 0x7ff8e010dbf0 2024-09-21T02:02:17.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.233+0000 7ff8c7fff640 1 -- 172.21.15.29:0/2025220992 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d4011d40 con 0x7ff8e010dbf0 2024-09-21T02:02:17.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.233+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01accc0 con 0x7ff8e010dbf0 2024-09-21T02:02:17.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.233+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01ad230 con 0x7ff8e010dbf0 2024-09-21T02:02:17.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.233+0000 7ff8c7fff640 1 -- 172.21.15.29:0/2025220992 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400f340 con 0x7ff8e010dbf0 2024-09-21T02:02:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.239+0000 7ff8c7fff640 1 -- 172.21.15.29:0/2025220992 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff8d4018020 con 0x7ff8e010dbf0 2024-09-21T02:02:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.239+0000 7ff8eb745740 1 Processor -- start 2024-09-21T02:02:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.240+0000 7ff8c7fff640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 0x7ff8c80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.240+0000 7ff8c7fff640 1 -- 172.21.15.29:0/2025220992 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff8d40e6af0 con 0x7ff8e010dbf0 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 -- start start 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d1b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d1b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f81c91ea0 con 0x560f81974070 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f81c812d0 con 0x560f8194b1b0 2024-09-21T02:02:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.241+0000 7ff8eb745740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560f81c850e0 con 0x560f81953390 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.242+0000 7ff8df59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.242+0000 7ff8df59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48304/0 (socket says 172.21.15.29:48304) 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.242+0000 7ff8df59e640 1 -- 172.21.15.29:0/3979298558 learned_addr learned my addr 172.21.15.29:0/3979298558 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.242+0000 7ff8df59e640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 0x7ff8c80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.243+0000 7ff8df59e640 1 -- 172.21.15.29:0/3979298558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 msgr2=0x560f81d1b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.243+0000 7ff8df59e640 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d1b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.243+0000 7ff8df59e640 1 -- 172.21.15.29:0/3979298558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 msgr2=0x560f81d1b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.243+0000 7ff8df59e640 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d1b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.243+0000 7ff8df59e640 1 -- 172.21.15.29:0/3979298558 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f819df650 con 0x560f81953390 2024-09-21T02:02:17.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.246+0000 7ff8df59e640 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff8d802ac00 tx=0x7ff8d8030a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9519459707ef995 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.250+0000 7ff8c57fa640 1 -- 172.21.15.29:0/3979298558 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d804d070 con 0x560f81953390 2024-09-21T02:02:17.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.250+0000 7ff8c57fa640 1 -- 172.21.15.29:0/3979298558 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d8048450 con 0x560f81953390 2024-09-21T02:02:17.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.250+0000 7ff8c57fa640 1 -- 172.21.15.29:0/3979298558 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8036040 con 0x560f81953390 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 -- 172.21.15.29:0/3979298558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 msgr2=0x560f81d1bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff8d802ac00 tx=0x7ff8d8030a40 comp rx=0 tx=0).stop 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8df59e640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 0x7ff8c80b8bf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8030910 tx=0x7ff8d8042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 -- 172.21.15.29:0/3979298558 shutdown_connections 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d1bee0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d1b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3979298558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d1b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.251+0000 7ff8eb745740 1 -- 172.21.15.29:0/3979298558 >> 172.21.15.29:0/3979298558 conn(0x560f81968460 msgr2=0x560f81d19350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.252+0000 7ff8eb745740 1 -- 172.21.15.29:0/3979298558 shutdown_connections 2024-09-21T02:02:17.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.252+0000 7ff8eb745740 1 -- 172.21.15.29:0/3979298558 wait complete. 2024-09-21T02:02:17.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.252+0000 7ff8eb745740 1 -- start start 2024-09-21T02:02:17.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 Processor -- start 2024-09-21T02:02:17.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81974070 0x560f81d32f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f81ae2700 con 0x560f8194b1b0 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f81c1c160 con 0x560f81974070 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560f81d29be0 con 0x560f81953390 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.253+0000 7ff8eb745740 10 client.0 _ll_register_callbacks cb 0x560f81c359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.255+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.255+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59118/0 (socket says 172.21.15.29:59118) 2024-09-21T02:02:17.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.255+0000 7ff8ded9d640 1 -- 172.21.15.29:0/4210264642 learned_addr learned my addr 172.21.15.29:0/4210264642 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.256+0000 7ff8df59e640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81974070 0x560f81d32f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.256+0000 7ff8de59c640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.257+0000 7ff8ded9d640 1 -- 172.21.15.29:0/4210264642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 msgr2=0x560f81d27110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.257+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d27110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.257+0000 7ff8ded9d640 1 -- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81974070 msgr2=0x560f81d32f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.257+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81974070 0x560f81d32f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.257+0000 7ff8ded9d640 1 -- 172.21.15.29:0/4210264642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f81c49c90 con 0x560f8194b1b0 2024-09-21T02:02:17.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.258+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40adcf0 tx=0x7ff8d4010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d8561ad89c852b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.258+0000 7ff8c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.259+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40ae460 con 0x560f8194b1b0 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.259+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d8048450 con 0x560f8194b1b0 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.259+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff8d40b5bc0 con 0x560f8194b1b0 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.260+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8d40b2840 con 0x560f8194b1b0 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.260+0000 7ff8eb745740 10 client.18396 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:17.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.260+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560f81d359e0 con 0x560f8194b1b0 2024-09-21T02:02:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.260+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d8048450 con 0x560f8194b1b0 2024-09-21T02:02:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff8d40aca30 con 0x560f8194b1b0 2024-09-21T02:02:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 10 client.18396 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 20 client.18396 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f819329f0 0x560f81d392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 4 client.18396 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.262+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f81c93df0 con 0x560f819329f0 2024-09-21T02:02:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.264+0000 7ff8de59c640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f819329f0 0x560f81d392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.271+0000 7ff8c2ff5640 10 client.18396 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:17.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.271+0000 7ff8de59c640 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f819329f0 0x560f81d392f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6968553a5bc746c7 server_cookie=c85eee44ff618cf7 in_seq=0 out_seq=0 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.273+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/4210264642 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f81c93df0 con 0x560f819329f0 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.273+0000 7ff8c2ff5640 10 client.18396 handle_command_reply: tid=0 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.273+0000 7ff8eb745740 1 client.18396 shutdown 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.273+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 msgr2=0x560f81d26bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40adcf0 tx=0x7ff8d4010660 comp rx=0 tx=0).stop 2024-09-21T02:02:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 shutdown_connections 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f819329f0 0x560f81d392f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81974070 0x560f81d32f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 --2- 172.21.15.29:0/4210264642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d26bd0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 >> 172.21.15.29:0/4210264642 conn(0x560f81968460 msgr2=0x560f81d1e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8c2ff5640 0 client.18396 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 shutdown_connections 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 1 -- 172.21.15.29:0/4210264642 wait complete. 2024-09-21T02:02:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.274+0000 7ff8eb745740 20 client.18396 trim_cache size 0 max 16384 2024-09-21T02:02:17.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.299+0000 7ff8eb745740 1 Processor -- start 2024-09-21T02:02:17.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 -- start start 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81953390 0x560f81d1bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81974070 0x560f81d25450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00e2c0 con 0x560f8194b1b0 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00fe00 con 0x560f81953390 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8eb745740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00e990 con 0x560f81974070 2024-09-21T02:02:17.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59126/0 (socket says 172.21.15.29:59126) 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.300+0000 7ff8ded9d640 1 -- 172.21.15.29:0/3841231925 learned_addr learned my addr 172.21.15.29:0/3841231925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8de59c640 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81953390 0x560f81d1bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 -- 172.21.15.29:0/3841231925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81974070 msgr2=0x560f81d25450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81974070 0x560f81d25450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 -- 172.21.15.29:0/3841231925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81953390 msgr2=0x560f81d1bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81953390 0x560f81d1bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 -- 172.21.15.29:0/3841231925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f81d27c70 con 0x560f8194b1b0 2024-09-21T02:02:17.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.301+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40aa2e0 tx=0x7ff8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ffeefc4b48bc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.303+0000 7ff8c4ff9640 1 -- 172.21.15.29:0/3841231925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40a9c00 con 0x560f8194b1b0 2024-09-21T02:02:17.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.303+0000 7ff8c4ff9640 1 -- 172.21.15.29:0/3841231925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d40ae460 con 0x560f8194b1b0 2024-09-21T02:02:17.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.303+0000 7ff8c4ff9640 1 -- 172.21.15.29:0/3841231925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40adb30 con 0x560f8194b1b0 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 -- 172.21.15.29:0/3841231925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 msgr2=0x560f81d1b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40aa2e0 tx=0x7ff8d40046c0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 -- 172.21.15.29:0/3841231925 shutdown_connections 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81974070 0x560f81d25450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f81953390 0x560f81d1bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 --2- 172.21.15.29:0/3841231925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f8194b1b0 0x560f81d1b990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 -- 172.21.15.29:0/3841231925 >> 172.21.15.29:0/3841231925 conn(0x560f81968460 msgr2=0x560f81d26b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 -- 172.21.15.29:0/3841231925 shutdown_connections 2024-09-21T02:02:17.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.304+0000 7ff8eb745740 1 -- 172.21.15.29:0/3841231925 wait complete. 2024-09-21T02:02:17.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 -- start start 2024-09-21T02:02:17.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 Processor -- start 2024-09-21T02:02:17.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d2f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d3e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00f730 con 0x560f81974070 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00f060 con 0x560f8194b1b0 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00c350 con 0x560f81953390 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.305+0000 7ff8eb745740 10 client.0 _ll_register_callbacks cb 0x7ff8d40aa450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48330/0 (socket says 172.21.15.29:48330) 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 -- 172.21.15.29:0/2251973219 learned_addr learned my addr 172.21.15.29:0/2251973219 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:17.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 -- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 msgr2=0x560f81d2f370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d2f370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 -- 172.21.15.29:0/2251973219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 msgr2=0x560f81d3e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d3e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 -- 172.21.15.29:0/2251973219 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f81d3eeb0 con 0x560f81953390 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8de59c640 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0004700 tx=0x7ff8d0021810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edeab3eedfef1644 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:17.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.306+0000 7ff8c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:17.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0004260 con 0x560f81953390 2024-09-21T02:02:17.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7ff8d0004400 con 0x560f81953390 2024-09-21T02:02:17.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d40ae460 con 0x560f81953390 2024-09-21T02:02:17.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8d00035f0 con 0x560f81953390 2024-09-21T02:02:17.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8eb745740 10 client.28183 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:17.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560f81da3980 con 0x560f81953390 2024-09-21T02:02:17.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.309+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d40ae460 con 0x560f81953390 2024-09-21T02:02:17.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.310+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff8d00338b0 con 0x560f81953390 2024-09-21T02:02:17.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.310+0000 7ff8eb745740 10 client.28183 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:17.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.310+0000 7ff8eb745740 10 client.28183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:17.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.310+0000 7ff8eb745740 20 client.28183 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:17.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.310+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f81907da0 0x560f81da6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:17.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8eb745740 4 client.28183 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:17.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f81c8f8a0 con 0x560f81907da0 2024-09-21T02:02:17.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f81907da0 0x560f81da6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:17.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8ded9d640 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f81907da0 0x560f81da6120 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ee00e649449f91 server_cookie=4ede6dbc1e7443d7 in_seq=0 out_seq=0 2024-09-21T02:02:17.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8c2ff5640 10 client.28183 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8c2ff5640 1 -- 172.21.15.29:0/2251973219 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f81c8f8a0 con 0x560f81907da0 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8c2ff5640 10 client.28183 handle_command_reply: tid=0 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.311+0000 7ff8eb745740 1 client.28183 shutdown 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 msgr2=0x560f81d3e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0004700 tx=0x7ff8d0021810 comp rx=0 tx=0).stop 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 shutdown_connections 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560f81907da0 0x560f81da6120 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f81974070 0x560f81d3e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f81953390 0x560f81d3e430 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 --2- 172.21.15.29:0/2251973219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f8194b1b0 0x560f81d2f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8c2ff5640 0 client.28183 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:17.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 >> 172.21.15.29:0/2251973219 conn(0x560f81968460 msgr2=0x560f81d25010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 shutdown_connections 2024-09-21T02:02:17.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.312+0000 7ff8eb745740 1 -- 172.21.15.29:0/2251973219 wait complete. 2024-09-21T02:02:17.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.313+0000 7ff8eb745740 20 client.28183 trim_cache size 0 max 16384 2024-09-21T02:02:17.315 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 msgr2=0x7ff8c80b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 0x7ff8c80b8bf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8030910 tx=0x7ff8d8042000 comp rx=0 tx=0).stop 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 msgr2=0x7ff8e011c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400b450 tx=0x7ff8d400b910 comp rx=0 tx=0).stop 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 shutdown_connections 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7ff8c80b66e0 0x7ff8c80b8bf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e011cba0 0x7ff8e0118d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8e0111cd0 0x7ff8e0118740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:17.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 --2- 172.21.15.29:0/2025220992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e010dbf0 0x7ff8e011c460 secure :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400b450 tx=0x7ff8d400b910 comp rx=0 tx=0).stop 2024-09-21T02:02:17.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.314+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 >> 172.21.15.29:0/2025220992 conn(0x7ff8e006e9f0 msgr2=0x7ff8e0110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:17.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.315+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 shutdown_connections 2024-09-21T02:02:17.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:17.315+0000 7ff8e49d0640 1 -- 172.21.15.29:0/2025220992 wait complete. 2024-09-21T02:02:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 820.0 s, 1909.0 tps, lat 52.306 ms stddev 61.539 2024-09-21T02:02:17.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:17 smithi190 ceph-mon[34368]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 MiB/s wr, 690 op/s 2024-09-21T02:02:18.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:17 smithi066 ceph-mon[35001]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 MiB/s wr, 690 op/s 2024-09-21T02:02:18.086 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:17 smithi029 ceph-mon[31228]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 MiB/s wr, 690 op/s 2024-09-21T02:02:19.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:02:19.128 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:02:19.496 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:19.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:19 smithi190 ceph-mon[34368]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-21T02:02:19.995 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:19 smithi029 ceph-mon[31228]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-21T02:02:20.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:19 smithi066 ceph-mon[35001]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-21T02:02:20.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.120+0000 7f6c26b53640 1 -- 172.21.15.29:0/3513110139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c201052b0 msgr2=0x7f6c2010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.120+0000 7f6c26b53640 1 --2- 172.21.15.29:0/3513110139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c201052b0 0x7f6c2010d8f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009900 tx=0x7f6c0c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.121+0000 7f6c26b53640 1 -- 172.21.15.29:0/3513110139 shutdown_connections 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.121+0000 7f6c26b53640 1 --2- 172.21.15.29:0/3513110139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2010de30 0x7f6c20110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.121+0000 7f6c26b53640 1 --2- 172.21.15.29:0/3513110139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c201052b0 0x7f6c2010d8f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.122+0000 7f6c26b53640 1 --2- 172.21.15.29:0/3513110139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c20104880 0x7f6c20104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.122+0000 7f6c26b53640 1 -- 172.21.15.29:0/3513110139 >> 172.21.15.29:0/3513110139 conn(0x7f6c20079730 msgr2=0x7f6c20079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.122+0000 7f6c26b53640 1 -- 172.21.15.29:0/3513110139 shutdown_connections 2024-09-21T02:02:20.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.122+0000 7f6c26b53640 1 -- 172.21.15.29:0/3513110139 wait complete. 2024-09-21T02:02:20.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.123+0000 7f6c26b53640 1 Processor -- start 2024-09-21T02:02:20.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.123+0000 7f6c26b53640 1 -- start start 2024-09-21T02:02:20.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c20104880 0x7f6c2019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c201052b0 0x7f6c2019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20113a70 con 0x7f6c20104880 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201138f0 con 0x7f6c2010de30 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c26b53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20113bf0 con 0x7f6c201052b0 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41606/0 (socket says 172.21.15.29:41606) 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 -- 172.21.15.29:0/1483877347 learned_addr learned my addr 172.21.15.29:0/1483877347 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c248ef640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c20104880 0x7f6c2019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c201052b0 0x7f6c2019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 -- 172.21.15.29:0/1483877347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c201052b0 msgr2=0x7f6c2019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c201052b0 0x7f6c2019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 -- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c20104880 msgr2=0x7f6c2019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c20104880 0x7f6c2019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.124+0000 7f6c250f0640 1 -- 172.21.15.29:0/1483877347 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c201a03f0 con 0x7f6c2010de30 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c250f0640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800eb10 tx=0x7f6c1800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c1dffb640 1 -- 172.21.15.29:0/1483877347 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004b20 con 0x7f6c2010de30 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c1dffb640 1 -- 172.21.15.29:0/1483877347 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c18004cc0 con 0x7f6c2010de30 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c1dffb640 1 -- 172.21.15.29:0/1483877347 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18010670 con 0x7f6c2010de30 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c201a06e0 con 0x7f6c2010de30 2024-09-21T02:02:20.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.125+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c20102670 con 0x7f6c2010de30 2024-09-21T02:02:20.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.128+0000 7f6c1dffb640 1 -- 172.21.15.29:0/1483877347 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6c18005020 con 0x7f6c2010de30 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.128+0000 7f6c2d0c7740 1 Processor -- start 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.128+0000 7f6c2d0c7740 1 -- start start 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb948c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb949180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5651fb8b6e60 con 0x5651fb570490 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5651fb8a6290 con 0x5651fb578670 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5651fb8aa0a0 con 0x5651fb599350 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41616/0 (socket says 172.21.15.29:41616) 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c1ffff640 1 -- 172.21.15.29:0/2070386467 learned_addr learned my addr 172.21.15.29:0/2070386467 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.129+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb948c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 -- 172.21.15.29:0/2070386467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 msgr2=0x5651fb948c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb948c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 -- 172.21.15.29:0/2070386467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 msgr2=0x5651fb949180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb949180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 -- 172.21.15.29:0/2070386467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fb604930 con 0x5651fb578670 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1dffb640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 0x7f6c100b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005940 tx=0x7f6c0c0058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07ade60c35361ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c1dffb640 1 -- 172.21.15.29:0/1483877347 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6c180e7990 con 0x7f6c2010de30 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.130+0000 7f6c0affd640 1 -- 172.21.15.29:0/2070386467 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c035040 con 0x5651fb578670 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.131+0000 7f6c0affd640 1 -- 172.21.15.29:0/2070386467 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c0316c0 con 0x5651fb578670 2024-09-21T02:02:20.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.131+0000 7f6c0affd640 1 -- 172.21.15.29:0/2070386467 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c035040 con 0x5651fb578670 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.132+0000 7f6c248ef640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 0x7f6c100b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/2070386467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 msgr2=0x5651fb9496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005940 tx=0x7f6c0c0058b0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/2070386467 shutdown_connections 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb9496c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb949180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/2070386467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb948c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/2070386467 >> 172.21.15.29:0/2070386467 conn(0x5651fb58d740 msgr2=0x5651fb946b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c248ef640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 0x7f6c100b8aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5651fb9478c0 tx=0x7f6c140097d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.133+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/2070386467 shutdown_connections 2024-09-21T02:02:20.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.134+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/2070386467 wait complete. 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.134+0000 7f6c2d0c7740 1 -- start start 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.134+0000 7f6c2d0c7740 1 Processor -- start 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.135+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb954800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.135+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb954d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.135+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5651fb707890 con 0x5651fb570490 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48364/0 (socket says 172.21.15.29:48364) 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c250f0640 1 -- 172.21.15.29:0/427253821 learned_addr learned my addr 172.21.15.29:0/427253821 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5651fb840d40 con 0x5651fb578670 2024-09-21T02:02:20.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5651fb9573c0 con 0x5651fb599350 2024-09-21T02:02:20.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.136+0000 7f6c248ef640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb954800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c2d0c7740 10 client.0 _ll_register_callbacks cb 0x5651fb8a4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 -- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 msgr2=0x5651fb954d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb954d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 -- 172.21.15.29:0/427253821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 msgr2=0x5651fb954800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb954800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fb960c40 con 0x5651fb599350 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c248ef640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb954800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.137+0000 7f6c250f0640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800cf20 tx=0x7f6c180ad420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551199ea98dadb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.138+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0c0316c0 con 0x5651fb599350 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.144+0000 7f6bfffff640 10 client.28195 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.144+0000 7f6bfffff640 1 -- 172.21.15.29:0/427253821 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180b2020 con 0x5651fb599350 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.144+0000 7f6bfffff640 1 -- 172.21.15.29:0/427253821 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6c180a9050 con 0x5651fb599350 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.144+0000 7f6bfffff640 1 -- 172.21.15.29:0/427253821 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6c180ae1d0 con 0x5651fb599350 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.145+0000 7f6c2d0c7740 10 client.28195 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:20.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.145+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=49}) -- 0x5651fb9634d0 con 0x5651fb599350 2024-09-21T02:02:20.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.146+0000 7f6bfffff640 1 -- 172.21.15.29:0/427253821 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6c180b21c0 con 0x5651fb599350 2024-09-21T02:02:20.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 10 client.28195 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:20.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 10 client.28195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:20.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 20 client.28195 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:20.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb557cd0 0x5651fb966e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 4 client.28195 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:20.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.147+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6c0c0316c0 con 0x5651fb557cd0 2024-09-21T02:02:20.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.150+0000 7f6c248ef640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb557cd0 0x5651fb966e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.164+0000 7f6c248ef640 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb557cd0 0x5651fb966e90 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4d312af87a9dbb server_cookie=3ab0acdcdf87ce62 in_seq=0 out_seq=0 2024-09-21T02:02:20.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.165+0000 7f6bfffff640 10 client.28195 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.169+0000 7f6bfffff640 1 -- 172.21.15.29:0/427253821 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6c0c0316c0 con 0x5651fb557cd0 2024-09-21T02:02:20.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.169+0000 7f6bfffff640 10 client.28195 handle_command_reply: tid=0 2024-09-21T02:02:20.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.169+0000 7f6c2d0c7740 1 client.28195 shutdown 2024-09-21T02:02:20.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.169+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 msgr2=0x5651fb9606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.169+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800cf20 tx=0x7f6c180ad420 comp rx=0 tx=0).stop 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 shutdown_connections 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb557cd0 0x5651fb966e90 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9606b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb578670 0x5651fb954d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/427253821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb954800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.175+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 >> 172.21.15.29:0/427253821 conn(0x5651fb58d740 msgr2=0x5651fb94be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.177+0000 7f6bfffff640 0 client.28195 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.183+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 shutdown_connections 2024-09-21T02:02:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.184+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/427253821 wait complete. 2024-09-21T02:02:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.184+0000 7f6c2d0c7740 20 client.28195 trim_cache size 0 max 16384 2024-09-21T02:02:20.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.223+0000 7f6c2d0c7740 1 Processor -- start 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.224+0000 7f6c2d0c7740 1 -- start start 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.225+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb570490 0x5651fb946360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.225+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 0x5651fb9468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.225+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ad200 con 0x5651fb578670 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c250f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48372/0 (socket says 172.21.15.29:48372) 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c250f0640 1 -- 172.21.15.29:0/1164219874 learned_addr learned my addr 172.21.15.29:0/1164219874 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 0x5651fb9468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c248ef640 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb570490 0x5651fb946360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201aed40 con 0x5651fb570490 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.226+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ad8d0 con 0x5651fb599350 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 -- 172.21.15.29:0/1164219874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 msgr2=0x5651fb9517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 -- 172.21.15.29:0/1164219874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb570490 msgr2=0x5651fb946360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb570490 0x5651fb946360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 -- 172.21.15.29:0/1164219874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fb953560 con 0x5651fb578670 2024-09-21T02:02:20.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.227+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 0x5651fb9468a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02a930 tx=0x7f6c0c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c79f6a72608a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.229+0000 7f6c0a7fc640 1 -- 172.21.15.29:0/1164219874 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c035040 con 0x5651fb578670 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.229+0000 7f6c0a7fc640 1 -- 172.21.15.29:0/1164219874 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c0316c0 con 0x5651fb578670 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.229+0000 7f6c0a7fc640 1 -- 172.21.15.29:0/1164219874 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0044e0 con 0x5651fb578670 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 msgr2=0x5651fb9468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 0x5651fb9468a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02a930 tx=0x7f6c0c009640 comp rx=0 tx=0).stop 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 shutdown_connections 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb599350 0x5651fb9517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb578670 0x5651fb9468a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/1164219874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb570490 0x5651fb946360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 >> 172.21.15.29:0/1164219874 conn(0x5651fb58d740 msgr2=0x5651fb9547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.230+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 shutdown_connections 2024-09-21T02:02:20.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.232+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/1164219874 wait complete. 2024-09-21T02:02:20.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.232+0000 7f6c2d0c7740 1 -- start start 2024-09-21T02:02:20.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.233+0000 7f6c2d0c7740 1 Processor -- start 2024-09-21T02:02:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb95cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 0x5651fb951670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c2d0c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb599350 0x5651fb951d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ae670 con 0x5651fb570490 2024-09-21T02:02:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c2d0c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201adfa0 con 0x5651fb599350 2024-09-21T02:02:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 0x5651fb951670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 0x5651fb951670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48382/0 (socket says 172.21.15.29:48382) 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.234+0000 7f6c1ffff640 1 -- 172.21.15.29:0/3624893222 learned_addr learned my addr 172.21.15.29:0/3624893222 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c250f0640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb599350 0x5651fb951d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ab1e0 con 0x5651fb578670 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb95cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c2d0c7740 10 client.0 _ll_register_callbacks cb 0x7f6c180a9970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:20.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 -- 172.21.15.29:0/3624893222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 msgr2=0x5651fb951670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 0x5651fb951670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 -- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb599350 msgr2=0x5651fb951d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb599350 0x5651fb951d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.235+0000 7f6c248ef640 1 -- 172.21.15.29:0/3624893222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fb95bb60 con 0x5651fb570490 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.236+0000 7f6c248ef640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb95cf90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5651fb9675a0 tx=0x7f6c14005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25341cb06be9960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.236+0000 7f6bfffff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.236+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14035060 con 0x5651fb570490 2024-09-21T02:02:20.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.236+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6c14020840 con 0x5651fb570490 2024-09-21T02:02:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.236+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0c0316c0 con 0x5651fb570490 2024-09-21T02:02:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.238+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6c14020a40 con 0x5651fb570490 2024-09-21T02:02:20.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.238+0000 7f6c2d0c7740 10 client.18414 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:20.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.238+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5651fb9d1a60 con 0x5651fb570490 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c0316c0 con 0x5651fb570490 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6c1402f070 con 0x5651fb570490 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 10 client.18414 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 20 client.18414 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb52d080 0x5651fb9d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 4 client.18414 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:02:20.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.239+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651fb8b8db0 con 0x5651fb52d080 2024-09-21T02:02:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.240+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb52d080 0x5651fb9d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.240+0000 7f6bfffff640 10 client.18414 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.247+0000 7f6c1ffff640 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb52d080 0x5651fb9d4170 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb845841ed6f1d35 server_cookie=f2f269a5462605ae in_seq=0 out_seq=0 2024-09-21T02:02:20.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.251+0000 7f6bfffff640 1 -- 172.21.15.29:0/3624893222 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651fb8b8db0 con 0x5651fb52d080 2024-09-21T02:02:20.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.251+0000 7f6bfffff640 10 client.18414 handle_command_reply: tid=0 2024-09-21T02:02:20.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.251+0000 7f6c2d0c7740 1 client.18414 shutdown 2024-09-21T02:02:20.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 msgr2=0x5651fb95cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb95cf90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5651fb9675a0 tx=0x7f6c14005f90 comp rx=0 tx=0).stop 2024-09-21T02:02:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 shutdown_connections 2024-09-21T02:02:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5651fb52d080 0x5651fb9d4170 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651fb599350 0x5651fb951d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651fb578670 0x5651fb951670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 --2- 172.21.15.29:0/3624893222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651fb570490 0x5651fb95cf90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 >> 172.21.15.29:0/3624893222 conn(0x5651fb58d740 msgr2=0x5651fb946360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6bfffff640 0 client.18414 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 shutdown_connections 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.252+0000 7f6c2d0c7740 1 -- 172.21.15.29:0/3624893222 wait complete. 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.253+0000 7f6c2d0c7740 20 client.18414 trim_cache size 0 max 16384 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "c48dc890-9260-4f8f-b08e-1fd9d06fa732", 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T02:02:20.254 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:02:20.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 msgr2=0x7f6c100b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 0x7f6c100b8aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5651fb9478c0 tx=0x7f6c140097d0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 msgr2=0x7f6c2019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800eb10 tx=0x7f6c1800c2d0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 shutdown_connections 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6c100b6590 0x7f6c100b8aa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c2010de30 0x7f6c2019fcc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c201052b0 0x7f6c2019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 --2- 172.21.15.29:0/1483877347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c20104880 0x7f6c2019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.254+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 >> 172.21.15.29:0/1483877347 conn(0x7f6c20079730 msgr2=0x7f6c201008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.255+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 shutdown_connections 2024-09-21T02:02:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:20.255+0000 7f6c26b53640 1 -- 172.21.15.29:0/1483877347 wait complete. 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.866+0000 7f37f51ed640 1 -- 172.21.15.29:0/1947609893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f0103650 msgr2=0x7f37f0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.866+0000 7f37f51ed640 1 --2- 172.21.15.29:0/1947609893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f0103650 0x7f37f0103a60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009900 tx=0x7f37e402a420 comp rx=0 tx=0).stop 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.866+0000 7f37f51ed640 1 -- 172.21.15.29:0/1947609893 shutdown_connections 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.866+0000 7f37f51ed640 1 --2- 172.21.15.29:0/1947609893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103ff0 0x7f37f010a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.867+0000 7f37f51ed640 1 --2- 172.21.15.29:0/1947609893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f0103650 0x7f37f0103a60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.867+0000 7f37f51ed640 1 --2- 172.21.15.29:0/1947609893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f01025d0 0x7f37f0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.867+0000 7f37f51ed640 1 -- 172.21.15.29:0/1947609893 >> 172.21.15.29:0/1947609893 conn(0x7f37f00fdf30 msgr2=0x7f37f01003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.870+0000 7f37f51ed640 1 -- 172.21.15.29:0/1947609893 shutdown_connections 2024-09-21T02:02:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.872+0000 7f37f51ed640 1 -- 172.21.15.29:0/1947609893 wait complete. 2024-09-21T02:02:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.874+0000 7f37f51ed640 1 Processor -- start 2024-09-21T02:02:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.875+0000 7f37f51ed640 1 -- start start 2024-09-21T02:02:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37f51ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 0x7f37f0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37f51ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 0x7f37f0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37f51ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f0103ff0 0x7f37f0198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37f51ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37f010d170 con 0x7f37f0103ff0 2024-09-21T02:02:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 0x7f37f0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 0x7f37f0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:41648/0 (socket says 172.21.15.29:41648) 2024-09-21T02:02:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37ee59c640 1 -- 172.21.15.29:0/2262625427 learned_addr learned my addr 172.21.15.29:0/2262625427 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.876+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37f010cff0 con 0x7f37f0103650 2024-09-21T02:02:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37f010d2f0 con 0x7f37f01025d0 2024-09-21T02:02:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 0x7f37f0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37ef59e640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f0103ff0 0x7f37f0198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 -- 172.21.15.29:0/2262625427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 msgr2=0x7f37f0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 0x7f37f0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 -- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f0103ff0 msgr2=0x7f37f0198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f0103ff0 0x7f37f0198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.877+0000 7f37eed9d640 1 -- 172.21.15.29:0/2262625427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f01997d0 con 0x7f37f01025d0 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.878+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 0x7f37f0197d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e9f0 tx=0x7f37e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.878+0000 7f37d7fff640 1 -- 172.21.15.29:0/2262625427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8020070 con 0x7f37f01025d0 2024-09-21T02:02:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.878+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f0199ac0 con 0x7f37f01025d0 2024-09-21T02:02:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.878+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a01b0 con 0x7f37f01025d0 2024-09-21T02:02:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.880+0000 7f37d7fff640 1 -- 172.21.15.29:0/2262625427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e800cd80 con 0x7f37f01025d0 2024-09-21T02:02:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.880+0000 7f37d7fff640 1 -- 172.21.15.29:0/2262625427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e801bb80 con 0x7f37f01025d0 2024-09-21T02:02:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 Processor -- start 2024-09-21T02:02:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 -- start start 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a662f0 0x55d2b3e3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e40820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3da96f0 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3d98b20 con 0x55d2b3a662f0 2024-09-21T02:02:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.882+0000 7f37fc236740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3d9c930 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48396/0 (socket says 172.21.15.29:48396) 2024-09-21T02:02:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 -- 172.21.15.29:0/80939818 learned_addr learned my addr 172.21.15.29:0/80939818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ee59c640 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e40820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 -- 172.21.15.29:0/80939818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a662f0 msgr2=0x55d2b3e3fcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a662f0 0x55d2b3e3fcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 -- 172.21.15.29:0/80939818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a6e4d0 msgr2=0x55d2b3e40820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e40820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 -- 172.21.15.29:0/80939818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b3d5e530 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37d7fff640 1 -- 172.21.15.29:0/2262625427 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f37e80243f0 con 0x7f37f01025d0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.883+0000 7f37ef59e640 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37e00099e0 tx=0x7f37e0009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a62d5deb41e7af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.884+0000 7f37d57fa640 1 -- 172.21.15.29:0/80939818 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0017070 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.884+0000 7f37d57fa640 1 -- 172.21.15.29:0/80939818 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e0013470 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.884+0000 7f37d57fa640 1 -- 172.21.15.29:0/80939818 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0012610 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.884+0000 7f37d7fff640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 0x7f37d80b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 -- 172.21.15.29:0/80939818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 msgr2=0x55d2b3e3f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 0x7f37d80b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37e00099e0 tx=0x7f37e0009ea0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37d7fff640 1 -- 172.21.15.29:0/2262625427 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f37e80e6bd0 con 0x7f37f01025d0 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 -- 172.21.15.29:0/80939818 shutdown_connections 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e40820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a662f0 0x55d2b3e3fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 --2- 172.21.15.29:0/80939818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e3f660 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.885+0000 7f37fc236740 1 -- 172.21.15.29:0/80939818 >> 172.21.15.29:0/80939818 conn(0x55d2b3a83300 msgr2=0x55d2b3e328e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.886+0000 7f37fc236740 1 -- 172.21.15.29:0/80939818 shutdown_connections 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.886+0000 7f37fc236740 1 -- 172.21.15.29:0/80939818 wait complete. 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.886+0000 7f37eed9d640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 0x7f37d80b8b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f37e80ad7a0 tx=0x7f37e8022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 -- start start 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 Processor -- start 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e49550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e4a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3cd0ab0 con 0x55d2b3a662f0 2024-09-21T02:02:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3d3e730 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d2b3e41340 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37fc236740 10 client.0 _ll_register_callbacks cb 0x55d2b3d926b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59258/0 (socket says 172.21.15.29:59258) 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37ef59e640 1 -- 172.21.15.29:0/2506047476 learned_addr learned my addr 172.21.15.29:0/2506047476 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.887+0000 7f37ee59c640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e49550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e4a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 -- 172.21.15.29:0/2506047476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a6e4d0 msgr2=0x55d2b3e49550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e49550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 -- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a8f1b0 msgr2=0x55d2b3e4a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e4a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 -- 172.21.15.29:0/2506047476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b3afa790 con 0x55d2b3a662f0 2024-09-21T02:02:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37ef59e640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f37e000ef20 tx=0x7f37e0013c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fc6bd23427599e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.888+0000 7f37d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.889+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0017040 con 0x55d2b3a662f0 2024-09-21T02:02:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.889+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f37e001c650 con 0x55d2b3a662f0 2024-09-21T02:02:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.889+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e0013470 con 0x55d2b3a662f0 2024-09-21T02:02:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.890+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f37e001c850 con 0x55d2b3a662f0 2024-09-21T02:02:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.890+0000 7f37fc236740 10 client.18426 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.890+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d2b3b73620 con 0x55d2b3a662f0 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e00123d0 con 0x55d2b3a662f0 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f37e0011350 con 0x55d2b3a662f0 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 10 client.18426 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 10 client.18426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 20 client.18426 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a4dd80 0x55d2b3e56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 4 client.18426 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37e0013470 con 0x55d2b3a4dd80 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.891+0000 7f37ee59c640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a4dd80 0x55d2b3e56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.892+0000 7f37d2ff5640 10 client.18426 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.892+0000 7f37ee59c640 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a4dd80 0x55d2b3e56130 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65878bceb5bac594 server_cookie=2f49e3d7faf4183e in_seq=0 out_seq=0 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37d2ff5640 1 -- 172.21.15.29:0/2506047476 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f37e0013470 con 0x55d2b3a4dd80 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37d2ff5640 10 client.18426 handle_command_reply: tid=0 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 client.18426 shutdown 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 msgr2=0x55d2b3e48e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f37e000ef20 tx=0x7f37e0013c30 comp rx=0 tx=0).stop 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 shutdown_connections 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a4dd80 0x55d2b3e56130 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e4a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e49550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 --2- 172.21.15.29:0/2506047476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a662f0 0x55d2b3e48e90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 >> 172.21.15.29:0/2506047476 conn(0x55d2b3a83300 msgr2=0x55d2b3e30ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37d2ff5640 0 client.18426 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 shutdown_connections 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 1 -- 172.21.15.29:0/2506047476 wait complete. 2024-09-21T02:02:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.893+0000 7f37fc236740 20 client.18426 trim_cache size 0 max 16384 2024-09-21T02:02:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.914+0000 7f37fc236740 1 Processor -- start 2024-09-21T02:02:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.914+0000 7f37fc236740 1 -- start start 2024-09-21T02:02:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e32e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc00e540 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc010080 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37fc236740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc00ec10 con 0x55d2b3a662f0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48414/0 (socket says 172.21.15.29:48414) 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37ef59e640 1 -- 172.21.15.29:0/584574758 learned_addr learned my addr 172.21.15.29:0/584574758 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37ef59e640 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.915+0000 7f37ee59c640 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e32e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 -- 172.21.15.29:0/584574758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 msgr2=0x55d2b3e32e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e32e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 -- 172.21.15.29:0/584574758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 msgr2=0x55d2b3e333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 -- 172.21.15.29:0/584574758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b3e2e0a0 con 0x55d2b3a662f0 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37ef59e640 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37e00097a0 tx=0x7f37e0009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acde4224b41a8a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37d4ff9640 1 -- 172.21.15.29:0/584574758 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0013470 con 0x55d2b3a662f0 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37d4ff9640 1 -- 172.21.15.29:0/584574758 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e0011ac0 con 0x55d2b3a662f0 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.916+0000 7f37d4ff9640 1 -- 172.21.15.29:0/584574758 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e001c450 con 0x55d2b3a662f0 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.917+0000 7f37fc236740 1 -- 172.21.15.29:0/584574758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 msgr2=0x55d2b3e32290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.917+0000 7f37fc236740 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37e00097a0 tx=0x7f37e0009ea0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 -- 172.21.15.29:0/584574758 shutdown_connections 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e32e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 --2- 172.21.15.29:0/584574758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e32290 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 -- 172.21.15.29:0/584574758 >> 172.21.15.29:0/584574758 conn(0x55d2b3a83300 msgr2=0x55d2b3e2d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 -- 172.21.15.29:0/584574758 shutdown_connections 2024-09-21T02:02:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 -- 172.21.15.29:0/584574758 wait complete. 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.918+0000 7f37fc236740 1 -- start start 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 Processor -- start 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc00f9b0 con 0x55d2b3a8f1b0 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc00f2e0 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc00c520 con 0x55d2b3a662f0 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37fc236740 10 client.0 _ll_register_callbacks cb 0x7f37e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e50d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48420/0 (socket says 172.21.15.29:48420) 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37ef59e640 1 -- 172.21.15.29:0/1940386816 learned_addr learned my addr 172.21.15.29:0/1940386816 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37ee59c640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.919+0000 7f37ef59e640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 -- 172.21.15.29:0/1940386816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 msgr2=0x55d2b3e50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e50d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 -- 172.21.15.29:0/1940386816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 msgr2=0x55d2b3e500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 -- 172.21.15.29:0/1940386816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b3e50820 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37ee59c640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e4fa10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f37e4031440 tx=0x7f37e402a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8164bba9f362405 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4035020 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f37e4004310 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e0011ac0 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f37e4056050 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37fc236740 10 client.28247 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.920+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d2b3ec0c70 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e0011ac0 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f37e403a040 con 0x55d2b3a6e4d0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 10 client.28247 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 10 client.28247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 20 client.28247 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a231c0 0x55d2b3ec3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 4 client.28247 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2b3dab640 con 0x55d2b3a231c0 2024-09-21T02:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.921+0000 7f37ef59e640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a231c0 0x55d2b3ec3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.922+0000 7f37d2ff5640 10 client.28247 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.922+0000 7f37ef59e640 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a231c0 0x55d2b3ec3610 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6930024ff9114f44 server_cookie=5e923550278d0260 in_seq=0 out_seq=0 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.929+0000 7f37d2ff5640 1 -- 172.21.15.29:0/1940386816 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2b3dab640 con 0x55d2b3a231c0 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37d2ff5640 10 client.28247 handle_command_reply: tid=0 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 client.28247 shutdown 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 msgr2=0x55d2b3e4fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e4fa10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f37e4031440 tx=0x7f37e402a9c0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 shutdown_connections 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55d2b3a231c0 0x55d2b3ec3610 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2b3a8f1b0 0x55d2b3e500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b3a6e4d0 0x55d2b3e4fa10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 --2- 172.21.15.29:0/1940386816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2b3a662f0 0x55d2b3e50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37d2ff5640 0 client.28247 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 >> 172.21.15.29:0/1940386816 conn(0x55d2b3a83300 msgr2=0x55d2b3e31dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 shutdown_connections 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 1 -- 172.21.15.29:0/1940386816 wait complete. 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.930+0000 7f37fc236740 20 client.28247 trim_cache size 0 max 16384 2024-09-21T02:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:02:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:02:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:02:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 msgr2=0x7f37d80b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 0x7f37d80b8b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f37e80ad7a0 tx=0x7f37e8022000 comp rx=0 tx=0).stop 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 msgr2=0x7f37f0197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 0x7f37f0197d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e9f0 tx=0x7f37e800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 shutdown_connections 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37d80b6600 0x7f37d80b8b10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37f0103ff0 0x7f37f0198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f0103650 0x7f37f0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 --2- 172.21.15.29:0/2262625427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37f01025d0 0x7f37f0197d90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 >> 172.21.15.29:0/2262625427 conn(0x7f37f00fdf30 msgr2=0x7f37f0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 shutdown_connections 2024-09-21T02:02:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:20.931+0000 7f37f51ed640 1 -- 172.21.15.29:0/2262625427 wait complete. 2024-09-21T02:02:20.976 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:20.977 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:20 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:20.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:02:20.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:20 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:02:21.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:02:21.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:20 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.459 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:21.845 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:21 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:02:21.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:21 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:21 smithi029 ceph-mon[31228]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-09-21T02:02:21.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:21 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.846 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:21 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:21 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:02:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:21 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:21 smithi190 ceph-mon[34368]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-09-21T02:02:21.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:21 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:21.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:21 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:21 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:02:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:21 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:21 smithi066 ceph-mon[35001]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-09-21T02:02:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:21 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:22.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:21 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:02:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 -- 172.21.15.29:0/1990268955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c10fad0 msgr2=0x7fa15c10fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1990268955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c10fad0 0x7fa15c10fee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009860 tx=0x7fa14c02a340 comp rx=0 tx=0).stop 2024-09-21T02:02:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 -- 172.21.15.29:0/1990268955 shutdown_connections 2024-09-21T02:02:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1990268955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c110420 0x7fa15c116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1990268955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c10fad0 0x7fa15c10fee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1990268955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 -- 172.21.15.29:0/1990268955 >> 172.21.15.29:0/1990268955 conn(0x7fa15c0780f0 msgr2=0x7fa15c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.211+0000 7fa163c0a640 1 -- 172.21.15.29:0/1990268955 shutdown_connections 2024-09-21T02:02:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.212+0000 7fa163c0a640 1 -- 172.21.15.29:0/1990268955 wait complete. 2024-09-21T02:02:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.213+0000 7fa163c0a640 1 Processor -- start 2024-09-21T02:02:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.213+0000 7fa163c0a640 1 -- start start 2024-09-21T02:02:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c083cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c10fad0 0x7fa15c07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 0x7fa15c07d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c11a090 con 0x7fa15c10fad0 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c119f10 con 0x7fa15c10a190 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa163c0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c11a210 con 0x7fa15c110420 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 0x7fa15c07d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 0x7fa15c07d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60914/0 (socket says 172.21.15.29:60914) 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa1621a7640 1 -- 172.21.15.29:0/1521083791 learned_addr learned my addr 172.21.15.29:0/1521083791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:22.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.214+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c083cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1611a5640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c10fad0 0x7fa15c07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1619a6640 1 -- 172.21.15.29:0/1521083791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 msgr2=0x7fa15c07d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 0x7fa15c07d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1619a6640 1 -- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c10fad0 msgr2=0x7fa15c07cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c10fad0 0x7fa15c07cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.215+0000 7fa1619a6640 1 -- 172.21.15.29:0/1521083791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c07dd70 con 0x7fa15c10a190 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c083cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e9f0 tx=0x7fa15800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa152ffd640 1 -- 172.21.15.29:0/1521083791 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15800cd80 con 0x7fa15c10a190 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa152ffd640 1 -- 172.21.15.29:0/1521083791 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1580035f0 con 0x7fa15c10a190 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c0829f0 con 0x7fa15c10a190 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa152ffd640 1 -- 172.21.15.29:0/1521083791 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158010670 con 0x7fa15c10a190 2024-09-21T02:02:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.216+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c082f50 con 0x7fa15c10a190 2024-09-21T02:02:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.218+0000 7fa16a17e740 1 Processor -- start 2024-09-21T02:02:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 -- start start 2024-09-21T02:02:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa152ffd640 1 -- 172.21.15.29:0/1521083791 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa1580037a0 con 0x7fa15c10a190 2024-09-21T02:02:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6076b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b1e5fece50 con 0x55b1e5cae390 2024-09-21T02:02:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b1e5fdc280 con 0x55b1e5ccf070 2024-09-21T02:02:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.219+0000 7fa16a17e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b1e5fe0090 con 0x55b1e5ca61b0 2024-09-21T02:02:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:47106/0 (socket says 172.21.15.29:47106) 2024-09-21T02:02:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1621a7640 1 -- 172.21.15.29:0/2724530980 learned_addr learned my addr 172.21.15.29:0/2724530980 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:22.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1611a5640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1619a6640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.220+0000 7fa1611a5640 1 -- 172.21.15.29:0/2724530980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 msgr2=0x55b1e6076b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa152ffd640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 0x7fa1480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1611a5640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6076b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1611a5640 1 -- 172.21.15.29:0/2724530980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 msgr2=0x55b1e6076650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1611a5640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1611a5640 1 -- 172.21.15.29:0/2724530980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e5d3a650 con 0x55b1e5cae390 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1621a7640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1619a6640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa152ffd640 1 -- 172.21.15.29:0/1521083791 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa1580e74a0 con 0x7fa15c10a190 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1611a5640 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60770d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a860 tx=0x7fa14c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabde9d8e9931497 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.221+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 0x7fa1480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.222+0000 7fa147fff640 1 -- 172.21.15.29:0/2724530980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c041070 con 0x55b1e5cae390 2024-09-21T02:02:22.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.222+0000 7fa147fff640 1 -- 172.21.15.29:0/2724530980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14c0044d0 con 0x55b1e5cae390 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa147fff640 1 -- 172.21.15.29:0/2724530980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c037040 con 0x55b1e5cae390 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 -- 172.21.15.29:0/2724530980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 msgr2=0x55b1e60770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 0x7fa1480b8aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa154009c50 tx=0x7fa154004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60770d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a860 tx=0x7fa14c0095a0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 -- 172.21.15.29:0/2724530980 shutdown_connections 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60770d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 --2- 172.21.15.29:0/2724530980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6076650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 -- 172.21.15.29:0/2724530980 >> 172.21.15.29:0/2724530980 conn(0x55b1e5cc3460 msgr2=0x55b1e6074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 -- 172.21.15.29:0/2724530980 shutdown_connections 2024-09-21T02:02:22.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.223+0000 7fa16a17e740 1 -- 172.21.15.29:0/2724530980 wait complete. 2024-09-21T02:02:22.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.224+0000 7fa16a17e740 1 -- start start 2024-09-21T02:02:22.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.224+0000 7fa16a17e740 1 Processor -- start 2024-09-21T02:02:22.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ca61b0 0x55b1e6081cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b1e5e3d700 con 0x55b1e5cae390 2024-09-21T02:02:22.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b1e5f77110 con 0x55b1e5ca61b0 2024-09-21T02:02:22.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b1e6084e20 con 0x55b1e5ccf070 2024-09-21T02:02:22.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa16a17e740 10 client.0 _ll_register_callbacks cb 0x55b1e5f90990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa1621a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60934/0 (socket says 172.21.15.29:60934) 2024-09-21T02:02:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa1621a7640 1 -- 172.21.15.29:0/4141222621 learned_addr learned my addr 172.21.15.29:0/4141222621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa1611a5640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.225+0000 7fa1619a6640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ca61b0 0x55b1e6081cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 -- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ca61b0 msgr2=0x55b1e6081cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ca61b0 0x55b1e6081cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 -- 172.21.15.29:0/4141222621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 msgr2=0x55b1e60821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 -- 172.21.15.29:0/4141222621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e5fa4c40 con 0x55b1e5ccf070 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1621a7640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa15400e780 tx=0x7fa154010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c66ead81ff7f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1457fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154025440 con 0x55b1e5ccf070 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa1540255e0 con 0x55b1e5ccf070 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.228+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa14c0044d0 con 0x55b1e5ccf070 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa1540257e0 con 0x55b1e5ccf070 2024-09-21T02:02:22.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 10 client.28228 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b1e6090af0 con 0x55b1e5ccf070 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15402a070 con 0x55b1e5ccf070 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa15400b040 con 0x55b1e5ccf070 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 10 client.28228 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 10 client.28228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 20 client.28228 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c8d9f0 0x55b1e6094060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 4 client.28228 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa14c0044d0 con 0x55b1e5c8d9f0 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.229+0000 7fa1619a6640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c8d9f0 0x55b1e6094060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.230+0000 7fa1457fa640 10 client.28228 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:22.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.230+0000 7fa1619a6640 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c8d9f0 0x55b1e6094060 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b85e4b678e9f9b server_cookie=fb317da26f9c4c45 in_seq=0 out_seq=0 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa1457fa640 1 -- 172.21.15.29:0/4141222621 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa14c0044d0 con 0x55b1e5c8d9f0 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa1457fa640 10 client.28228 handle_command_reply: tid=0 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 client.28228 shutdown 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 msgr2=0x55b1e608e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa15400e780 tx=0x7fa154010040 comp rx=0 tx=0).stop 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 shutdown_connections 2024-09-21T02:02:22.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c8d9f0 0x55b1e6094060 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e608e160 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e60821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa1457fa640 0 client.28228 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 --2- 172.21.15.29:0/4141222621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ca61b0 0x55b1e6081cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 >> 172.21.15.29:0/4141222621 conn(0x55b1e5cc3460 msgr2=0x55b1e6079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 shutdown_connections 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 1 -- 172.21.15.29:0/4141222621 wait complete. 2024-09-21T02:02:22.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.231+0000 7fa16a17e740 20 client.28228 trim_cache size 0 max 16384 2024-09-21T02:02:22.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.253+0000 7fa16a17e740 1 Processor -- start 2024-09-21T02:02:22.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 -- start start 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e608e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6073b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1be160 con 0x55b1e5cae390 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1bfca0 con 0x55b1e5ccf070 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa16a17e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1be830 con 0x55b1e5ca61b0 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60938/0 (socket says 172.21.15.29:60938) 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 -- 172.21.15.29:0/3768406693 learned_addr learned my addr 172.21.15.29:0/3768406693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1621a7640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1611a5640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e608e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 -- 172.21.15.29:0/3768406693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 msgr2=0x55b1e6073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 -- 172.21.15.29:0/3768406693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 msgr2=0x55b1e608e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e608e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1619a6640 1 -- 172.21.15.29:0/3768406693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e6078940 con 0x55b1e5ca61b0 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1621a7640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.254+0000 7fa1611a5640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e608e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.255+0000 7fa1619a6640 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ad5d0 tx=0x7fa1580a9d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=255fcc7853532c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.255+0000 7fa1477fe640 1 -- 172.21.15.29:0/3768406693 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1580b2120 con 0x55b1e5ca61b0 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.255+0000 7fa1477fe640 1 -- 172.21.15.29:0/3768406693 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1580b7060 con 0x55b1e5ca61b0 2024-09-21T02:02:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.255+0000 7fa1477fe640 1 -- 172.21.15.29:0/3768406693 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1580e70b0 con 0x55b1e5ca61b0 2024-09-21T02:02:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 -- 172.21.15.29:0/3768406693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 msgr2=0x55b1e6082170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ad5d0 tx=0x7fa1580a9d20 comp rx=0 tx=0).stop 2024-09-21T02:02:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 -- 172.21.15.29:0/3768406693 shutdown_connections 2024-09-21T02:02:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5ccf070 0x55b1e6073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5cae390 0x55b1e608e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 --2- 172.21.15.29:0/3768406693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ca61b0 0x55b1e6082170 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 -- 172.21.15.29:0/3768406693 >> 172.21.15.29:0/3768406693 conn(0x55b1e5cc3460 msgr2=0x55b1e6081a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 -- 172.21.15.29:0/3768406693 shutdown_connections 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.256+0000 7fa16a17e740 1 -- 172.21.15.29:0/3768406693 wait complete. 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.257+0000 7fa16a17e740 1 -- start start 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.257+0000 7fa16a17e740 1 Processor -- start 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5cae390 0x55b1e6085de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e60863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1bf5d0 con 0x55b1e5ca61b0 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1bef00 con 0x55b1e5cae390 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1bc140 con 0x55b1e5ccf070 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa16a17e740 10 client.0 _ll_register_callbacks cb 0x7fa15400e6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1619a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1619a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53418/0 (socket says 172.21.15.29:53418) 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1619a6640 1 -- 172.21.15.29:0/1868593058 learned_addr learned my addr 172.21.15.29:0/1868593058 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e60863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1611a5640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5cae390 0x55b1e6085de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 -- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5cae390 msgr2=0x55b1e6085de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5cae390 0x55b1e6085de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 -- 172.21.15.29:0/1868593058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 msgr2=0x55b1e608a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1621a7640 1 -- 172.21.15.29:0/1868593058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e6086b30 con 0x55b1e5ccf070 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.258+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1621a7640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e60863a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa154030bd0 tx=0x7fa154004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1031588ff0e952 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1457fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15403e020 con 0x55b1e5ccf070 2024-09-21T02:02:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fa154030640 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1580b7060 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa15400d830 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa16a17e740 10 client.28234 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b1e60fea10 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.259+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1580b7060 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa154039070 con 0x55b1e5ccf070 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 10 client.28234 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 10 client.28234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 20 client.28234 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c62da0 0x55b1e61011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 4 client.28234 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:02:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1e5feeda0 con 0x55b1e5c62da0 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.260+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c62da0 0x55b1e61011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.262+0000 7fa1619a6640 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c62da0 0x55b1e61011d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b25219550eb495ac server_cookie=be45f597cce8964d in_seq=0 out_seq=0 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.262+0000 7fa1457fa640 10 client.28234 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.262+0000 7fa1457fa640 1 -- 172.21.15.29:0/1868593058 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55b1e5feeda0 con 0x55b1e5c62da0 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.262+0000 7fa1457fa640 10 client.28234 handle_command_reply: tid=0 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 client.28234 shutdown 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 msgr2=0x55b1e60863a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e60863a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa154030bd0 tx=0x7fa154004ab0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 shutdown_connections 2024-09-21T02:02:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55b1e5c62da0 0x55b1e61011d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "c48dc890-9260-4f8f-b08e-1fd9d06fa732": { 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "c48dc890-9260-4f8f-b08e-1fd9d06fa732", 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "c48dc890-9260-4f8f-b08e-1fd9d06fa732": { 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:02:20.247270+0000", 2024-09-21T02:02:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:02:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:02:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:02:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:22.277 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b1e5ccf070 0x55b1e60863a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1e5cae390 0x55b1e6085de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 --2- 172.21.15.29:0/1868593058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e5ca61b0 0x55b1e608a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 >> 172.21.15.29:0/1868593058 conn(0x55b1e5cc3460 msgr2=0x55b1e6077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa1457fa640 0 client.28234 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 shutdown_connections 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 1 -- 172.21.15.29:0/1868593058 wait complete. 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.263+0000 7fa16a17e740 20 client.28234 trim_cache size 0 max 16384 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 msgr2=0x7fa1480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 0x7fa1480b8aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa154009c50 tx=0x7fa154004a40 comp rx=0 tx=0).stop 2024-09-21T02:02:22.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 msgr2=0x7fa15c083cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c083cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e9f0 tx=0x7fa15800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 shutdown_connections 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fa1480b6590 0x7fa1480b8aa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa15c110420 0x7fa15c07d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15c10fad0 0x7fa15c07cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 --2- 172.21.15.29:0/1521083791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa15c10a190 0x7fa15c083cc0 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e9f0 tx=0x7fa15800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.270+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 >> 172.21.15.29:0/1521083791 conn(0x7fa15c0780f0 msgr2=0x7fa15c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.278+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 shutdown_connections 2024-09-21T02:02:22.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:22.278+0000 7fa163c0a640 1 -- 172.21.15.29:0/1521083791 wait complete. 2024-09-21T02:02:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 825.0 s, 1488.2 tps, lat 66.516 ms stddev 70.563 2024-09-21T02:02:23.194 INFO:tasks.cephfs.filesystem:scrub status for tag:c48dc890-9260-4f8f-b08e-1fd9d06fa732 - {'path': '/', 'tag': 'c48dc890-9260-4f8f-b08e-1fd9d06fa732', 'options': 'recursive,force'} 2024-09-21T02:02:23.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:23 smithi190 ceph-mon[34368]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-09-21T02:02:24.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:23 smithi066 ceph-mon[35001]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-09-21T02:02:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:23 smithi029 ceph-mon[31228]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-09-21T02:02:25.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:25 smithi190 ceph-mon[34368]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.7 MiB/s wr, 700 op/s 2024-09-21T02:02:26.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:25 smithi066 ceph-mon[35001]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.7 MiB/s wr, 700 op/s 2024-09-21T02:02:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:25 smithi029 ceph-mon[31228]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.7 MiB/s wr, 700 op/s 2024-09-21T02:02:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 830.0 s, 1777.8 tps, lat 56.994 ms stddev 67.286 2024-09-21T02:02:27.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:27 smithi190 ceph-mon[34368]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.5 MiB/s wr, 784 op/s 2024-09-21T02:02:28.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:27 smithi066 ceph-mon[35001]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.5 MiB/s wr, 784 op/s 2024-09-21T02:02:28.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:27 smithi029 ceph-mon[31228]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.5 MiB/s wr, 784 op/s 2024-09-21T02:02:29.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:29 smithi190 ceph-mon[34368]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.5 MiB/s wr, 631 op/s 2024-09-21T02:02:30.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:29 smithi066 ceph-mon[35001]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.5 MiB/s wr, 631 op/s 2024-09-21T02:02:30.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:29 smithi029 ceph-mon[31228]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.5 MiB/s wr, 631 op/s 2024-09-21T02:02:31.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:31 smithi190 ceph-mon[34368]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.7 MiB/s wr, 732 op/s 2024-09-21T02:02:32.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:31 smithi066 ceph-mon[35001]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.7 MiB/s wr, 732 op/s 2024-09-21T02:02:32.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:31 smithi029 ceph-mon[31228]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.7 MiB/s wr, 732 op/s 2024-09-21T02:02:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 835.0 s, 2054.2 tps, lat 48.616 ms stddev 59.906 2024-09-21T02:02:33.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:33 smithi190 ceph-mon[34368]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 820 op/s 2024-09-21T02:02:34.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:33 smithi066 ceph-mon[35001]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 820 op/s 2024-09-21T02:02:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 -- 172.21.15.29:0/3252345019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 msgr2=0x7fbeec102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 --2- 172.21.15.29:0/3252345019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec102a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00dc90 tx=0x7fbedc02e370 comp rx=0 tx=0).stop 2024-09-21T02:02:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 -- 172.21.15.29:0/3252345019 shutdown_connections 2024-09-21T02:02:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 --2- 172.21.15.29:0/3252345019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103fc0 0x7fbeec10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 --2- 172.21.15.29:0/3252345019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeec103620 0x7fbeec103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 --2- 172.21.15.29:0/3252345019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec102a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 -- 172.21.15.29:0/3252345019 >> 172.21.15.29:0/3252345019 conn(0x7fbeec0fdee0 msgr2=0x7fbeec100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 -- 172.21.15.29:0/3252345019 shutdown_connections 2024-09-21T02:02:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.068+0000 7fbef4362640 1 -- 172.21.15.29:0/3252345019 wait complete. 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.069+0000 7fbef4362640 1 Processor -- start 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 -- start start 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103620 0x7fbeec1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeec103fc0 0x7fbeec198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef20fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef20fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:32930/0 (socket says 172.21.15.29:32930) 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef18fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103620 0x7fbeec1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10d120 con 0x7fbeec103620 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef20fe640 1 -- 172.21.15.29:0/2760713291 learned_addr learned my addr 172.21.15.29:0/2760713291 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10cfa0 con 0x7fbeec103fc0 2024-09-21T02:02:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.070+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10d2a0 con 0x7fbeec1025a0 2024-09-21T02:02:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 -- 172.21.15.29:0/2760713291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeec103fc0 msgr2=0x7fbeec198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeec103fc0 0x7fbeec198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 -- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103620 msgr2=0x7fbeec1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103620 0x7fbeec1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 -- 172.21.15.29:0/2760713291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec199780 con 0x7fbeec1025a0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.071+0000 7fbef20fe640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d5c0 tx=0x7fbedc00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.072+0000 7fbee37fe640 1 -- 172.21.15.29:0/2760713291 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc005d50 con 0x7fbeec1025a0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.072+0000 7fbee37fe640 1 -- 172.21.15.29:0/2760713291 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbedc00b5d0 con 0x7fbeec1025a0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.072+0000 7fbee37fe640 1 -- 172.21.15.29:0/2760713291 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc005d50 con 0x7fbeec1025a0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.072+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec199a10 con 0x7fbeec1025a0 2024-09-21T02:02:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.072+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec19ff80 con 0x7fbeec1025a0 2024-09-21T02:02:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.075+0000 7fbefb2d2740 1 Processor -- start 2024-09-21T02:02:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 -- start start 2024-09-21T02:02:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a959660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5802f0 0x55957a959cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55957a8c36f0 con 0x55957a5884d0 2024-09-21T02:02:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55957a8b2b20 con 0x55957a5a91b0 2024-09-21T02:02:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55957a8b6930 con 0x55957a5802f0 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbef18fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbef18fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47406/0 (socket says 172.21.15.29:47406) 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.076+0000 7fbef18fd640 1 -- 172.21.15.29:0/379382731 learned_addr learned my addr 172.21.15.29:0/379382731 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 -- 172.21.15.29:0/379382731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5802f0 msgr2=0x55957a959cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5802f0 0x55957a959cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 -- 172.21.15.29:0/379382731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 msgr2=0x55957a959660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a959660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 -- 172.21.15.29:0/379382731 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957a878530 con 0x55957a5884d0 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbef18fd640 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbee800bd10 tx=0x7fbee80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f0e993b5695b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.077+0000 7fbee0bfe640 1 -- 172.21.15.29:0/379382731 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004030 con 0x55957a5884d0 2024-09-21T02:02:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.078+0000 7fbee0bfe640 1 -- 172.21.15.29:0/379382731 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee80035f0 con 0x55957a5884d0 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.078+0000 7fbee37fe640 1 -- 172.21.15.29:0/2760713291 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbedc046020 con 0x7fbeec1025a0 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.078+0000 7fbee0bfe640 1 -- 172.21.15.29:0/379382731 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011680 con 0x55957a5884d0 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 -- 172.21.15.29:0/379382731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 msgr2=0x55957a95a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbee800bd10 tx=0x7fbee80107a0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbee37fe640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 0x7fbed80b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 -- 172.21.15.29:0/379382731 shutdown_connections 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5884d0 0x55957a95a820 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5802f0 0x55957a959cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/379382731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a959660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 -- 172.21.15.29:0/379382731 >> 172.21.15.29:0/379382731 conn(0x55957a59d300 msgr2=0x55957a9550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbef28ff640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 0x7fbed80b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbee37fe640 1 -- 172.21.15.29:0/2760713291 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbedc044020 con 0x7fbeec1025a0 2024-09-21T02:02:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 -- 172.21.15.29:0/379382731 shutdown_connections 2024-09-21T02:02:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.079+0000 7fbefb2d2740 1 -- 172.21.15.29:0/379382731 wait complete. 2024-09-21T02:02:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.080+0000 7fbef28ff640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 0x7fbed80b8e90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009250 tx=0x7fbee40091e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.080+0000 7fbefb2d2740 1 -- start start 2024-09-21T02:02:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.080+0000 7fbefb2d2740 1 Processor -- start 2024-09-21T02:02:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.080+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.080+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5884d0 0x55957a963530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a9640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55957a7eaab0 con 0x55957a5802f0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55957a858730 con 0x55957a5a91b0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55957a95b340 con 0x55957a5884d0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47412/0 (socket says 172.21.15.29:47412) 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 -- 172.21.15.29:0/341137853 learned_addr learned my addr 172.21.15.29:0/341137853 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef18fd640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5884d0 0x55957a963530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbefb2d2740 10 client.0 _ll_register_callbacks cb 0x55957a8ac6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 -- 172.21.15.29:0/341137853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5884d0 msgr2=0x55957a963530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5884d0 0x55957a963530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 -- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 msgr2=0x55957a9640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a9640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 -- 172.21.15.29:0/341137853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957a614790 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbef20fe640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0c8b80 tx=0x7fbedc0c8510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5053c1c1945d5f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.081+0000 7fbed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0cc0c0 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee80035f0 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbedc00a320 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fbedc0cccb0 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbefb2d2740 10 client.18462 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.082+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55957a68d620 con 0x55957a5802f0 2024-09-21T02:02:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.083+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee80035f0 con 0x55957a5802f0 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fbedc0c7f00 con 0x55957a5802f0 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 10 client.18462 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 10 client.18462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 20 client.18462 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a567d80 0x55957a970580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 4 client.18462 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957a8c5640 con 0x55957a567d80 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbef18fd640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a567d80 0x55957a970580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbed5ffb640 10 client.18462 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.084+0000 7fbef18fd640 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a567d80 0x55957a970580 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdaae3ef660cd52f server_cookie=ac9b826278aed637 in_seq=0 out_seq=0 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbed5ffb640 1 -- 172.21.15.29:0/341137853 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55957a8c5640 con 0x55957a567d80 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbed5ffb640 10 client.18462 handle_command_reply: tid=0 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 client.18462 shutdown 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 msgr2=0x55957a962e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0c8b80 tx=0x7fbedc0c8510 comp rx=0 tx=0).stop 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 shutdown_connections 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a567d80 0x55957a970580 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbed5ffb640 0 client.18462 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a5a91b0 0x55957a9640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a5884d0 0x55957a963530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/341137853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a962e70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 >> 172.21.15.29:0/341137853 conn(0x55957a59d300 msgr2=0x55957a94aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 shutdown_connections 2024-09-21T02:02:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.086+0000 7fbefb2d2740 1 -- 172.21.15.29:0/341137853 wait complete. 2024-09-21T02:02:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.087+0000 7fbefb2d2740 20 client.18462 trim_cache size 0 max 16384 2024-09-21T02:02:34.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:33 smithi029 ceph-mon[31228]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 820 op/s 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 Processor -- start 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 -- start start 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a59d300 0x55957a954a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a567d80 0x55957a961380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1a8d60 con 0x55957a5802f0 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1aa8a0 con 0x55957a567d80 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1a9430 con 0x55957a59d300 2024-09-21T02:02:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbef28ff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbef28ff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47418/0 (socket says 172.21.15.29:47418) 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbef28ff640 1 -- 172.21.15.29:0/3807346054 learned_addr learned my addr 172.21.15.29:0/3807346054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbef18fd640 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a567d80 0x55957a961380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.109+0000 7fbef20fe640 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a59d300 0x55957a954a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 -- 172.21.15.29:0/3807346054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a59d300 msgr2=0x55957a954a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a59d300 0x55957a954a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 -- 172.21.15.29:0/3807346054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a567d80 msgr2=0x55957a961380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a567d80 0x55957a961380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 -- 172.21.15.29:0/3807346054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957a961e00 con 0x55957a5802f0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbef28ff640 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbee4004ca0 tx=0x7fbee40094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc81a74ad655939 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbed7fff640 1 -- 172.21.15.29:0/3807346054 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4021070 con 0x55957a5802f0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbed7fff640 1 -- 172.21.15.29:0/3807346054 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee4004cd0 con 0x55957a5802f0 2024-09-21T02:02:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.110+0000 7fbed7fff640 1 -- 172.21.15.29:0/3807346054 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4005330 con 0x55957a5802f0 2024-09-21T02:02:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- 172.21.15.29:0/3807346054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 msgr2=0x55957a9618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbee4004ca0 tx=0x7fbee40094e0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- 172.21.15.29:0/3807346054 shutdown_connections 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a9618c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a567d80 0x55957a961380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/3807346054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a59d300 0x55957a954a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- 172.21.15.29:0/3807346054 >> 172.21.15.29:0/3807346054 conn(0x55957a9708b0 msgr2=0x55957a9475c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- 172.21.15.29:0/3807346054 shutdown_connections 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- 172.21.15.29:0/3807346054 wait complete. 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.111+0000 7fbefb2d2740 1 -- start start 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.112+0000 7fbefb2d2740 1 Processor -- start 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a567d80 0x55957a96aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a59d300 0x55957a969c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1aa1d0 con 0x55957a5802f0 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1a9b00 con 0x55957a59d300 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1a6d40 con 0x55957a567d80 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbefb2d2740 10 client.0 _ll_register_callbacks cb 0x7fbedc00d5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbef18fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbef28ff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a59d300 0x55957a969c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbef18fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47434/0 (socket says 172.21.15.29:47434) 2024-09-21T02:02:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.113+0000 7fbef18fd640 1 -- 172.21.15.29:0/1136018503 learned_addr learned my addr 172.21.15.29:0/1136018503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 -- 172.21.15.29:0/1136018503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a567d80 msgr2=0x55957a96aa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a567d80 0x55957a96aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 -- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a59d300 msgr2=0x55957a969c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a59d300 0x55957a969c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 -- 172.21.15.29:0/1136018503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957a96a1b0 con 0x55957a5802f0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbef18fd640 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbee800baa0 tx=0x7fbee8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35e9bfde32f28aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8043060 con 0x55957a5802f0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbee80052b0 con 0x55957a5802f0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee4004cd0 con 0x55957a5802f0 2024-09-21T02:02:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fbee80054b0 con 0x55957a5802f0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbefb2d2740 10 client.18468 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.114+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55957a969080 con 0x55957a5802f0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee4004cd0 con 0x55957a5802f0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fbee8011cc0 con 0x55957a5802f0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 10 client.18468 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 10 client.18468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 20 client.18468 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a5884d0 0x55957a94e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 4 client.18468 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55957a8c10f0 con 0x55957a5884d0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.115+0000 7fbef20fe640 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a5884d0 0x55957a94e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.116+0000 7fbed5ffb640 10 client.18468 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.116+0000 7fbef20fe640 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a5884d0 0x55957a94e6c0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529b7b55710fc28c server_cookie=5cc3a5ceee684096 in_seq=0 out_seq=0 2024-09-21T02:02:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.123+0000 7fbed5ffb640 1 -- 172.21.15.29:0/1136018503 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55957a8c10f0 con 0x55957a5884d0 2024-09-21T02:02:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.123+0000 7fbed5ffb640 10 client.18468 handle_command_reply: tid=0 2024-09-21T02:02:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.123+0000 7fbefb2d2740 1 client.18468 shutdown 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 msgr2=0x55957a969700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbee800baa0 tx=0x7fbee8003b20 comp rx=0 tx=0).stop 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 shutdown_connections 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55957a5884d0 0x55957a94e6c0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957a59d300 0x55957a969c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55957a5802f0 0x55957a969700 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 --2- 172.21.15.29:0/1136018503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55957a567d80 0x55957a96aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 >> 172.21.15.29:0/1136018503 conn(0x55957a9708b0 msgr2=0x55957a9611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbed5ffb640 0 client.18468 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 shutdown_connections 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.124+0000 7fbefb2d2740 1 -- 172.21.15.29:0/1136018503 wait complete. 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbefb2d2740 20 client.18468 trim_cache size 0 max 16384 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:02:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 msgr2=0x7fbed80b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 0x7fbed80b8e90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009250 tx=0x7fbee40091e0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 msgr2=0x7fbeec197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d5c0 tx=0x7fbedc00bab0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 shutdown_connections 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbed80b6980 0x7fbed80b8e90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeec103fc0 0x7fbeec198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeec103620 0x7fbeec1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 --2- 172.21.15.29:0/2760713291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeec1025a0 0x7fbeec197d40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 >> 172.21.15.29:0/2760713291 conn(0x7fbeec0fdee0 msgr2=0x7fbeec106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 shutdown_connections 2024-09-21T02:02:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:34.125+0000 7fbef4362640 1 -- 172.21.15.29:0/2760713291 wait complete. 2024-09-21T02:02:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:35 smithi190 ceph-mon[34368]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-09-21T02:02:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:02:35.978 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:02:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:02:35.979 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:35 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:02:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:35 smithi066 ceph-mon[35001]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-09-21T02:02:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:02:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:02:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:02:36.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:35 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:02:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:35 smithi029 ceph-mon[31228]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-09-21T02:02:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:02:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:02:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:02:36.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:35 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:02:36.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:36 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:02:37.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:36 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:02:37.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:36 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:02:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 840.0 s, 1627.4 tps, lat 60.666 ms stddev 68.664 2024-09-21T02:02:38.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:37 smithi066 ceph-mon[35001]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-21T02:02:38.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:37 smithi029 ceph-mon[31228]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-21T02:02:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:37 smithi190 ceph-mon[34368]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-21T02:02:40.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:39 smithi066 ceph-mon[35001]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 595 op/s 2024-09-21T02:02:40.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:39 smithi029 ceph-mon[31228]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 595 op/s 2024-09-21T02:02:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:39 smithi190 ceph-mon[34368]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 595 op/s 2024-09-21T02:02:42.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:41 smithi066 ceph-mon[35001]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 652 op/s 2024-09-21T02:02:42.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:41 smithi029 ceph-mon[31228]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 652 op/s 2024-09-21T02:02:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:41 smithi190 ceph-mon[34368]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 652 op/s 2024-09-21T02:02:42.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 845.0 s, 1428.2 tps, lat 70.981 ms stddev 81.246 2024-09-21T02:02:44.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:43 smithi066 ceph-mon[35001]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-09-21T02:02:44.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:43 smithi029 ceph-mon[31228]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-09-21T02:02:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:43 smithi190 ceph-mon[34368]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-09-21T02:02:46.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:45 smithi066 ceph-mon[35001]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T02:02:46.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:45 smithi029 ceph-mon[31228]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T02:02:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:45 smithi190 ceph-mon[34368]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-09-21T02:02:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.278+0000 7fc5e3455640 1 -- 172.21.15.29:0/889491731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 msgr2=0x7fc5dc103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.278+0000 7fc5e3455640 1 --2- 172.21.15.29:0/889491731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 0x7fc5dc103b30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009be0 tx=0x7fc5d402a6f0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 -- 172.21.15.29:0/889491731 shutdown_connections 2024-09-21T02:02:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 --2- 172.21.15.29:0/889491731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 --2- 172.21.15.29:0/889491731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc104740 0x7fc5dc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 --2- 172.21.15.29:0/889491731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 0x7fc5dc103b30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 -- 172.21.15.29:0/889491731 >> 172.21.15.29:0/889491731 conn(0x7fc5dc0ff020 msgr2=0x7fc5dc1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.279+0000 7fc5e3455640 1 -- 172.21.15.29:0/889491731 shutdown_connections 2024-09-21T02:02:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.280+0000 7fc5e3455640 1 -- 172.21.15.29:0/889491731 wait complete. 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.280+0000 7fc5e3455640 1 Processor -- start 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 -- start start 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 0x7fc5dc191810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc104740 0x7fc5dc191e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc06f900 con 0x7fc5dc1050e0 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc06f780 con 0x7fc5dc1036c0 2024-09-21T02:02:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e3455640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc06fa80 con 0x7fc5dc104740 2024-09-21T02:02:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e19f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e19f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48404/0 (socket says 172.21.15.29:48404) 2024-09-21T02:02:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.281+0000 7fc5e19f2640 1 -- 172.21.15.29:0/2510845699 learned_addr learned my addr 172.21.15.29:0/2510845699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.282+0000 7fc5e19f2640 1 -- 172.21.15.29:0/2510845699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc104740 msgr2=0x7fc5dc191e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.282+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc104740 0x7fc5dc191e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.282+0000 7fc5e19f2640 1 -- 172.21.15.29:0/2510845699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 msgr2=0x7fc5dc191810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.282+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 0x7fc5dc191810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.282+0000 7fc5e19f2640 1 -- 172.21.15.29:0/2510845699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc193220 con 0x7fc5dc1050e0 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.283+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801e920 tx=0x7fc5d801ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.283+0000 7fc5d27fc640 1 -- 172.21.15.29:0/2510845699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8028070 con 0x7fc5dc1050e0 2024-09-21T02:02:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.283+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc193510 con 0x7fc5dc1050e0 2024-09-21T02:02:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.283+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc199be0 con 0x7fc5dc1050e0 2024-09-21T02:02:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.284+0000 7fc5d27fc640 1 -- 172.21.15.29:0/2510845699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d8005020 con 0x7fc5dc1050e0 2024-09-21T02:02:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.284+0000 7fc5d27fc640 1 -- 172.21.15.29:0/2510845699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8004d10 con 0x7fc5dc1050e0 2024-09-21T02:02:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.287+0000 7fc5ea3bf740 1 Processor -- start 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5d27fc640 1 -- 172.21.15.29:0/2510845699 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc5d8021470 con 0x7fc5dc1050e0 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 -- start start 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5190420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5652f50fd6f0 con 0x5652f4dba2f0 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652f50ecb20 con 0x5652f4de31b0 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5652f50f0930 con 0x5652f4dc24d0 2024-09-21T02:02:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5e09f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.289+0000 7fc5e09f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52456/0 (socket says 172.21.15.29:52456) 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 -- 172.21.15.29:0/1639883835 learned_addr learned my addr 172.21.15.29:0/1639883835 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5190420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 -- 172.21.15.29:0/1639883835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 msgr2=0x5652f5190420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5190420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 -- 172.21.15.29:0/1639883835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 msgr2=0x5652f518f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 -- 172.21.15.29:0/1639883835 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f50b2530 con 0x5652f4de31b0 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc001f70 tx=0x7fc5cc009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56ded94725d15d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5d27fc640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 0x7fc5c80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5c77fe640 1 -- 172.21.15.29:0/1639883835 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc012040 con 0x5652f4de31b0 2024-09-21T02:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5c77fe640 1 -- 172.21.15.29:0/1639883835 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc0075a0 con 0x5652f4de31b0 2024-09-21T02:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.290+0000 7fc5c77fe640 1 -- 172.21.15.29:0/1639883835 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0049d0 con 0x5652f4de31b0 2024-09-21T02:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5d27fc640 1 -- 172.21.15.29:0/2510845699 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5d80f7c60 con 0x7fc5dc1050e0 2024-09-21T02:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 0x7fc5c80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1639883835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 msgr2=0x5652f518f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 0x7fc5c80b8b70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40309d0 tx=0x7fc5d4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc001f70 tx=0x7fc5cc009e00 comp rx=0 tx=0).stop 2024-09-21T02:02:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1639883835 shutdown_connections 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5190420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.291+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1639883835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f518f260 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.292+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1639883835 >> 172.21.15.29:0/1639883835 conn(0x5652f4dd7300 msgr2=0x5652f51868e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.292+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1639883835 shutdown_connections 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.292+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1639883835 wait complete. 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.292+0000 7fc5ea3bf740 1 -- start start 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.292+0000 7fc5ea3bf740 1 Processor -- start 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5199130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f5199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5652f5024ab0 con 0x5652f4de31b0 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652f5092730 con 0x5652f4dba2f0 2024-09-21T02:02:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5652f5190f40 con 0x5652f4dc24d0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5ea3bf740 10 client.0 _ll_register_callbacks cb 0x5652f50e66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52462/0 (socket says 172.21.15.29:52462) 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5e11f1640 1 -- 172.21.15.29:0/3694644426 learned_addr learned my addr 172.21.15.29:0/3694644426 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f5199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.293+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 -- 172.21.15.29:0/3694644426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 msgr2=0x5652f5199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 -- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 msgr2=0x5652f5198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 -- 172.21.15.29:0/3694644426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f4e4e790 con 0x5652f4de31b0 2024-09-21T02:02:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.294+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f5199ca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801a620 tx=0x7fc5d80bd630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6baf2231c26800 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.295+0000 7fc5c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:02:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.295+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8031040 con 0x5652f4de31b0 2024-09-21T02:02:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.295+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5d801ab00 con 0x5652f4de31b0 2024-09-21T02:02:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.295+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc0075a0 con 0x5652f4de31b0 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.295+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc5d80a91a0 con 0x5652f4de31b0 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.296+0000 7fc5ea3bf740 10 client.18477 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.296+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5652f4ec7620 con 0x5652f4de31b0 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc0075a0 con 0x5652f4de31b0 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d801f040 con 0x5652f4de31b0 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 10 client.18477 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 20 client.18477 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4da1d80 0x5652f51a6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 4 client.18477 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f50ff640 con 0x5652f4da1d80 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.297+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4da1d80 0x5652f51a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.298+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4da1d80 0x5652f51a6310 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b5945db96c1aeb server_cookie=83bed37d3b646bd0 in_seq=0 out_seq=0 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.298+0000 7fc5c4ff9640 10 client.18477 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/3694644426 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652f50ff640 con 0x5652f4da1d80 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5c4ff9640 10 client.18477 handle_command_reply: tid=0 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 client.18477 shutdown 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 msgr2=0x5652f5199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f5199ca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801a620 tx=0x7fc5d80bd630 comp rx=0 tx=0).stop 2024-09-21T02:02:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 shutdown_connections 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4da1d80 0x5652f51a6310 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f5199ca0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f5199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/3694644426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dba2f0 0x5652f5198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 >> 172.21.15.29:0/3694644426 conn(0x5652f4dd7300 msgr2=0x5652f5184ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5c4ff9640 0 client.18477 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 shutdown_connections 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.299+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/3694644426 wait complete. 2024-09-21T02:02:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.300+0000 7fc5ea3bf740 20 client.18477 trim_cache size 0 max 16384 2024-09-21T02:02:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 Processor -- start 2024-09-21T02:02:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 -- start start 2024-09-21T02:02:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 0x5652f5186220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f51815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 0x5652f5186220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 0x5652f5186220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:60550/0 (socket says 172.21.15.29:60550) 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5e09f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5e09f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:52476/0 (socket says 172.21.15.29:52476) 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000e4a0 con 0x5652f4de31b0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000ffe0 con 0x5652f4dc24d0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.320+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000eb70 con 0x5652f4dba2f0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e11f1640 1 -- 172.21.15.29:0/429174746 learned_addr learned my addr 172.21.15.29:0/429174746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f51815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 -- 172.21.15.29:0/429174746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 msgr2=0x5652f5186220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 0x5652f5186220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 -- 172.21.15.29:0/429174746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 msgr2=0x5652f51815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f51815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 -- 172.21.15.29:0/429174746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f519a860 con 0x5652f4dc24d0 2024-09-21T02:02:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.321+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc001890 tx=0x7fc5cc0015e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351436ce2fe43cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.322+0000 7fc5c6ffd640 1 -- 172.21.15.29:0/429174746 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc001020 con 0x5652f4dc24d0 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.322+0000 7fc5c6ffd640 1 -- 172.21.15.29:0/429174746 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc0075a0 con 0x5652f4dc24d0 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.322+0000 7fc5c6ffd640 1 -- 172.21.15.29:0/429174746 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc008c90 con 0x5652f4dc24d0 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.322+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/429174746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 msgr2=0x5652f5197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.322+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc001890 tx=0x7fc5cc0015e0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/429174746 shutdown_connections 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4de31b0 0x5652f51815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4dc24d0 0x5652f5197c90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/429174746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dba2f0 0x5652f5186220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/429174746 >> 172.21.15.29:0/429174746 conn(0x5652f4dd7300 msgr2=0x5652f51901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/429174746 shutdown_connections 2024-09-21T02:02:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/429174746 wait complete. 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 -- start start 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.323+0000 7fc5ea3bf740 1 Processor -- start 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f518e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f519ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000f910 con 0x5652f4dba2f0 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000f240 con 0x5652f4de31b0 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c000c480 con 0x5652f4dc24d0 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5ea3bf740 10 client.0 _ll_register_callbacks cb 0x7fc5d8004760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e11f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48450/0 (socket says 172.21.15.29:48450) 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e11f1640 1 -- 172.21.15.29:0/1542088487 learned_addr learned my addr 172.21.15.29:0/1542088487 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f519ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e09f0640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f518e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e19f2640 1 -- 172.21.15.29:0/1542088487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 msgr2=0x5652f518e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f518e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e19f2640 1 -- 172.21.15.29:0/1542088487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 msgr2=0x5652f518e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.324+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5e19f2640 1 -- 172.21.15.29:0/1542088487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f519b4d0 con 0x5652f4de31b0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5e19f2640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f519ad70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80bb070 tx=0x7fc5d80bd430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec38b9c5cf79c9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8031070 con 0x5652f4de31b0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fc5d80bcdd0 con 0x5652f4de31b0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.325+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc0075a0 con 0x5652f4de31b0 2024-09-21T02:02:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.326+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc5d80c7820 con 0x5652f4de31b0 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.326+0000 7fc5ea3bf740 10 client.28289 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.326+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5652f51a0830 con 0x5652f4de31b0 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.326+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc0075a0 con 0x5652f4de31b0 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.326+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d80c3a70 con 0x5652f4de31b0 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.327+0000 7fc5ea3bf740 10 client.28289 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.327+0000 7fc5ea3bf740 10 client.28289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.327+0000 7fc5ea3bf740 20 client.28289 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.328+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4d771c0 0x5652f52175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.328+0000 7fc5ea3bf740 4 client.28289 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:02:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.328+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652f50fb0f0 con 0x5652f4d771c0 2024-09-21T02:02:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.328+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4d771c0 0x5652f52175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.328+0000 7fc5e11f1640 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4d771c0 0x5652f52175e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3cb4f37a26b091 server_cookie=e7434940c7f2ed7f in_seq=0 out_seq=0 2024-09-21T02:02:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.329+0000 7fc5c4ff9640 10 client.28289 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.336+0000 7fc5c4ff9640 1 -- 172.21.15.29:0/1542088487 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652f50fb0f0 con 0x5652f4d771c0 2024-09-21T02:02:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.336+0000 7fc5c4ff9640 10 client.28289 handle_command_reply: tid=0 2024-09-21T02:02:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 client.28289 shutdown 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 msgr2=0x5652f519ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f519ad70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80bb070 tx=0x7fc5d80bd430 comp rx=0 tx=0).stop 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 shutdown_connections 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5652f4d771c0 0x5652f52175e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652f4de31b0 0x5652f519ad70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652f4dc24d0 0x5652f518e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5c4ff9640 0 client.28289 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 --2- 172.21.15.29:0/1542088487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5652f4dba2f0 0x5652f518e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 >> 172.21.15.29:0/1542088487 conn(0x5652f4dd7300 msgr2=0x5652f5186220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 shutdown_connections 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.337+0000 7fc5ea3bf740 1 -- 172.21.15.29:0/1542088487 wait complete. 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.338+0000 7fc5ea3bf740 20 client.28289 trim_cache size 0 max 16384 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:02:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 msgr2=0x7fc5c80b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 0x7fc5c80b8b70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40309d0 tx=0x7fc5d4000f40 comp rx=0 tx=0).stop 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 msgr2=0x7fc5dc192a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801e920 tx=0x7fc5d801ede0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 shutdown_connections 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fc5c80b6660 0x7fc5c80b8b70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5dc1050e0 0x7fc5dc192a00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc104740 0x7fc5dc191e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 --2- 172.21.15.29:0/2510845699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5dc1036c0 0x7fc5dc191810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.341+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 >> 172.21.15.29:0/2510845699 conn(0x7fc5dc0ff020 msgr2=0x7fc5dc0722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.342+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 shutdown_connections 2024-09-21T02:02:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:02:47.342+0000 7fc5e3455640 1 -- 172.21.15.29:0/2510845699 wait complete. 2024-09-21T02:02:47.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 850.0 s, 1248.2 tps, lat 79.078 ms stddev 82.813 2024-09-21T02:02:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:47 smithi066 ceph-mon[35001]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-21T02:02:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:48.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:02:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:47 smithi029 ceph-mon[31228]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-21T02:02:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:48.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:02:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:47 smithi190 ceph-mon[34368]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-21T02:02:48.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:48.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:48.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:48.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:02:50.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:49 smithi066 ceph-mon[35001]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-09-21T02:02:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:49 smithi029 ceph-mon[31228]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-09-21T02:02:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:49 smithi190 ceph-mon[34368]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-09-21T02:02:52.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:51 smithi066 ceph-mon[35001]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-09-21T02:02:52.096 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:51 smithi029 ceph-mon[31228]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-09-21T02:02:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:51 smithi190 ceph-mon[34368]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-09-21T02:02:52.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 855.0 s, 1565.2 tps, lat 64.016 ms stddev 74.286 2024-09-21T02:02:53.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:02:53.567 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:54.024 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:53 smithi029 ceph-mon[31228]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T02:02:54.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:53 smithi066 ceph-mon[35001]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T02:02:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:53 smithi190 ceph-mon[34368]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T02:02:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 -- 172.21.15.29:0/4045666550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 msgr2=0x7fac081098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 --2- 172.21.15.29:0/4045666550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac081098d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009900 tx=0x7fabf802a410 comp rx=0 tx=0).stop 2024-09-21T02:02:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 -- 172.21.15.29:0/4045666550 shutdown_connections 2024-09-21T02:02:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 --2- 172.21.15.29:0/4045666550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac08110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 --2- 172.21.15.29:0/4045666550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac081098d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 --2- 172.21.15.29:0/4045666550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 0x7fac08103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.150+0000 7fac0f49b640 1 -- 172.21.15.29:0/4045666550 >> 172.21.15.29:0/4045666550 conn(0x7fac080ff610 msgr2=0x7fac08101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.151+0000 7fac0f49b640 1 -- 172.21.15.29:0/4045666550 shutdown_connections 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.151+0000 7fac0f49b640 1 -- 172.21.15.29:0/4045666550 wait complete. 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 Processor -- start 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 -- start start 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 0x7fac0819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac0819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac0819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113ad0 con 0x7fac081094c0 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113950 con 0x7fac08109e10 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0f49b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113c50 con 0x7fac08103b80 2024-09-21T02:02:54.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.152+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac0819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac0819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56646/0 (socket says 172.21.15.29:56646) 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0ca36640 1 -- 172.21.15.29:0/736185362 learned_addr learned my addr 172.21.15.29:0/736185362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0d237640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 0x7fac0819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac0819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 -- 172.21.15.29:0/736185362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 msgr2=0x7fac0819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 0x7fac0819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 -- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 msgr2=0x7fac0819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac0819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 -- 172.21.15.29:0/736185362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac081a0430 con 0x7fac08109e10 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.153+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac0819fd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ec50 tx=0x7fac0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.154+0000 7fabfe7fc640 1 -- 172.21.15.29:0/736185362 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00003b70 con 0x7fac08109e10 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.154+0000 7fabfe7fc640 1 -- 172.21.15.29:0/736185362 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac00003d10 con 0x7fac08109e10 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.154+0000 7fabfe7fc640 1 -- 172.21.15.29:0/736185362 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac000107f0 con 0x7fac08109e10 2024-09-21T02:02:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.154+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac081a0720 con 0x7fac08109e10 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.154+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac080755f0 con 0x7fac08109e10 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.157+0000 7fac15a0f740 1 Processor -- start 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.157+0000 7fabfe7fc640 1 -- 172.21.15.29:0/736185362 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fac00004020 con 0x7fac08109e10 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.157+0000 7fac15a0f740 1 -- start start 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587129b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a58712a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a58709fe50 con 0x55a586d61390 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a58708f280 con 0x55a586d591b0 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fac15a0f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a587093090 con 0x55a586d82070 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.158+0000 7fabfe7fc640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 0x7fabf40c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fabfe7fc640 1 -- 172.21.15.29:0/736185362 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac000e7940 con 0x7fac08109e10 2024-09-21T02:02:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38580/0 (socket says 172.21.15.29:38580) 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 -- 172.21.15.29:0/326644877 learned_addr learned my addr 172.21.15.29:0/326644877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a58712a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 -- 172.21.15.29:0/326644877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 msgr2=0x55a587129b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587129b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 -- 172.21.15.29:0/326644877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 msgr2=0x55a58712a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.159+0000 7fac0ca36640 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a58712a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.160+0000 7fac0ca36640 1 -- 172.21.15.29:0/326644877 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a586ded650 con 0x55a586d82070 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.160+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 0x7fabf40c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.160+0000 7fac0ca36640 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf8005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7636d7a23c343dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fabf37fe640 1 -- 172.21.15.29:0/326644877 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8044070 con 0x55a586d82070 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fabf37fe640 1 -- 172.21.15.29:0/326644877 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf80316a0 con 0x55a586d82070 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fabf37fe640 1 -- 172.21.15.29:0/326644877 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8035030 con 0x55a586d82070 2024-09-21T02:02:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fac15a0f740 1 -- 172.21.15.29:0/326644877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 msgr2=0x55a587129650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fac15a0f740 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf8005db0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.161+0000 7fac0da38640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 0x7fabf40c12b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac0000f010 tx=0x7fac00004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 -- 172.21.15.29:0/326644877 shutdown_connections 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a58712a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587129b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 --2- 172.21.15.29:0/326644877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587129650 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 -- 172.21.15.29:0/326644877 >> 172.21.15.29:0/326644877 conn(0x55a586d76460 msgr2=0x55a587127540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 -- 172.21.15.29:0/326644877 shutdown_connections 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 -- 172.21.15.29:0/326644877 wait complete. 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 -- start start 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.162+0000 7fac15a0f740 1 Processor -- start 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 0x55a587134d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d61390 0x55a587135290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587141110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a586ef0700 con 0x55a586d591b0 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a58702a110 con 0x55a586d61390 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a587137e20 con 0x55a586d82070 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac15a0f740 10 client.0 _ll_register_callbacks cb 0x55a587043990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 0x55a587134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 0x55a587134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56666/0 (socket says 172.21.15.29:56666) 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 -- 172.21.15.29:0/3663763463 learned_addr learned my addr 172.21.15.29:0/3663763463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0d237640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d61390 0x55a587135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 -- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d61390 msgr2=0x55a587135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d61390 0x55a587135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 -- 172.21.15.29:0/3663763463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 msgr2=0x55a587134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 0x55a587134d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.163+0000 7fac0ca36640 1 -- 172.21.15.29:0/3663763463 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a587057c40 con 0x55a586d82070 2024-09-21T02:02:54.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac0ca36640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587141110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabf8005f70 tx=0x7fabf8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1de2bd09aa71f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8035040 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf80316a0 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fabf8004020 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fabf8044040 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 10 client.28273 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a587143b90 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf80316a0 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fabf80035f0 con 0x55a586d82070 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 10 client.28273 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 10 client.28273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 20 client.28273 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d409f0 0x55a587146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 4 client.28273 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.164+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5870a1da0 con 0x55a586d409f0 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.165+0000 7fac0d237640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d409f0 0x55a587146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.165+0000 7fac0d237640 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d409f0 0x55a587146ef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ec0b651e12b4a8 server_cookie=e327d1c209dca338 in_seq=0 out_seq=0 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.165+0000 7fabf0ff9640 10 client.28273 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fabf0ff9640 1 -- 172.21.15.29:0/3663763463 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5870a1da0 con 0x55a586d409f0 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fabf0ff9640 10 client.28273 handle_command_reply: tid=0 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 client.28273 shutdown 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 msgr2=0x55a587141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587141110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabf8005f70 tx=0x7fabf8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 shutdown_connections 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d409f0 0x55a587146ef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fabf0ff9640 0 client.28273 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d82070 0x55a587141110 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d61390 0x55a587135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 --2- 172.21.15.29:0/3663763463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d591b0 0x55a587134d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 >> 172.21.15.29:0/3663763463 conn(0x55a586d76460 msgr2=0x55a58712c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 shutdown_connections 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 1 -- 172.21.15.29:0/3663763463 wait complete. 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.166+0000 7fac15a0f740 20 client.28273 trim_cache size 0 max 16384 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.192+0000 7fac15a0f740 1 Processor -- start 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.192+0000 7fac15a0f740 1 -- start start 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.192+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d409f0 0x55a58712b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.192+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.192+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac15a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac081ad140 con 0x55a586d61390 2024-09-21T02:02:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac15a0f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac081aec80 con 0x55a586d409f0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac15a0f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac081ad810 con 0x55a586d591b0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38608/0 (socket says 172.21.15.29:38608) 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 -- 172.21.15.29:0/1373723243 learned_addr learned my addr 172.21.15.29:0/1373723243 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56668/0 (socket says 172.21.15.29:56668) 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0ca36640 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d409f0 0x55a58712b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 -- 172.21.15.29:0/1373723243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d409f0 msgr2=0x55a58712b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d409f0 0x55a58712b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 -- 172.21.15.29:0/1373723243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 msgr2=0x55a587126b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 -- 172.21.15.29:0/1373723243 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a587127e30 con 0x55a586d591b0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0ca36640 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fac0d237640 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac04009ad0 tx=0x7fac04004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df93024adaae8080 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.193+0000 7fabf2ffd640 1 -- 172.21.15.29:0/1373723243 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04003d50 con 0x55a586d591b0 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fabf2ffd640 1 -- 172.21.15.29:0/1373723243 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac040054e0 con 0x55a586d591b0 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fabf2ffd640 1 -- 172.21.15.29:0/1373723243 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04012450 con 0x55a586d591b0 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 -- 172.21.15.29:0/1373723243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 msgr2=0x55a587136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac04009ad0 tx=0x7fac04004410 comp rx=0 tx=0).stop 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 -- 172.21.15.29:0/1373723243 shutdown_connections 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d61390 0x55a587126b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d591b0 0x55a587136980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 --2- 172.21.15.29:0/1373723243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d409f0 0x55a58712b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.194+0000 7fac15a0f740 1 -- 172.21.15.29:0/1373723243 >> 172.21.15.29:0/1373723243 conn(0x55a586d76460 msgr2=0x55a587136270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 -- 172.21.15.29:0/1373723243 shutdown_connections 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 -- 172.21.15.29:0/1373723243 wait complete. 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 -- start start 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 Processor -- start 2024-09-21T02:02:54.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 0x55a58714a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.195+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac15a0f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d61390 0x55a587148090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac15a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac081ae5b0 con 0x55a586d409f0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac15a0f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac081adee0 con 0x55a586d591b0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac15a0f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac081ab120 con 0x55a586d61390 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac15a0f740 10 client.0 _ll_register_callbacks cb 0x7fabf8031850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0ca36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 0x55a58714a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50534/0 (socket says 172.21.15.29:50534) 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 -- 172.21.15.29:0/215154232 learned_addr learned my addr 172.21.15.29:0/215154232 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0ca36640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d61390 0x55a587148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 -- 172.21.15.29:0/215154232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d61390 msgr2=0x55a587148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d61390 0x55a587148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 -- 172.21.15.29:0/215154232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 msgr2=0x55a58714a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 0x55a58714a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0d237640 1 -- 172.21.15.29:0/215154232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5871487d0 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.196+0000 7fac0ca36640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 0x55a58714a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.197+0000 7fac0d237640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fac04004b40 tx=0x7fac04009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a61008e4359990 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.197+0000 7fabf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.197+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04029070 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.197+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fac04003b50 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.198+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac040054e0 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.199+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fac04037050 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.199+0000 7fac15a0f740 10 client.28304 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.199+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a5871b1530 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.208+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac04003d50 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.210+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fac04004ba0 con 0x55a586d591b0 2024-09-21T02:02:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.210+0000 7fac15a0f740 10 client.28304 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.210+0000 7fac15a0f740 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.211+0000 7fac15a0f740 20 client.28304 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.211+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d82070 0x55a5871b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.211+0000 7fac15a0f740 4 client.28304 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.211+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fac040054e0 con 0x55a586d82070 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.211+0000 7fac0ca36640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d82070 0x55a5871b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.219+0000 7fabf0ff9640 10 client.28304 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.219+0000 7fac0ca36640 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d82070 0x55a5871b3b40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12c7bb27a2adc06 server_cookie=7ef32f1b23687aea in_seq=0 out_seq=0 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.220+0000 7fabf0ff9640 1 -- 172.21.15.29:0/215154232 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x7fac040054e0 con 0x55a586d82070 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.220+0000 7fabf0ff9640 10 client.28304 handle_command_reply: tid=0 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.220+0000 7fac15a0f740 1 client.28304 shutdown 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.220+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 msgr2=0x55a587147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.220+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fac04004b40 tx=0x7fac04009940 comp rx=0 tx=0).stop 2024-09-21T02:02:54.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 shutdown_connections 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a586d82070 0x55a5871b3b40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a586d61390 0x55a587148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a586d591b0 0x55a587147b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 --2- 172.21.15.29:0/215154232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a586d409f0 0x55a58714a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 >> 172.21.15.29:0/215154232 conn(0x55a586d76460 msgr2=0x55a58712ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fabf0ff9640 0 client.28304 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T02:02:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "c48dc890-9260-4f8f-b08e-1fd9d06fa732": { 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:02:20.247270+0000", 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:02:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:02:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:02:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:02:54.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 shutdown_connections 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 1 -- 172.21.15.29:0/215154232 wait complete. 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.223+0000 7fac15a0f740 20 client.28304 trim_cache size 0 max 16384 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 msgr2=0x7fabf40c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 0x7fabf40c12b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac0000f010 tx=0x7fac00004a40 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 msgr2=0x7fac0819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac0819fd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ec50 tx=0x7fac0000c6a0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 shutdown_connections 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fabf40beda0 0x7fabf40c12b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac08109e10 0x7fac0819fd00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac081094c0 0x7fac0819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 --2- 172.21.15.29:0/736185362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac08103b80 0x7fac0819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 >> 172.21.15.29:0/736185362 conn(0x7fac080ff610 msgr2=0x7fac08101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 shutdown_connections 2024-09-21T02:02:54.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:54.232+0000 7fac0f49b640 1 -- 172.21.15.29:0/736185362 wait complete. 2024-09-21T02:02:55.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:02:55.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:56.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:55 smithi190 ceph-mon[34368]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-21T02:02:56.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:55 smithi066 ceph-mon[35001]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-21T02:02:56.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:55 smithi029 ceph-mon[31228]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-21T02:02:56.194 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.790+0000 7f93463f0640 1 -- 172.21.15.29:0/3196855326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93401094c0 msgr2=0x7f93401098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.790+0000 7f93463f0640 1 --2- 172.21.15.29:0/3196855326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93401094c0 0x7f93401098d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9338009be0 tx=0x7f933802a710 comp rx=0 tx=0).stop 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 -- 172.21.15.29:0/3196855326 shutdown_connections 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 --2- 172.21.15.29:0/3196855326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9340109e10 0x7f9340110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 --2- 172.21.15.29:0/3196855326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93401094c0 0x7f93401098d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 --2- 172.21.15.29:0/3196855326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9340103b80 0x7f9340103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 -- 172.21.15.29:0/3196855326 >> 172.21.15.29:0/3196855326 conn(0x7f93400ff610 msgr2=0x7f9340101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 -- 172.21.15.29:0/3196855326 shutdown_connections 2024-09-21T02:02:56.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.791+0000 7f93463f0640 1 -- 172.21.15.29:0/3196855326 wait complete. 2024-09-21T02:02:56.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.792+0000 7f93463f0640 1 Processor -- start 2024-09-21T02:02:56.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.792+0000 7f93463f0640 1 -- start start 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.792+0000 7f93463f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9340103b80 0x7f9340118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f93463f0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f93463f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9340109e10 0x7f93401132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f93463f0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9340111740 con 0x7f93401094c0 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f93463f0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93401115c0 con 0x7f9340103b80 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f93463f0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93401118c0 con 0x7f9340109e10 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56700/0 (socket says 172.21.15.29:56700) 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.793+0000 7f933f7fe640 1 -- 172.21.15.29:0/2528150325 learned_addr learned my addr 172.21.15.29:0/2528150325 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 -- 172.21.15.29:0/2528150325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9340109e10 msgr2=0x7f93401132e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9340109e10 0x7f93401132e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 -- 172.21.15.29:0/2528150325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9340103b80 msgr2=0x7f9340118ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9340103b80 0x7f9340118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 -- 172.21.15.29:0/2528150325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9340113b00 con 0x7f93401094c0 2024-09-21T02:02:56.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.794+0000 7f933f7fe640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f933802ad00 tx=0x7f9338031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:56.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.796+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338041070 con 0x7f93401094c0 2024-09-21T02:02:56.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.796+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933803caf0 con 0x7f93401094c0 2024-09-21T02:02:56.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.796+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338036040 con 0x7f93401094c0 2024-09-21T02:02:56.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.799+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9340113d90 con 0x7f93401094c0 2024-09-21T02:02:56.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.799+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9340072fe0 con 0x7f93401094c0 2024-09-21T02:02:56.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.801+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9340103f90 con 0x7f93401094c0 2024-09-21T02:02:56.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.807+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9338044070 con 0x7f93401094c0 2024-09-21T02:02:56.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.808+0000 7f933d7fa640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 0x7f93280b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.812+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f93380309c0 con 0x7f93401094c0 2024-09-21T02:02:56.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.813+0000 7f933ffff640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 0x7f93280b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.814+0000 7f933ffff640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 0x7f93280b8bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9330005fd0 tx=0x7f9330005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:56.819+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93380cee10 con 0x7f93401094c0 2024-09-21T02:02:57.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.129+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9340113f60 con 0x7f93401094c0 2024-09-21T02:02:57.132 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:02:57.132 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":49,"btime":"2024-09-21T02:01:30:578913+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:01:30.250452+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T02:02:57.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.130+0000 7f933d7fa640 1 -- 172.21.15.29:0/2528150325 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11970 (secure 0 0 0) 0x7f93380d31f0 con 0x7f93401094c0 2024-09-21T02:02:57.133 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 49 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 msgr2=0x7f93280b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 0x7f93280b8bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9330005fd0 tx=0x7f9330005850 comp rx=0 tx=0).stop 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 msgr2=0x7f9340119460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f933802ad00 tx=0x7f9338031670 comp rx=0 tx=0).stop 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 shutdown_connections 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f93280b66c0 0x7f93280b8bd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9340109e10 0x7f93401132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93401094c0 0x7f9340119460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 --2- 172.21.15.29:0/2528150325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9340103b80 0x7f9340118ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 >> 172.21.15.29:0/2528150325 conn(0x7f93400ff610 msgr2=0x7f9340106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.133+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 shutdown_connections 2024-09-21T02:02:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:57.134+0000 7f93463f0640 1 -- 172.21.15.29:0/2528150325 wait complete. 2024-09-21T02:02:57.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 860.0 s, 1202.4 tps, lat 84.055 ms stddev 93.852 2024-09-21T02:02:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:57 smithi066 ceph-mon[35001]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-21T02:02:58.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:57 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2528150325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:57 smithi029 ceph-mon[31228]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-21T02:02:58.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:57 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2528150325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:58.136 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:02:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:57 smithi190 ceph-mon[34368]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-21T02:02:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:57 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2528150325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:58.533 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.144+0000 7fe3c9ffb640 1 -- 172.21.15.29:0/1091998534 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00035f0 con 0x7fe3cc11cc30 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1091998534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc11cc30 msgr2=0x7fe3cc11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1091998534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc11cc30 0x7fe3cc11f140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009900 tx=0x7fe3c002a410 comp rx=0 tx=0).stop 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1091998534 shutdown_connections 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1091998534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc11cc30 0x7fe3cc11f140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1091998534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1091998534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3cc111f40 0x7fe3cc10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.145+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1091998534 >> 172.21.15.29:0/1091998534 conn(0x7fe3cc070ec0 msgr2=0x7fe3cc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.146+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1091998534 shutdown_connections 2024-09-21T02:02:59.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.146+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1091998534 wait complete. 2024-09-21T02:02:59.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.147+0000 7fe3d15f4640 1 Processor -- start 2024-09-21T02:02:59.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.147+0000 7fe3d15f4640 1 -- start start 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc111f40 0x7fe3cc1a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3cc11cc30 0x7fe3cc1aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc122900 con 0x7fe3cc11cc30 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc122780 con 0x7fe3cc111f40 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3d15f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc122a80 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3caffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3caffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38640/0 (socket says 172.21.15.29:38640) 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3caffd640 1 -- 172.21.15.29:0/1162002195 learned_addr learned my addr 172.21.15.29:0/1162002195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:59.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.148+0000 7fe3ca7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc111f40 0x7fe3cc1a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 -- 172.21.15.29:0/1162002195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc111f40 msgr2=0x7fe3cc1a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc111f40 0x7fe3cc1a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 -- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3cc11cc30 msgr2=0x7fe3cc1aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3cc11cc30 0x7fe3cc1aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 -- 172.21.15.29:0/1162002195 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc1af180 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3caffd640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d770 tx=0x7fe3bc00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3b3fff640 1 -- 172.21.15.29:0/1162002195 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc017070 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3b3fff640 1 -- 172.21.15.29:0/1162002195 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3bc0054d0 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.149+0000 7fe3b3fff640 1 -- 172.21.15.29:0/1162002195 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc004c60 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.150+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc1af470 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.150+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc1b5bd0 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 Processor -- start 2024-09-21T02:02:59.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 -- start start 2024-09-21T02:02:59.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3b3fff640 1 -- 172.21.15.29:0/1162002195 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe3bc004e00 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 0x55edf5ba9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55edf5b1eea0 con 0x55edf5801070 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55edf5b0e2d0 con 0x55edf57e0390 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.154+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55edf5b120e0 con 0x55edf57d81b0 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56734/0 (socket says 172.21.15.29:56734) 2024-09-21T02:02:59.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/3897813069 learned_addr learned my addr 172.21.15.29:0/3897813069 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3caffd640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 0x55edf5ba9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/3897813069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 msgr2=0x55edf5ba8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/3897813069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 msgr2=0x55edf5ba9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 0x55edf5ba9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/3897813069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf586c650 con 0x55edf5801070 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.155+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 0x55edf5ba9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.156+0000 7fe3b3fff640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 0x7fe3b40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.156+0000 7fe3caffd640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 0x7fe3b40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.156+0000 7fe3b3fff640 1 -- 172.21.15.29:0/1162002195 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe3bc004a00 con 0x7fe3cc10a9e0 2024-09-21T02:02:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.156+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a930 tx=0x7fe3c0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f629b5e6813c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.157+0000 7fe3b17fa640 1 -- 172.21.15.29:0/3897813069 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0040030 con 0x55edf5801070 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.157+0000 7fe3caffd640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 0x7fe3b40b8c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55edf5ba7320 tx=0x7fe3bc094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.157+0000 7fe3b17fa640 1 -- 172.21.15.29:0/3897813069 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c0004780 con 0x55edf5801070 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.157+0000 7fe3b17fa640 1 -- 172.21.15.29:0/3897813069 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0040030 con 0x55edf5801070 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- 172.21.15.29:0/3897813069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 msgr2=0x55edf5ba86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a930 tx=0x7fe3c0003680 comp rx=0 tx=0).stop 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- 172.21.15.29:0/3897813069 shutdown_connections 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57e0390 0x55edf5ba9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/3897813069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5ba86a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- 172.21.15.29:0/3897813069 >> 172.21.15.29:0/3897813069 conn(0x55edf57f5460 msgr2=0x55edf5ba6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- 172.21.15.29:0/3897813069 shutdown_connections 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- 172.21.15.29:0/3897813069 wait complete. 2024-09-21T02:02:59.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 -- start start 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.158+0000 7fe3d7b68740 1 Processor -- start 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57e0390 0x55edf5bb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5bc0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55edf596f700 con 0x55edf5801070 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55edf5aa9160 con 0x55edf57d81b0 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55edf5bb6e20 con 0x55edf57e0390 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3d7b68740 10 client.0 _ll_register_callbacks cb 0x55edf5ac29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5bc0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5bc0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56738/0 (socket says 172.21.15.29:56738) 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1649420195 learned_addr learned my addr 172.21.15.29:0/1649420195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57e0390 0x55edf5bb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1649420195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57e0390 msgr2=0x55edf5bb4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57e0390 0x55edf5bb4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1649420195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 msgr2=0x55edf5bc0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5bc0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.159+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1649420195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf5ad6c90 con 0x55edf57d81b0 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb3cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005f30 tx=0x7fe3c0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa5216cd3d8eb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0040030 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c0004780 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe3c0041070 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe3c0031500 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3d7b68740 10 client.28319 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55edf5bc2ad0 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.160+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c0004780 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c00566b0 con 0x55edf57d81b0 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 10 client.28319 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 10 client.28319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 20 client.28319 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57bf9f0 0x55edf5bc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 4 client.28319 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:02:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf5b20df0 con 0x55edf57bf9f0 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.161+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57bf9f0 0x55edf5bc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.162+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57bf9f0 0x55edf5bc6020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fea7ee108b615b44 server_cookie=40bcd565d7c9e8d7 in_seq=0 out_seq=0 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.162+0000 7fe3aeff5640 10 client.28319 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.162+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1649420195 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55edf5b20df0 con 0x55edf57bf9f0 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.162+0000 7fe3aeff5640 10 client.28319 handle_command_reply: tid=0 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.162+0000 7fe3d7b68740 1 client.28319 shutdown 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 msgr2=0x55edf5bb3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb3cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005f30 tx=0x7fe3c0005fd0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 shutdown_connections 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57bf9f0 0x55edf5bc6020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf5801070 0x55edf5bc0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3aeff5640 0 client.28319 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57e0390 0x55edf5bb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1649420195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb3cf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 >> 172.21.15.29:0/1649420195 conn(0x55edf57f5460 msgr2=0x55edf5bab880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 shutdown_connections 2024-09-21T02:02:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1649420195 wait complete. 2024-09-21T02:02:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.163+0000 7fe3d7b68740 20 client.28319 trim_cache size 0 max 16384 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 Processor -- start 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 -- start start 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 0x55edf5ba5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1be180 con 0x55edf57f5460 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1bfcc0 con 0x55edf57bf9f0 2024-09-21T02:02:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.177+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1be850 con 0x55edf57d81b0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56746/0 (socket says 172.21.15.29:56746) 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1167994240 learned_addr learned my addr 172.21.15.29:0/1167994240 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 0x55edf5ba5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1167994240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 msgr2=0x55edf5ba6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1167994240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 msgr2=0x55edf5bb5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1167994240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf5bb4fb0 con 0x55edf57bf9f0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.178+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 0x55edf5ba5bc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55edf5bc6750 tx=0x7fe3c4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63a89c5216f70a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3b0ff9640 1 -- 172.21.15.29:0/1167994240 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4004240 con 0x55edf57bf9f0 2024-09-21T02:02:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3b0ff9640 1 -- 172.21.15.29:0/1167994240 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c4028070 con 0x55edf57bf9f0 2024-09-21T02:02:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3b0ff9640 1 -- 172.21.15.29:0/1167994240 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4023ab0 con 0x55edf57bf9f0 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1167994240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 msgr2=0x55edf5ba5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 0x55edf5ba5bc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55edf5bc6750 tx=0x7fe3c4009500 comp rx=0 tx=0).stop 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1167994240 shutdown_connections 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57d81b0 0x55edf5ba6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.179+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57bf9f0 0x55edf5ba5bc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1167994240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57f5460 0x55edf5bb5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1167994240 >> 172.21.15.29:0/1167994240 conn(0x55edf5baf640 msgr2=0x55edf5bb5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1167994240 shutdown_connections 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1167994240 wait complete. 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 -- start start 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 Processor -- start 2024-09-21T02:02:59.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 0x55edf5bbc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.180+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3d7b68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57f5460 0x55edf5bb83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1bf5f0 con 0x55edf57bf9f0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1bef20 con 0x55edf57d81b0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3d7b68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc1bc160 con 0x55edf57f5460 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3d7b68740 10 client.0 _ll_register_callbacks cb 0x7fe3c0004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 0x55edf5bbc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3cb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 0x55edf5bbc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56752/0 (socket says 172.21.15.29:56752) 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3cb7fe640 1 -- 172.21.15.29:0/1129431071 learned_addr learned my addr 172.21.15.29:0/1129431071 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57f5460 0x55edf5bb83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1129431071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57f5460 msgr2=0x55edf5bb83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57f5460 0x55edf5bb83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1129431071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 msgr2=0x55edf5bbc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:02:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 0x55edf5bbc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 -- 172.21.15.29:0/1129431071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf5bb8b30 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3ca7fc640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb7de0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55edf5ba6f90 tx=0x7fe3c4003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dbb14f133ccb6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.181+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4030070 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c4028070 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe3c40044d0 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe3c403c050 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 10 client.28325 fetch_fsmap learned FSMap version 49 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55edf5c309a0 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c4028070 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c4023400 con 0x55edf57d81b0 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 10 client.28325 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 10 client.28325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 20 client.28325 populate_metadata read hostname 'smithi029' 2024-09-21T02:02:59.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57e0390 0x55edf5c330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 4 client.28325 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.182+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edf5b1c8a0 con 0x55edf57e0390 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57e0390 0x55edf5c330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3cb7fe640 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57e0390 0x55edf5c330e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12c2862876e1d61a server_cookie=8d597429a24f2d07 in_seq=0 out_seq=0 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3aeff5640 10 client.28325 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3aeff5640 1 -- 172.21.15.29:0/1129431071 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edf5b1c8a0 con 0x55edf57e0390 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3aeff5640 10 client.28325 handle_command_reply: tid=0 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 client.28325 shutdown 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 msgr2=0x55edf5bb7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb7de0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55edf5ba6f90 tx=0x7fe3c4003ea0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 shutdown_connections 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55edf57e0390 0x55edf5c330e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edf57f5460 0x55edf5bb83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55edf57d81b0 0x55edf5bb7de0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 --2- 172.21.15.29:0/1129431071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55edf57bf9f0 0x55edf5bbc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 >> 172.21.15.29:0/1129431071 conn(0x55edf5baf640 msgr2=0x55edf5bb5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3aeff5640 0 client.28325 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 shutdown_connections 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.183+0000 7fe3d7b68740 1 -- 172.21.15.29:0/1129431071 wait complete. 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d7b68740 20 client.28325 trim_cache size 0 max 16384 2024-09-21T02:02:59.190 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 msgr2=0x7fe3b40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 0x7fe3b40b8c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55edf5ba7320 tx=0x7fe3bc094040 comp rx=0 tx=0).stop 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 msgr2=0x7fe3cc1a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d770 tx=0x7fe3bc00dc30 comp rx=0 tx=0).stop 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 shutdown_connections 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe3b40b6710 0x7fe3b40b8c20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3cc11cc30 0x7fe3cc1aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3cc111f40 0x7fe3cc1a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 --2- 172.21.15.29:0/1162002195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3cc10a9e0 0x7fe3cc1a9210 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 >> 172.21.15.29:0/1162002195 conn(0x7fe3cc070ec0 msgr2=0x7fe3cc071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 shutdown_connections 2024-09-21T02:02:59.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:02:59.184+0000 7fe3d15f4640 1 -- 172.21.15.29:0/1162002195 wait complete. 2024-09-21T02:03:00.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:02:59 smithi066 ceph-mon[35001]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-21T02:03:00.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:02:59 smithi029 ceph-mon[31228]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-21T02:03:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:02:59 smithi190 ceph-mon[34368]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-21T02:03:00.236 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:03:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 -- 172.21.15.29:0/4264530073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db0075c90 msgr2=0x7f6db00760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 --2- 172.21.15.29:0/4264530073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db0075c90 0x7f6db00760a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009860 tx=0x7f6da002a360 comp rx=0 tx=0).stop 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 -- 172.21.15.29:0/4264530073 shutdown_connections 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 --2- 172.21.15.29:0/4264530073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db010b6b0 0x7f6db010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 --2- 172.21.15.29:0/4264530073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db0075c90 0x7f6db00760a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 --2- 172.21.15.29:0/4264530073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0077090 0x7f6db0075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.480+0000 7f6db8415640 1 -- 172.21.15.29:0/4264530073 >> 172.21.15.29:0/4264530073 conn(0x7f6db00ff020 msgr2=0x7f6db01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- 172.21.15.29:0/4264530073 shutdown_connections 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- 172.21.15.29:0/4264530073 wait complete. 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 Processor -- start 2024-09-21T02:03:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- start start 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0075c90 0x7f6db019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 0x7f6db019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 0x7f6db019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0111510 con 0x7f6db0077090 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0111390 con 0x7f6db010b6b0 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.481+0000 7f6db8415640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0111690 con 0x7f6db0075c90 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 0x7f6db019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 0x7f6db019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56762/0 (socket says 172.21.15.29:56762) 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 0x7f6db019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db59b0640 1 -- 172.21.15.29:0/3334521565 learned_addr learned my addr 172.21.15.29:0/3334521565 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db61b1640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0075c90 0x7f6db019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 -- 172.21.15.29:0/3334521565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0075c90 msgr2=0x7f6db019c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0075c90 0x7f6db019c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 -- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 msgr2=0x7f6db019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 0x7f6db019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 -- 172.21.15.29:0/3334521565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db019db10 con 0x7f6db010b6b0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6db69b2640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 0x7f6db019d380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005fd0 tx=0x7f6da800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.482+0000 7f6da77fe640 1 -- 172.21.15.29:0/3334521565 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004d20 con 0x7f6db010b6b0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.483+0000 7f6da77fe640 1 -- 172.21.15.29:0/3334521565 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8003e20 con 0x7f6db010b6b0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.483+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db019de00 con 0x7f6db010b6b0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.483+0000 7f6da77fe640 1 -- 172.21.15.29:0/3334521565 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800b7d0 con 0x7f6db010b6b0 2024-09-21T02:03:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.483+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db01a4580 con 0x7f6db010b6b0 2024-09-21T02:03:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.486+0000 7f6dbf45f740 1 Processor -- start 2024-09-21T02:03:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.486+0000 7f6da77fe640 1 -- 172.21.15.29:0/3334521565 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6da801d020 con 0x7f6db010b6b0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 -- start start 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea55fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ea556c6f0 con 0x555ea52314d0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ea555bb20 con 0x555ea52292f0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555ea555f930 con 0x555ea52521b0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50634/0 (socket says 172.21.15.29:50634) 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 -- 172.21.15.29:0/2612516391 learned_addr learned my addr 172.21.15.29:0/2612516391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:38712/0 (socket says 172.21.15.29:38712) 2024-09-21T02:03:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db59b0640 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea55fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 -- 172.21.15.29:0/2612516391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 msgr2=0x555ea55f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.487+0000 7f6db61b1640 1 -- 172.21.15.29:0/2612516391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 msgr2=0x555ea55fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6db61b1640 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea55fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6db61b1640 1 -- 172.21.15.29:0/2612516391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea5521530 con 0x555ea52292f0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6db61b1640 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ad00 tx=0x7f6dac00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1fb5e2cf3e0cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6da4bfe640 1 -- 172.21.15.29:0/2612516391 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01d070 con 0x555ea52292f0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6da4bfe640 1 -- 172.21.15.29:0/2612516391 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac018af0 con 0x555ea52292f0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6da4bfe640 1 -- 172.21.15.29:0/2612516391 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac010b90 con 0x555ea52292f0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6da77fe640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 0x7f6d9c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6db59b0640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 0x7f6d9c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.488+0000 7f6da77fe640 1 -- 172.21.15.29:0/3334521565 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6da80e73c0 con 0x7f6db010b6b0 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 -- 172.21.15.29:0/2612516391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 msgr2=0x555ea55fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6db59b0640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 0x7f6d9c0b8aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005eb0 tx=0x7f6da00304f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ad00 tx=0x7f6dac00f790 comp rx=0 tx=0).stop 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 -- 172.21.15.29:0/2612516391 shutdown_connections 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea55fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea55fa4c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/2612516391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 -- 172.21.15.29:0/2612516391 >> 172.21.15.29:0/2612516391 conn(0x555ea5246300 msgr2=0x555ea55f58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 -- 172.21.15.29:0/2612516391 shutdown_connections 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.489+0000 7f6dbf45f740 1 -- 172.21.15.29:0/2612516391 wait complete. 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.490+0000 7f6dbf45f740 1 -- start start 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.490+0000 7f6dbf45f740 1 Processor -- start 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.492+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.492+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52314d0 0x555ea5603c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.492+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 0x555ea56047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.492+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db69b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 0x555ea56047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50646/0 (socket says 172.21.15.29:50646) 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db69b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 0x555ea56047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56790/0 (socket says 172.21.15.29:56790) 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 -- 172.21.15.29:0/1002103717 learned_addr learned my addr 172.21.15.29:0/1002103717 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ea5493ab0 con 0x555ea52521b0 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ea5501730 con 0x555ea52292f0 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555ea55fbb30 con 0x555ea52314d0 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 -- 172.21.15.29:0/1002103717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52314d0 msgr2=0x555ea5603c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52314d0 0x555ea5603c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 -- 172.21.15.29:0/1002103717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 msgr2=0x555ea56047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 0x555ea56047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea52bd790 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6db61b1640 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6dac018650 tx=0x7f6dac018680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b216862101286c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6dbf45f740 10 client.0 _ll_register_callbacks cb 0x555ea55556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.493+0000 7f6d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6d99ffb640 1 -- 172.21.15.29:0/1002103717 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01d070 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac018af0 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6d99ffb640 1 -- 172.21.15.29:0/1002103717 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6dac010a90 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6d99ffb640 1 -- 172.21.15.29:0/1002103717 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6dac010c90 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6dbf45f740 10 client.28337 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=49}) -- 0x555ea5336620 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6d99ffb640 1 -- 172.21.15.29:0/1002103717 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6dac005040 con 0x555ea52292f0 2024-09-21T02:03:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6dbf45f740 10 client.28337 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.494+0000 7f6dbf45f740 10 client.28337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6dbf45f740 20 client.28337 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea5210d80 0x555ea56107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6dbf45f740 4 client.28337 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6dac018af0 con 0x555ea5210d80 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6db59b0640 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea5210d80 0x555ea56107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6d99ffb640 10 client.28337 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.495+0000 7f6db59b0640 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea5210d80 0x555ea56107b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796e629a165f1351 server_cookie=ce29a6d9e86f2cfa in_seq=0 out_seq=0 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6d99ffb640 1 -- 172.21.15.29:0/1002103717 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6dac018af0 con 0x555ea5210d80 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6d99ffb640 10 client.28337 handle_command_reply: tid=0 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 client.28337 shutdown 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 msgr2=0x555ea56035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6dac018650 tx=0x7f6dac018680 comp rx=0 tx=0).stop 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 shutdown_connections 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea5210d80 0x555ea56107b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52521b0 0x555ea56047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52314d0 0x555ea5603c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6d99ffb640 0 client.28337 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1002103717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56035a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 >> 172.21.15.29:0/1002103717 conn(0x555ea5246300 msgr2=0x555ea55f3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 shutdown_connections 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.496+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1002103717 wait complete. 2024-09-21T02:03:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.497+0000 7f6dbf45f740 20 client.28337 trim_cache size 0 max 16384 2024-09-21T02:03:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.518+0000 7f6dbf45f740 1 Processor -- start 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.518+0000 7f6dbf45f740 1 -- start start 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea560b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01acf50 con 0x555ea52314d0 2024-09-21T02:03:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01aea90 con 0x555ea52292f0 2024-09-21T02:03:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ad620 con 0x555ea52521b0 2024-09-21T02:03:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea560b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50650/0 (socket says 172.21.15.29:50650) 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db61b1640 1 -- 172.21.15.29:0/1336906421 learned_addr learned my addr 172.21.15.29:0/1336906421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db69b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.519+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea560b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56800/0 (socket says 172.21.15.29:56800) 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 -- 172.21.15.29:0/1336906421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 msgr2=0x555ea56021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 -- 172.21.15.29:0/1336906421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 msgr2=0x555ea560b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea560b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 -- 172.21.15.29:0/1336906421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea55f0f50 con 0x555ea52521b0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6db69b2640 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f5e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6da80e5240 tx=0x7f6da80accd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fee93fab0b14e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6d9bfff640 1 -- 172.21.15.29:0/1336906421 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da801a030 con 0x555ea52521b0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6d9bfff640 1 -- 172.21.15.29:0/1336906421 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da80adc60 con 0x555ea52521b0 2024-09-21T02:03:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.520+0000 7f6d9bfff640 1 -- 172.21.15.29:0/1336906421 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a8eb0 con 0x555ea52521b0 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1336906421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 msgr2=0x555ea55f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f5e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6da80e5240 tx=0x7f6da80accd0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1336906421 shutdown_connections 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea55f5e30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52314d0 0x555ea560b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/1336906421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52292f0 0x555ea56021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1336906421 >> 172.21.15.29:0/1336906421 conn(0x555ea5246300 msgr2=0x555ea5601ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1336906421 shutdown_connections 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- 172.21.15.29:0/1336906421 wait complete. 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- start start 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 Processor -- start 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 0x555ea5609f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea560a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ae3c0 con 0x555ea52292f0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01adcf0 con 0x555ea52314d0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01aaf30 con 0x555ea52521b0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6dbf45f740 10 client.0 _ll_register_callbacks cb 0x7f6dac01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6db59b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 0x555ea5609f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6db61b1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56802/0 (socket says 172.21.15.29:56802) 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6db61b1640 1 -- 172.21.15.29:0/705892190 learned_addr learned my addr 172.21.15.29:0/705892190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.522+0000 7f6db69b2640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea560a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 -- 172.21.15.29:0/705892190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 msgr2=0x555ea560a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea560a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 -- 172.21.15.29:0/705892190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 msgr2=0x555ea560b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 -- 172.21.15.29:0/705892190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea560ab80 con 0x555ea52314d0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db61b1640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6db59b0640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 0x555ea5609f50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6da0030380 tx=0x7f6da00056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809c2a6416d7bc5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0037040 con 0x555ea52314d0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da80adc60 con 0x555ea52314d0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f6da0003dd0 con 0x555ea52314d0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6da0032440 con 0x555ea52314d0 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6dbf45f740 10 client.28343 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.523+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555ea56839c0 con 0x555ea52314d0 2024-09-21T02:03:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.524+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da80adc60 con 0x555ea52314d0 2024-09-21T02:03:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6da004d040 con 0x555ea52314d0 2024-09-21T02:03:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 10 client.28343 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 10 client.28343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 20 client.28343 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea51e61c0 0x555ea5686260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 4 client.28343 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.526+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ea556e640 con 0x555ea51e61c0 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.527+0000 7f6db61b1640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea51e61c0 0x555ea5686260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.527+0000 7f6db61b1640 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea51e61c0 0x555ea5686260 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9e614f7d969c18 server_cookie=429c5a1f435134e2 in_seq=0 out_seq=0 2024-09-21T02:03:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.527+0000 7f6d99ffb640 10 client.28343 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.618+0000 7f6d99ffb640 1 -- 172.21.15.29:0/705892190 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ea556e640 con 0x555ea51e61c0 2024-09-21T02:03:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.618+0000 7f6d99ffb640 10 client.28343 handle_command_reply: tid=0 2024-09-21T02:03:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.618+0000 7f6dbf45f740 1 client.28343 shutdown 2024-09-21T02:03:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.618+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 msgr2=0x555ea5609f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.618+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 0x555ea5609f50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6da0030380 tx=0x7f6da00056d0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 shutdown_connections 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x555ea51e61c0 0x555ea5686260 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ea52521b0 0x555ea560a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea52314d0 0x555ea5609f50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 --2- 172.21.15.29:0/705892190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea52292f0 0x555ea560b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 >> 172.21.15.29:0/705892190 conn(0x555ea5246300 msgr2=0x555ea5601680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6d99ffb640 0 client.28343 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 shutdown_connections 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.619+0000 7f6dbf45f740 1 -- 172.21.15.29:0/705892190 wait complete. 2024-09-21T02:03:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.620+0000 7f6dbf45f740 20 client.28343 trim_cache size 0 max 16384 2024-09-21T02:03:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:03:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:03:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:03:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:03:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.622+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 msgr2=0x7f6d9c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.622+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 0x7f6d9c0b8aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005eb0 tx=0x7f6da00304f0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.623+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 msgr2=0x7f6db019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.623+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 0x7f6db019d380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005fd0 tx=0x7f6da800c740 comp rx=0 tx=0).stop 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 shutdown_connections 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f6d9c0b6590 0x7f6d9c0b8aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6db010b6b0 0x7f6db019d380 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db0077090 0x7f6db019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 --2- 172.21.15.29:0/3334521565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db0075c90 0x7f6db019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 >> 172.21.15.29:0/3334521565 conn(0x7f6db00ff020 msgr2=0x7f6db01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 shutdown_connections 2024-09-21T02:03:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:00.624+0000 7f6db8415640 1 -- 172.21.15.29:0/3334521565 wait complete. 2024-09-21T02:03:00.652 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 -- 172.21.15.29:0/4139719120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54141083a0 msgr2=0x7f54141087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 --2- 172.21.15.29:0/4139719120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54141083a0 0x7f54141087b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5408009900 tx=0x7f540802a3f0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 -- 172.21.15.29:0/4139719120 shutdown_connections 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 --2- 172.21.15.29:0/4139719120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 0x7f541410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 --2- 172.21.15.29:0/4139719120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54141083a0 0x7f54141087b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 --2- 172.21.15.29:0/4139719120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414102a60 0x7f5414102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.514+0000 7f541a1dd640 1 -- 172.21.15.29:0/4139719120 >> 172.21.15.29:0/4139719120 conn(0x7f5414079730 msgr2=0x7f5414079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.515+0000 7f541a1dd640 1 -- 172.21.15.29:0/4139719120 shutdown_connections 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.515+0000 7f541a1dd640 1 -- 172.21.15.29:0/4139719120 wait complete. 2024-09-21T02:03:01.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.515+0000 7f541a1dd640 1 Processor -- start 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 -- start start 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54141083a0 0x7f541419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 0x7f541419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5414112960 con 0x7f5414102a60 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54141127e0 con 0x7f5414108cf0 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f541a1dd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5414112ae0 con 0x7f54141083a0 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56828/0 (socket says 172.21.15.29:56828) 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f54137fe640 1 -- 172.21.15.29:0/328309182 learned_addr learned my addr 172.21.15.29:0/328309182 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f5412ffd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54141083a0 0x7f541419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.516+0000 7f5413fff640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 0x7f541419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 -- 172.21.15.29:0/328309182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54141083a0 msgr2=0x7f541419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54141083a0 0x7f541419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 -- 172.21.15.29:0/328309182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 msgr2=0x7f541419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 0x7f541419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 -- 172.21.15.29:0/328309182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54141a6500 con 0x7f5414102a60 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.517+0000 7f54137fe640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9f0 tx=0x7f540400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.518+0000 7f5410ff9640 1 -- 172.21.15.29:0/328309182 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400cd80 con 0x7f5414102a60 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.518+0000 7f5410ff9640 1 -- 172.21.15.29:0/328309182 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54040054d0 con 0x7f5414102a60 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.518+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54141a67f0 con 0x7f5414102a60 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.518+0000 7f5410ff9640 1 -- 172.21.15.29:0/328309182 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404010690 con 0x7f5414102a60 2024-09-21T02:03:01.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.518+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54141a6d00 con 0x7f5414102a60 2024-09-21T02:03:01.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5410ff9640 1 -- 172.21.15.29:0/328309182 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5404010830 con 0x7f5414102a60 2024-09-21T02:03:01.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 Processor -- start 2024-09-21T02:03:01.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 -- start start 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1ea9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 0x556c1ea9cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c1ea12ea0 con 0x556c1e6cc1b0 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c1ea022d0 con 0x556c1e6f5070 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.523+0000 7f5420751740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556c1ea060e0 con 0x556c1e6d4390 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1ea9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5413fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5413fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50696/0 (socket says 172.21.15.29:50696) 2024-09-21T02:03:01.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5413fff640 1 -- 172.21.15.29:0/3610002513 learned_addr learned my addr 172.21.15.29:0/3610002513 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:01.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 0x556c1ea9cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 -- 172.21.15.29:0/3610002513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 msgr2=0x556c1ea9c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 -- 172.21.15.29:0/3610002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 msgr2=0x556c1ea9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1ea9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 -- 172.21.15.29:0/3610002513 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c1e760650 con 0x556c1e6d4390 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5410ff9640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 0x7f53fc0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5413fff640 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:01.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.524+0000 7f5412ffd640 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 0x556c1ea9cd40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f5408032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37fdf0efa1b3321c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.525+0000 7f53f9ffb640 1 -- 172.21.15.29:0/3610002513 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408035040 con 0x556c1e6d4390 2024-09-21T02:03:01.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.525+0000 7f53f9ffb640 1 -- 172.21.15.29:0/3610002513 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5408004030 con 0x556c1e6d4390 2024-09-21T02:03:01.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.525+0000 7f53f9ffb640 1 -- 172.21.15.29:0/3610002513 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408005430 con 0x556c1e6d4390 2024-09-21T02:03:01.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.525+0000 7f5413fff640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 0x7f53fc0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.525+0000 7f5410ff9640 1 -- 172.21.15.29:0/328309182 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54040e73a0 con 0x7f5414102a60 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 -- 172.21.15.29:0/3610002513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 msgr2=0x556c1ea9cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5413fff640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 0x7f53fc0b8ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f540c011870 tx=0x7f540c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 0x556c1ea9cd40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f5408032b20 comp rx=0 tx=0).stop 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 -- 172.21.15.29:0/3610002513 shutdown_connections 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6d4390 0x556c1ea9cd40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1ea9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 --2- 172.21.15.29:0/3610002513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6f5070 0x556c1ea9c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 -- 172.21.15.29:0/3610002513 >> 172.21.15.29:0/3610002513 conn(0x556c1e6e9460 msgr2=0x556c1ea9a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 -- 172.21.15.29:0/3610002513 shutdown_connections 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.526+0000 7f5420751740 1 -- 172.21.15.29:0/3610002513 wait complete. 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 -- start start 2024-09-21T02:03:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 Processor -- start 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1eaae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 0x556c1eaa8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c1e863700 con 0x556c1e6cc1b0 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c1e99d160 con 0x556c1e6d4390 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556c1eaaed70 con 0x556c1e6f5070 2024-09-21T02:03:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5420751740 10 client.0 _ll_register_callbacks cb 0x556c1e9b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.527+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1eaae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50710/0 (socket says 172.21.15.29:50710) 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 -- 172.21.15.29:0/1917661586 learned_addr learned my addr 172.21.15.29:0/1917661586 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5413fff640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 0x556c1eaa8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 -- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 msgr2=0x556c1eaa8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 0x556c1eaa8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 -- 172.21.15.29:0/1917661586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 msgr2=0x556c1eaae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1eaae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 -- 172.21.15.29:0/1917661586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c1e9cac90 con 0x556c1e6d4390 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5413fff640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 0x556c1eaa8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f54137fe640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1eaae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5412ffd640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54080096c0 tx=0x7f5408009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64df11b8ce8e388b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f53f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540804d020 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54080316a0 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.528+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5408004030 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5408035040 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 10 client.28355 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c1eabef60 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5408004030 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f540805a020 con 0x556c1e6d4390 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 10 client.28355 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 10 client.28355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 20 client.28355 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6b39f0 0x556c1eac22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 4 client.28355 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c1ea14df0 con 0x556c1e6b39f0 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.529+0000 7f54137fe640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6b39f0 0x556c1eac22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.530+0000 7f53f2ffd640 10 client.28355 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.530+0000 7f54137fe640 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6b39f0 0x556c1eac22c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c82da35737d3555 server_cookie=d809837f510daf72 in_seq=0 out_seq=0 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f53f2ffd640 1 -- 172.21.15.29:0/1917661586 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c1ea14df0 con 0x556c1e6b39f0 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f53f2ffd640 10 client.28355 handle_command_reply: tid=0 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 client.28355 shutdown 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 msgr2=0x556c1eaa7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54080096c0 tx=0x7f5408009260 comp rx=0 tx=0).stop 2024-09-21T02:03:01.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 shutdown_connections 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6b39f0 0x556c1eac22c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f53f2ffd640 0 client.28355 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6f5070 0x556c1eaa8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eaa7910 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 --2- 172.21.15.29:0/1917661586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6cc1b0 0x556c1eaae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.531+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 >> 172.21.15.29:0/1917661586 conn(0x556c1e6e9460 msgr2=0x556c1ea9f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.533+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 shutdown_connections 2024-09-21T02:03:01.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.533+0000 7f5420751740 1 -- 172.21.15.29:0/1917661586 wait complete. 2024-09-21T02:03:01.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.533+0000 7f5420751740 20 client.28355 trim_cache size 0 max 16384 2024-09-21T02:03:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 Processor -- start 2024-09-21T02:03:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 -- start start 2024-09-21T02:03:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6b39f0 0x556c1eaa6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 0x556c1eaa8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f540000e420 con 0x556c1e6d4390 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f540000ff60 con 0x556c1e6cc1b0 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5420751740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f540000eaf0 con 0x556c1e6b39f0 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5413fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 0x556c1eaa8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:50718/0 (socket says 172.21.15.29:50718) 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.550+0000 7f5412ffd640 1 -- 172.21.15.29:0/3726503258 learned_addr learned my addr 172.21.15.29:0/3726503258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f54137fe640 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6b39f0 0x556c1eaa6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5413fff640 1 -- 172.21.15.29:0/3726503258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6b39f0 msgr2=0x556c1eaa6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5413fff640 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6b39f0 0x556c1eaa6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5413fff640 1 -- 172.21.15.29:0/3726503258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 msgr2=0x556c1eaa66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5413fff640 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5413fff640 1 -- 172.21.15.29:0/3726503258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c1eaa8390 con 0x556c1e6d4390 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.551+0000 7f5412ffd640 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.552+0000 7f5413fff640 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 0x556c1eaa8e30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f540c009fd0 tx=0x7f540c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8683ed7664a85ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.552+0000 7f53f97fa640 1 -- 172.21.15.29:0/3726503258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c003d40 con 0x556c1e6d4390 2024-09-21T02:03:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.552+0000 7f53f97fa640 1 -- 172.21.15.29:0/3726503258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c004590 con 0x556c1e6d4390 2024-09-21T02:03:01.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.552+0000 7f53f97fa640 1 -- 172.21.15.29:0/3726503258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c005790 con 0x556c1e6d4390 2024-09-21T02:03:01.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 -- 172.21.15.29:0/3726503258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 msgr2=0x556c1eaa8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 0x556c1eaa8e30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f540c009fd0 tx=0x7f540c01a000 comp rx=0 tx=0).stop 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 -- 172.21.15.29:0/3726503258 shutdown_connections 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6d4390 0x556c1eaa8e30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6cc1b0 0x556c1eaa66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 --2- 172.21.15.29:0/3726503258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6b39f0 0x556c1eaa6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 -- 172.21.15.29:0/3726503258 >> 172.21.15.29:0/3726503258 conn(0x556c1e6e9460 msgr2=0x556c1ea94e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 -- 172.21.15.29:0/3726503258 shutdown_connections 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.553+0000 7f5420751740 1 -- 172.21.15.29:0/3726503258 wait complete. 2024-09-21T02:03:01.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 -- start start 2024-09-21T02:03:01.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 Processor -- start 2024-09-21T02:03:01.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 0x556c1eab8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eab9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f540000f890 con 0x556c1e6b39f0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f540000f1c0 con 0x556c1e6d4390 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f540000c400 con 0x556c1e6cc1b0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5420751740 10 client.0 _ll_register_callbacks cb 0x7f54040ead50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f54137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56872/0 (socket says 172.21.15.29:56872) 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f54137fe640 1 -- 172.21.15.29:0/4031016768 learned_addr learned my addr 172.21.15.29:0/4031016768 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.554+0000 7f5412ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 0x556c1eab8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5413fff640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eab9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 -- 172.21.15.29:0/4031016768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 msgr2=0x556c1eab9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eab9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 -- 172.21.15.29:0/4031016768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 msgr2=0x556c1eabd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 -- 172.21.15.29:0/4031016768 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c1eab9b10 con 0x556c1e6cc1b0 2024-09-21T02:03:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f54137fe640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:01.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5412ffd640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 0x556c1eab8ec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5408004c30 tx=0x7f5408005390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfdeba366fb15b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f53f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540805f020 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f54080316a0 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.555+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f540c004590 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f540804d040 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 10 client.28330 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c1eb2c8f0 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c004590 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f540805a040 con 0x556c1e6cc1b0 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 10 client.28330 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 10 client.28330 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:03:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 20 client.28330 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6f5070 0x556c1eb2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 4 client.28330 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.556+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c1ea108a0 con 0x556c1e6f5070 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.557+0000 7f54137fe640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6f5070 0x556c1eb2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.557+0000 7f53f2ffd640 10 client.28330 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.557+0000 7f54137fe640 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6f5070 0x556c1eb2f040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65a5b282ac4011bb server_cookie=9b6225452373c185 in_seq=0 out_seq=0 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.558+0000 7f53f2ffd640 1 -- 172.21.15.29:0/4031016768 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c1ea108a0 con 0x556c1e6f5070 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.558+0000 7f53f2ffd640 10 client.28330 handle_command_reply: tid=0 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.558+0000 7f5420751740 1 client.28330 shutdown 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 msgr2=0x556c1eab8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 0x556c1eab8ec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5408004c30 tx=0x7f5408005390 comp rx=0 tx=0).stop 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 shutdown_connections 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c1e6d4390 0x556c1eab9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x556c1e6f5070 0x556c1eb2f040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c1e6cc1b0 0x556c1eab8ec0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.559+0000 7f53f2ffd640 0 client.28330 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.560+0000 7f5420751740 1 --2- 172.21.15.29:0/4031016768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c1e6b39f0 0x556c1eabd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.560+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 >> 172.21.15.29:0/4031016768 conn(0x556c1e6e9460 msgr2=0x556c1eaa60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.560+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 shutdown_connections 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.560+0000 7f5420751740 1 -- 172.21.15.29:0/4031016768 wait complete. 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.560+0000 7f5420751740 20 client.28330 trim_cache size 0 max 16384 2024-09-21T02:03:01.564 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 msgr2=0x7f53fc0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 0x7f53fc0b8ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f540c011870 tx=0x7f540c009500 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 msgr2=0x7f541419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9f0 tx=0x7f540400edf0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 shutdown_connections 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f53fc0b65c0 0x7f53fc0b8ad0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5414108cf0 0x7f541419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54141083a0 0x7f541419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 --2- 172.21.15.29:0/328309182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5414102a60 0x7f541419b270 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 >> 172.21.15.29:0/328309182 conn(0x7f5414079730 msgr2=0x7f54141043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 shutdown_connections 2024-09-21T02:03:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:01.561+0000 7f541a1dd640 1 -- 172.21.15.29:0/328309182 wait complete. 2024-09-21T02:03:02.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:01 smithi066 ceph-mon[35001]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-09-21T02:03:02.081 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:01 smithi029 ceph-mon[31228]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-09-21T02:03:02.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:01 smithi190 ceph-mon[34368]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-09-21T02:03:02.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:03:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:02.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 865.0 s, 1172.8 tps, lat 83.596 ms stddev 113.592 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.776+0000 7f047359e640 1 -- 172.21.15.29:0/2897625772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0474107a20 msgr2=0x7f0474107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.776+0000 7f047359e640 1 --2- 172.21.15.29:0/2897625772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0474107a20 0x7f0474107f00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f046c009cd0 tx=0x7f046c02a8f0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 -- 172.21.15.29:0/2897625772 shutdown_connections 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 --2- 172.21.15.29:0/2897625772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0474107a20 0x7f0474107f00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 --2- 172.21.15.29:0/2897625772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f0474107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 --2- 172.21.15.29:0/2897625772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741025e0 0x7f04741029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 -- 172.21.15.29:0/2897625772 >> 172.21.15.29:0/2897625772 conn(0x7f04740fe310 msgr2=0x7f0474100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 -- 172.21.15.29:0/2897625772 shutdown_connections 2024-09-21T02:03:03.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.777+0000 7f047359e640 1 -- 172.21.15.29:0/2897625772 wait complete. 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.778+0000 7f047359e640 1 Processor -- start 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.778+0000 7f047359e640 1 -- start start 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.778+0000 7f047359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04741025e0 0x7f04741a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474107a20 0x7f04741a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047359e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0474119f10 con 0x7f04741025e0 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047359e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0474119d90 con 0x7f0474107a20 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f047411a090 con 0x7f0474102f30 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45490/0 (socket says 172.21.15.29:45490) 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 -- 172.21.15.29:0/1018374080 learned_addr learned my addr 172.21.15.29:0/1018374080 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f047259c640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04741025e0 0x7f04741a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0472d9d640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474107a20 0x7f04741a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 -- 172.21.15.29:0/1018374080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474107a20 msgr2=0x7f04741a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474107a20 0x7f04741a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 -- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04741025e0 msgr2=0x7f04741a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04741025e0 0x7f04741a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.779+0000 7f0471d9b640 1 -- 172.21.15.29:0/1018374080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741a6830 con 0x7f0474102f30 2024-09-21T02:03:03.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f0471d9b640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f046800d740 tx=0x7f046800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f04637fe640 1 -- 172.21.15.29:0/1018374080 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468017070 con 0x7f0474102f30 2024-09-21T02:03:03.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a6b20 con 0x7f0474102f30 2024-09-21T02:03:03.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f04637fe640 1 -- 172.21.15.29:0/1018374080 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046800de50 con 0x7f0474102f30 2024-09-21T02:03:03.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f04637fe640 1 -- 172.21.15.29:0/1018374080 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046800f620 con 0x7f0474102f30 2024-09-21T02:03:03.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.780+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741ad280 con 0x7f0474102f30 2024-09-21T02:03:03.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.783+0000 7f04637fe640 1 -- 172.21.15.29:0/1018374080 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f046801a070 con 0x7f0474102f30 2024-09-21T02:03:03.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.783+0000 7f047f05d740 1 Processor -- start 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 -- start start 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564898728ea0 con 0x56489840b070 2024-09-21T02:03:03.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648987182d0 con 0x5648983ea390 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047f05d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56489871c0e0 con 0x5648983e21b0 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.784+0000 7f047259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45498/0 (socket says 172.21.15.29:45498) 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 -- 172.21.15.29:0/1151107015 learned_addr learned my addr 172.21.15.29:0/1151107015 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f0472d9d640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 -- 172.21.15.29:0/1151107015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 msgr2=0x5648987b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f0471d9b640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 -- 172.21.15.29:0/1151107015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 msgr2=0x5648987b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f047259c640 1 -- 172.21.15.29:0/1151107015 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564898476650 con 0x5648983e21b0 2024-09-21T02:03:03.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.785+0000 7f0471d9b640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:03.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f04637fe640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 0x7f045c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f047259c640 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0464009bc0 tx=0x7f046400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d73dbd3bcfb1d49e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f0472d9d640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 0x7f045c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f04637fe640 1 -- 172.21.15.29:0/1018374080 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f04680e6d30 con 0x7f0474102f30 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f0460ff9640 1 -- 172.21.15.29:0/1151107015 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464016070 con 0x5648983e21b0 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f0460ff9640 1 -- 172.21.15.29:0/1151107015 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04640054d0 con 0x5648983e21b0 2024-09-21T02:03:03.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.786+0000 7f0460ff9640 1 -- 172.21.15.29:0/1151107015 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464011470 con 0x5648983e21b0 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 -- 172.21.15.29:0/1151107015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 msgr2=0x5648987b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0464009bc0 tx=0x7f046400e6c0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f0472d9d640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 0x7f045c0b8aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f046c004af0 tx=0x7f046c005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 -- 172.21.15.29:0/1151107015 shutdown_connections 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987b2da0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 --2- 172.21.15.29:0/1151107015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 -- 172.21.15.29:0/1151107015 >> 172.21.15.29:0/1151107015 conn(0x5648983ff460 msgr2=0x5648987b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 -- 172.21.15.29:0/1151107015 shutdown_connections 2024-09-21T02:03:03.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.788+0000 7f047f05d740 1 -- 172.21.15.29:0/1151107015 wait complete. 2024-09-21T02:03:03.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.789+0000 7f047f05d740 1 -- start start 2024-09-21T02:03:03.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.789+0000 7f047f05d740 1 Processor -- start 2024-09-21T02:03:03.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564898579700 con 0x56489840b070 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45502/0 (socket says 172.21.15.29:45502) 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047259c640 1 -- 172.21.15.29:0/797926411 learned_addr learned my addr 172.21.15.29:0/797926411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f0471d9b640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648986b3160 con 0x5648983ea390 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.793+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5648987c1040 con 0x5648983e21b0 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f047f05d740 10 client.0 _ll_register_callbacks cb 0x5648986cc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0472d9d640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 -- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 msgr2=0x5648987bdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987bdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 -- 172.21.15.29:0/797926411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 msgr2=0x5648987ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648986e0c90 con 0x5648983ea390 2024-09-21T02:03:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0472d9d640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0471d9b640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987be480 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f04680acb70 tx=0x7f04680ac5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e4504a1f18e0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680ea020 con 0x5648983ea390 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0468094270 con 0x5648983ea390 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.794+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04640054d0 con 0x5648983ea390 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0468021030 con 0x5648983ea390 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 10 client.28370 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5648987cccd0 con 0x5648983ea390 2024-09-21T02:03:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04640054d0 con 0x5648983ea390 2024-09-21T02:03:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f04680abc60 con 0x5648983ea390 2024-09-21T02:03:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 10 client.28370 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 10 client.28370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:03:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 20 client.28370 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5648983c99f0 0x5648987d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 4 client.28370 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.795+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56489872adf0 con 0x5648983c99f0 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.796+0000 7f047259c640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5648983c99f0 0x5648987d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.797+0000 7f0459ffb640 10 client.28370 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.797+0000 7f047259c640 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5648983c99f0 0x5648987d0240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae2cafe3b6ef021c server_cookie=4b249b67741c3618 in_seq=0 out_seq=0 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f0459ffb640 1 -- 172.21.15.29:0/797926411 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56489872adf0 con 0x5648983c99f0 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f0459ffb640 10 client.28370 handle_command_reply: tid=0 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 client.28370 shutdown 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 msgr2=0x5648987be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987be480 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f04680acb70 tx=0x7f04680ac5e0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 shutdown_connections 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x5648983c99f0 0x5648987d0240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56489840b070 0x5648987ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648983ea390 0x5648987be480 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 --2- 172.21.15.29:0/797926411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f0459ffb640 0 client.28370 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 >> 172.21.15.29:0/797926411 conn(0x5648983ff460 msgr2=0x5648987b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 shutdown_connections 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 1 -- 172.21.15.29:0/797926411 wait complete. 2024-09-21T02:03:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.798+0000 7f047f05d740 20 client.28370 trim_cache size 0 max 16384 2024-09-21T02:03:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 Processor -- start 2024-09-21T02:03:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 -- start start 2024-09-21T02:03:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983ea390 0x5648987afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b58b0 con 0x5648983e21b0 2024-09-21T02:03:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b73f0 con 0x56489840b070 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047f05d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b5f80 con 0x5648983ea390 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45460/0 (socket says 172.21.15.29:45460) 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f047259c640 1 -- 172.21.15.29:0/2825075482 learned_addr learned my addr 172.21.15.29:0/2825075482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f0471d9b640 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983ea390 0x5648987afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.815+0000 7f0472d9d640 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 -- 172.21.15.29:0/2825075482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983ea390 msgr2=0x5648987afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983ea390 0x5648987afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 -- 172.21.15.29:0/2825075482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 msgr2=0x5648987b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 -- 172.21.15.29:0/2825075482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648987c0680 con 0x5648983e21b0 2024-09-21T02:03:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f047259c640 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0464009b10 tx=0x7f0464004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f844aa1be6e950e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f045bfff640 1 -- 172.21.15.29:0/2825075482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464016040 con 0x5648983e21b0 2024-09-21T02:03:03.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f045bfff640 1 -- 172.21.15.29:0/2825075482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0464005850 con 0x5648983e21b0 2024-09-21T02:03:03.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.816+0000 7f045bfff640 1 -- 172.21.15.29:0/2825075482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640054d0 con 0x5648983e21b0 2024-09-21T02:03:03.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- 172.21.15.29:0/2825075482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 msgr2=0x5648987bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0464009b10 tx=0x7f0464004bd0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- 172.21.15.29:0/2825075482 shutdown_connections 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983ea390 0x5648987afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 --2- 172.21.15.29:0/2825075482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983e21b0 0x5648987bfb70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- 172.21.15.29:0/2825075482 >> 172.21.15.29:0/2825075482 conn(0x5648983ff460 msgr2=0x5648987bf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- 172.21.15.29:0/2825075482 shutdown_connections 2024-09-21T02:03:03.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- 172.21.15.29:0/2825075482 wait complete. 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 -- start start 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.818+0000 7f047f05d740 1 Processor -- start 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 0x5648987c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b6d20 con 0x5648983ea390 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b6650 con 0x56489840b070 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b3890 con 0x5648983e21b0 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047f05d740 10 client.0 _ll_register_callbacks cb 0x7f04680acc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f047259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0471d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 0x5648987c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39790/0 (socket says 172.21.15.29:39790) 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 -- 172.21.15.29:0/3248415922 learned_addr learned my addr 172.21.15.29:0/3248415922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 -- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 msgr2=0x5648987c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 -- 172.21.15.29:0/3248415922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 msgr2=0x5648987c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 0x5648987c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0472d9d640 1 -- 172.21.15.29:0/3248415922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564898839cc0 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.819+0000 7f0471d9b640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 0x5648987c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0472d9d640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f046c009cd0 tx=0x7f046c02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46241dbec1e6a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c003d00 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f046c042030 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0464005850 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f046c004040 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f047f05d740 10 client.28376 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56489883af70 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0464005850 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.820+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f046c0558a0 con 0x56489840b070 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 10 client.28376 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 10 client.28376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:03:03.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 20 client.28376 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:03.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56489839eda0 0x56489883d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:03.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 4 client.28376 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:03.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648987268a0 con 0x56489839eda0 2024-09-21T02:03:03.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.821+0000 7f047259c640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56489839eda0 0x56489883d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.823+0000 7f0459ffb640 10 client.28376 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.823+0000 7f047259c640 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56489839eda0 0x56489883d760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52dab660de8db6a0 server_cookie=6ba466c8be20061d in_seq=0 out_seq=0 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.823+0000 7f0459ffb640 1 -- 172.21.15.29:0/3248415922 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648987268a0 con 0x56489839eda0 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.823+0000 7f0459ffb640 10 client.28376 handle_command_reply: tid=0 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.823+0000 7f047f05d740 1 client.28376 shutdown 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 msgr2=0x5648987c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f046c009cd0 tx=0x7f046c02a8f0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 shutdown_connections 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56489839eda0 0x56489883d760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56489840b070 0x5648987c2d70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648983ea390 0x5648987c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 --2- 172.21.15.29:0/3248415922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648983e21b0 0x5648987c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 >> 172.21.15.29:0/3248415922 conn(0x5648983ff460 msgr2=0x5648987b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f0459ffb640 0 client.28376 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 shutdown_connections 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 1 -- 172.21.15.29:0/3248415922 wait complete. 2024-09-21T02:03:03.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047f05d740 20 client.28376 trim_cache size 0 max 16384 2024-09-21T02:03:03.827 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 msgr2=0x7f045c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 0x7f045c0b8aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f046c004af0 tx=0x7f046c005d80 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 msgr2=0x7f04741a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.824+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f046800d740 tx=0x7f046800dc00 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 shutdown_connections 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f045c0b6590 0x7f045c0b8aa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474107a20 0x7f04741a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0474102f30 0x7f04741a0e60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 --2- 172.21.15.29:0/1018374080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04741025e0 0x7f04741a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 >> 172.21.15.29:0/1018374080 conn(0x7f04740fe310 msgr2=0x7f0474105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:03.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 shutdown_connections 2024-09-21T02:03:03.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:03.825+0000 7f047359e640 1 -- 172.21.15.29:0/1018374080 wait complete. 2024-09-21T02:03:04.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:03 smithi066 ceph-mon[35001]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-21T02:03:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:03 smithi029 ceph-mon[31228]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-21T02:03:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:03 smithi190 ceph-mon[34368]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-21T02:03:04.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:03:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3950594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f581083a0 msgr2=0x7f1f581087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3950594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f581083a0 0x7f1f581087b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009900 tx=0x7f1f4c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3950594063 shutdown_connections 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3950594063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f58108cf0 0x7f1f5810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3950594063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f581083a0 0x7f1f581087b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3950594063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58102a60 0x7f1f58102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3950594063 >> 172.21.15.29:0/3950594063 conn(0x7f1f58079730 msgr2=0x7f1f58079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3950594063 shutdown_connections 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.625+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3950594063 wait complete. 2024-09-21T02:03:05.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.626+0000 7f1f5e4a2640 1 Processor -- start 2024-09-21T02:03:05.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 -- start start 2024-09-21T02:03:05.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f58102a60 0x7f1f58074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f581083a0 0x7f1f5806f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58112960 con 0x7f1f58108cf0 2024-09-21T02:03:05.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581127e0 con 0x7f1f58102a60 2024-09-21T02:03:05.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.632+0000 7f1f5e4a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58112ae0 con 0x7f1f581083a0 2024-09-21T02:03:05.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.634+0000 7f1f5ca3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.634+0000 7f1f5ca3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45488/0 (socket says 172.21.15.29:45488) 2024-09-21T02:03:05.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.634+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/3367435682 learned_addr learned my addr 172.21.15.29:0/3367435682 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:05.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.635+0000 7f1f577fe640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f581083a0 0x7f1f5806f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.636+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/3367435682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f581083a0 msgr2=0x7f1f5806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.636+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f581083a0 0x7f1f5806f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.636+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/3367435682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f58102a60 msgr2=0x7f1f58074fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:03:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.636+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f58102a60 0x7f1f58074fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.636+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/3367435682 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f581aee30 con 0x7f1f58108cf0 2024-09-21T02:03:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.637+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009fd0 tx=0x7f1f5000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.638+0000 7f1f557fa640 1 -- 172.21.15.29:0/3367435682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50017070 con 0x7f1f58108cf0 2024-09-21T02:03:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.638+0000 7f1f557fa640 1 -- 172.21.15.29:0/3367435682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f50003e20 con 0x7f1f58108cf0 2024-09-21T02:03:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.638+0000 7f1f557fa640 1 -- 172.21.15.29:0/3367435682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50004820 con 0x7f1f58108cf0 2024-09-21T02:03:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.641+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581094f0 con 0x7f1f58108cf0 2024-09-21T02:03:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.641+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581099b0 con 0x7f1f58108cf0 2024-09-21T02:03:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.645+0000 7f1f64a16740 1 Processor -- start 2024-09-21T02:03:05.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 -- start start 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45490/0 (socket says 172.21.15.29:45490) 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614aa841ea0 con 0x5614aa524070 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614aa8312d0 con 0x5614aa503390 2024-09-21T02:03:05.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f64a16740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5614aa8350e0 con 0x5614aa4fb1b0 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.651+0000 7f1f577fe640 1 -- 172.21.15.29:0/1497813503 learned_addr learned my addr 172.21.15.29:0/1497813503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f557fa640 1 -- 172.21.15.29:0/3367435682 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1f50004140 con 0x7f1f58108cf0 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 -- 172.21.15.29:0/1497813503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 msgr2=0x5614aa8d4010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d4010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 -- 172.21.15.29:0/1497813503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 msgr2=0x5614aa8d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 -- 172.21.15.29:0/1497813503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614aa58f650 con 0x5614aa524070 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.652+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005940 tx=0x7f1f4c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f630df582c6ea7c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.658+0000 7f1f557fa640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 0x7f1f400b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.659+0000 7f1f557fa640 1 -- 172.21.15.29:0/3367435682 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f500e6fd0 con 0x7f1f58108cf0 2024-09-21T02:03:05.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.659+0000 7f1f3e7fc640 1 -- 172.21.15.29:0/1497813503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c035030 con 0x5614aa524070 2024-09-21T02:03:05.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.659+0000 7f1f3e7fc640 1 -- 172.21.15.29:0/1497813503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4c005050 con 0x5614aa524070 2024-09-21T02:03:05.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.659+0000 7f1f3e7fc640 1 -- 172.21.15.29:0/1497813503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0044e0 con 0x5614aa524070 2024-09-21T02:03:05.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.660+0000 7f1f57fff640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 0x7f1f400b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.661+0000 7f1f64a16740 1 -- 172.21.15.29:0/1497813503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 msgr2=0x5614aa8d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.661+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005940 tx=0x7f1f4c0058b0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.661+0000 7f1f57fff640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 0x7f1f400b8bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5614aa8d2750 tx=0x7f1f48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.662+0000 7f1f64a16740 1 -- 172.21.15.29:0/1497813503 shutdown_connections 2024-09-21T02:03:05.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.662+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.662+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.662+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1497813503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8d3ad0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.662+0000 7f1f64a16740 1 -- 172.21.15.29:0/1497813503 >> 172.21.15.29:0/1497813503 conn(0x5614aa518460 msgr2=0x5614aa8d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.665+0000 7f1f64a16740 1 -- 172.21.15.29:0/1497813503 shutdown_connections 2024-09-21T02:03:05.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.668+0000 7f1f64a16740 1 -- 172.21.15.29:0/1497813503 wait complete. 2024-09-21T02:03:05.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.668+0000 7f1f64a16740 1 -- start start 2024-09-21T02:03:05.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 Processor -- start 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614aa692700 con 0x5614aa524070 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614aa7cc160 con 0x5614aa503390 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5614aa8e2250 con 0x5614aa4fb1b0 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.669+0000 7f1f64a16740 10 client.0 _ll_register_callbacks cb 0x5614aa7e59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:05.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.672+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.673+0000 7f1f57fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.673+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39806/0 (socket says 172.21.15.29:39806) 2024-09-21T02:03:05.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.673+0000 7f1f577fe640 1 -- 172.21.15.29:0/1550351055 learned_addr learned my addr 172.21.15.29:0/1550351055 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.673+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 -- 172.21.15.29:0/1550351055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 msgr2=0x5614aa8df280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8df280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 -- 172.21.15.29:0/1550351055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 msgr2=0x5614aa8eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 -- 172.21.15.29:0/1550351055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614aa7f9c90 con 0x5614aa503390 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f577fe640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005790 tx=0x7f1f4c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee806d5c2919775f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.674+0000 7f1f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:05.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.675+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c071060 con 0x5614aa503390 2024-09-21T02:03:05.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.675+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f4c004250 con 0x5614aa503390 2024-09-21T02:03:05.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.675+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4c005050 con 0x5614aa503390 2024-09-21T02:03:05.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.676+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1f4c035040 con 0x5614aa503390 2024-09-21T02:03:05.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.677+0000 7f1f64a16740 10 client.28385 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:05.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.677+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5614aa8ee050 con 0x5614aa503390 2024-09-21T02:03:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4c005050 con 0x5614aa503390 2024-09-21T02:03:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1f4c0035f0 con 0x5614aa503390 2024-09-21T02:03:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 10 client.28385 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 10 client.28385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:03:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 20 client.28385 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4e29f0 0x5614aa8f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 4 client.28385 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614aa843df0 con 0x5614aa4e29f0 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.681+0000 7f1f57fff640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4e29f0 0x5614aa8f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.682+0000 7f1f57fff640 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4e29f0 0x5614aa8f13d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=968cc5058d84a6d server_cookie=ac484a1acad3b911 in_seq=0 out_seq=0 2024-09-21T02:03:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.682+0000 7f1f377fe640 10 client.28385 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.683+0000 7f1f377fe640 1 -- 172.21.15.29:0/1550351055 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614aa843df0 con 0x5614aa4e29f0 2024-09-21T02:03:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.683+0000 7f1f377fe640 10 client.28385 handle_command_reply: tid=0 2024-09-21T02:03:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.683+0000 7f1f64a16740 1 client.28385 shutdown 2024-09-21T02:03:05.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 msgr2=0x5614aa8df7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005790 tx=0x7f1f4c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:05.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 shutdown_connections 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4e29f0 0x5614aa8f13d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa524070 0x5614aa8eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa503390 0x5614aa8df7c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 --2- 172.21.15.29:0/1550351055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa4fb1b0 0x5614aa8df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 >> 172.21.15.29:0/1550351055 conn(0x5614aa518460 msgr2=0x5614aa8d6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f377fe640 0 client.28385 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.684+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 shutdown_connections 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.685+0000 7f1f64a16740 1 -- 172.21.15.29:0/1550351055 wait complete. 2024-09-21T02:03:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.685+0000 7f1f64a16740 20 client.28385 trim_cache size 0 max 16384 2024-09-21T02:03:05.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.707+0000 7f1f64a16740 1 Processor -- start 2024-09-21T02:03:05.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.707+0000 7f1f64a16740 1 -- start start 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5819e8b0 con 0x5614aa4fb1b0 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a03f0 con 0x5614aa524070 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f64a16740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5819ef80 con 0x5614aa503390 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45554/0 (socket says 172.21.15.29:45554) 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f577fe640 1 -- 172.21.15.29:0/4117744137 learned_addr learned my addr 172.21.15.29:0/4117744137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.708+0000 7f1f57fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/4117744137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 msgr2=0x5614aa8eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/4117744137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 msgr2=0x5614aa8d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 -- 172.21.15.29:0/4117744137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614aa8d5ff0 con 0x5614aa524070 2024-09-21T02:03:05.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f577fe640 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8eb980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1f500e4a80 tx=0x7f1f500acae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520796a9184b6297 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f3dffb640 1 -- 172.21.15.29:0/4117744137 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500ad890 con 0x5614aa524070 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f3dffb640 1 -- 172.21.15.29:0/4117744137 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f500b1c70 con 0x5614aa524070 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.709+0000 7f1f3dffb640 1 -- 172.21.15.29:0/4117744137 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50098430 con 0x5614aa524070 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 -- 172.21.15.29:0/4117744137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 msgr2=0x5614aa8eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8eb980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1f500e4a80 tx=0x7f1f500acae0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 -- 172.21.15.29:0/4117744137 shutdown_connections 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8eb980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 --2- 172.21.15.29:0/4117744137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.710+0000 7f1f64a16740 1 -- 172.21.15.29:0/4117744137 >> 172.21.15.29:0/4117744137 conn(0x5614aa518460 msgr2=0x5614aa8d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.711+0000 7f1f64a16740 1 -- 172.21.15.29:0/4117744137 shutdown_connections 2024-09-21T02:03:05.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.711+0000 7f1f64a16740 1 -- 172.21.15.29:0/4117744137 wait complete. 2024-09-21T02:03:05.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.711+0000 7f1f64a16740 1 -- start start 2024-09-21T02:03:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 Processor -- start 2024-09-21T02:03:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8e3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5819fd20 con 0x5614aa4fb1b0 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5819f650 con 0x5614aa524070 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5819c890 con 0x5614aa503390 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f64a16740 10 client.0 _ll_register_callbacks cb 0x7f1f4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45560/0 (socket says 172.21.15.29:45560) 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f577fe640 1 -- 172.21.15.29:0/188501994 learned_addr learned my addr 172.21.15.29:0/188501994 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f57fff640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.712+0000 7f1f5ca3f640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 -- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 msgr2=0x5614aa8e3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8e3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 -- 172.21.15.29:0/188501994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 msgr2=0x5614aa8e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 -- 172.21.15.29:0/188501994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614aa8e3c20 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f577fe640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005790 tx=0x7f1f4c003de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b641705d2b6d1bab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c07b020 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f1f4c004450 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f500b1c70 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1f4c071030 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.713+0000 7f1f64a16740 10 client.28360 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5614aa95bbe0 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f500b1c70 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1f4c004960 con 0x5614aa503390 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 10 client.28360 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 10 client.28360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:03:05.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 20 client.28360 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4b7da0 0x5614aa95e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.714+0000 7f1f64a16740 4 client.28360 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.715+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614aa83f8a0 con 0x5614aa4b7da0 2024-09-21T02:03:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.715+0000 7f1f57fff640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4b7da0 0x5614aa95e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.716+0000 7f1f57fff640 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4b7da0 0x5614aa95e340 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a8fe33b2ee03322 server_cookie=90e4170e99c089b in_seq=0 out_seq=0 2024-09-21T02:03:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.716+0000 7f1f377fe640 10 client.28360 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.716+0000 7f1f377fe640 1 -- 172.21.15.29:0/188501994 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614aa83f8a0 con 0x5614aa4b7da0 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.716+0000 7f1f377fe640 10 client.28360 handle_command_reply: tid=0 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.716+0000 7f1f64a16740 1 client.28360 shutdown 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 msgr2=0x5614aa8e3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005790 tx=0x7f1f4c003de0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 shutdown_connections 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x5614aa4b7da0 0x5614aa95e340 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614aa524070 0x5614aa8e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5614aa503390 0x5614aa8e3040 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005790 tx=0x7f1f4c003de0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 --2- 172.21.15.29:0/188501994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614aa4fb1b0 0x5614aa8e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 >> 172.21.15.29:0/188501994 conn(0x5614aa518460 msgr2=0x5614aa8d6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.717+0000 7f1f377fe640 0 client.28360 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.718+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 shutdown_connections 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.718+0000 7f1f64a16740 1 -- 172.21.15.29:0/188501994 wait complete. 2024-09-21T02:03:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.718+0000 7f1f64a16740 20 client.28360 trim_cache size 0 max 16384 2024-09-21T02:03:05.721 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:05.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 msgr2=0x7f1f400b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 0x7f1f400b8bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5614aa8d2750 tx=0x7f1f48005f90 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 msgr2=0x7f1f5806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009fd0 tx=0x7f1f5000b260 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 shutdown_connections 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f1f400b66e0 0x7f1f400b8bf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f58108cf0 0x7f1f5806f960 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f581083a0 0x7f1f5806f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 --2- 172.21.15.29:0/3367435682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f58102a60 0x7f1f58074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 >> 172.21.15.29:0/3367435682 conn(0x7f1f58079730 msgr2=0x7f1f58101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 shutdown_connections 2024-09-21T02:03:05.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:05.719+0000 7f1f5e4a2640 1 -- 172.21.15.29:0/3367435682 wait complete. 2024-09-21T02:03:06.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:05 smithi066 ceph-mon[35001]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-21T02:03:06.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:05 smithi029 ceph-mon[31228]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-21T02:03:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:05 smithi190 ceph-mon[34368]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-21T02:03:06.531 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:03:06.920 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:07.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 -- 172.21.15.29:0/1363120100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 msgr2=0x7f1844102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 --2- 172.21.15.29:0/1363120100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844102e70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1834009900 tx=0x7f183402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 -- 172.21.15.29:0/1363120100 shutdown_connections 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 --2- 172.21.15.29:0/1363120100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844108cf0 0x7f184410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 --2- 172.21.15.29:0/1363120100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18441083a0 0x7f18441087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 --2- 172.21.15.29:0/1363120100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844102e70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 -- 172.21.15.29:0/1363120100 >> 172.21.15.29:0/1363120100 conn(0x7f1844079730 msgr2=0x7f1844079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 -- 172.21.15.29:0/1363120100 shutdown_connections 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.525+0000 7f1849ce7640 1 -- 172.21.15.29:0/1363120100 wait complete. 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.526+0000 7f1849ce7640 1 Processor -- start 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.526+0000 7f1849ce7640 1 -- start start 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1844108cf0 0x7f1844104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1844112960 con 0x7f1844102a60 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18441127e0 con 0x7f18441083a0 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1849ce7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1844112ae0 con 0x7f1844108cf0 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39854/0 (socket says 172.21.15.29:39854) 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 -- 172.21.15.29:0/36391438 learned_addr learned my addr 172.21.15.29:0/36391438 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f18437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f1843fff640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1844108cf0 0x7f1844104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 -- 172.21.15.29:0/36391438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1844108cf0 msgr2=0x7f1844104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1844108cf0 0x7f1844104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 -- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 msgr2=0x7f1844104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 -- 172.21.15.29:0/36391438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18441a8720 con 0x7f18441083a0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.527+0000 7f183affd640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e9c0 tx=0x7f182c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.528+0000 7f18417fa640 1 -- 172.21.15.29:0/36391438 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cd80 con 0x7f18441083a0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.528+0000 7f18417fa640 1 -- 172.21.15.29:0/36391438 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f182c0054d0 con 0x7f18441083a0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.528+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18441a8a10 con 0x7f18441083a0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.528+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a8fa0 con 0x7f18441083a0 2024-09-21T02:03:07.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.528+0000 7f18417fa640 1 -- 172.21.15.29:0/36391438 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c010430 con 0x7f18441083a0 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 Processor -- start 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 -- start start 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f18417fa640 1 -- 172.21.15.29:0/36391438 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f182c0105d0 con 0x7f18441083a0 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561803cebea0 con 0x5618039a51b0 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561803cdb2d0 con 0x5618039ce070 2024-09-21T02:03:07.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f185025b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561803cdf0e0 con 0x5618039ad390 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f1843fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f18437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f1843fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39868/0 (socket says 172.21.15.29:39868) 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f18437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45544/0 (socket says 172.21.15.29:45544) 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f1843fff640 1 -- 172.21.15.29:0/2635849215 learned_addr learned my addr 172.21.15.29:0/2635849215 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:07.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.532+0000 7f183affd640 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18417fa640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 0x7f18300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18417fa640 1 -- 172.21.15.29:0/36391438 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f182c0e7450 con 0x7f18441083a0 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18437fe640 1 -- 172.21.15.29:0/2635849215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 msgr2=0x561803d76120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18437fe640 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d76120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18437fe640 1 -- 172.21.15.29:0/2635849215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 msgr2=0x561803d756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18437fe640 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f18437fe640 1 -- 172.21.15.29:0/2635849215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561803a39650 con 0x5618039a51b0 2024-09-21T02:03:07.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f1843fff640 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:07.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.533+0000 7f1843fff640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 0x7f18300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.534+0000 7f18437fe640 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1844105ca0 tx=0x7f1834032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611b653b16ba1fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.534+0000 7f1843fff640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 0x7f18300b8aa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f183c00f3c0 tx=0x7f183c011530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.537+0000 7f1839ffb640 1 -- 172.21.15.29:0/2635849215 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834040070 con 0x5618039a51b0 2024-09-21T02:03:07.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.537+0000 7f1839ffb640 1 -- 172.21.15.29:0/2635849215 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1834004030 con 0x5618039a51b0 2024-09-21T02:03:07.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.537+0000 7f1839ffb640 1 -- 172.21.15.29:0/2635849215 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834035040 con 0x5618039a51b0 2024-09-21T02:03:07.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 -- 172.21.15.29:0/2635849215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 msgr2=0x561803d75be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1844105ca0 tx=0x7f1834032b20 comp rx=0 tx=0).stop 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 -- 172.21.15.29:0/2635849215 shutdown_connections 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d76120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d75be0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 --2- 172.21.15.29:0/2635849215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 -- 172.21.15.29:0/2635849215 >> 172.21.15.29:0/2635849215 conn(0x5618039c2460 msgr2=0x561803d73590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 -- 172.21.15.29:0/2635849215 shutdown_connections 2024-09-21T02:03:07.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.538+0000 7f185025b740 1 -- 172.21.15.29:0/2635849215 wait complete. 2024-09-21T02:03:07.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 -- start start 2024-09-21T02:03:07.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 Processor -- start 2024-09-21T02:03:07.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d80de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ad390 0x561803d81320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561803b3c700 con 0x5618039a51b0 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561803c76160 con 0x5618039ad390 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561803d83e20 con 0x5618039ce070 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f185025b740 10 client.0 _ll_register_callbacks cb 0x561803c8f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f1843fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f1843fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45604/0 (socket says 172.21.15.29:45604) 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f1843fff640 1 -- 172.21.15.29:0/3868793139 learned_addr learned my addr 172.21.15.29:0/3868793139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.539+0000 7f18437fe640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d80de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f183affd640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ad390 0x561803d81320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 -- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ad390 msgr2=0x561803d81320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ad390 0x561803d81320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 -- 172.21.15.29:0/3868793139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 msgr2=0x561803d80de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d80de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 -- 172.21.15.29:0/3868793139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561803ca3c90 con 0x5618039ce070 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1843fff640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f183c004290 tx=0x7f183c009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2204cd764dfc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c004b20 con 0x5618039ce070 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1834004030 con 0x5618039ce070 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.540+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f183c003e00 con 0x5618039ce070 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.541+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f183c004cc0 con 0x5618039ce070 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.541+0000 7f185025b740 10 client.28372 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:07.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.541+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561803d8fc40 con 0x5618039ce070 2024-09-21T02:03:07.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.541+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1834004030 con 0x5618039ce070 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f183c00e8f0 con 0x5618039ce070 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 10 client.28372 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 10 client.28372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 20 client.28372 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56180398c9f0 0x561803d935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 4 client.28372 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.549+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561803ceddf0 con 0x56180398c9f0 2024-09-21T02:03:07.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.553+0000 7f18437fe640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56180398c9f0 0x561803d935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.562+0000 7f18437fe640 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56180398c9f0 0x561803d935a0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09487570c336677 server_cookie=43026b0b9b7b1f28 in_seq=0 out_seq=0 2024-09-21T02:03:07.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.565+0000 7f1822ffd640 10 client.28372 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:07.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.569+0000 7f1822ffd640 1 -- 172.21.15.29:0/3868793139 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561803ceddf0 con 0x56180398c9f0 2024-09-21T02:03:07.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.570+0000 7f1822ffd640 10 client.28372 handle_command_reply: tid=0 2024-09-21T02:03:07.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.570+0000 7f185025b740 1 client.28372 shutdown 2024-09-21T02:03:07.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.570+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 msgr2=0x561803d8d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.570+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f183c004290 tx=0x7f183c009e60 comp rx=0 tx=0).stop 2024-09-21T02:03:07.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 shutdown_connections 2024-09-21T02:03:07.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x56180398c9f0 0x561803d935a0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ce070 0x561803d8d170 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ad390 0x561803d81320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 --2- 172.21.15.29:0/3868793139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d80de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 >> 172.21.15.29:0/3868793139 conn(0x5618039c2460 msgr2=0x561803d78830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.571+0000 7f1822ffd640 0 client.28372 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:07.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.574+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 shutdown_connections 2024-09-21T02:03:07.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.574+0000 7f185025b740 1 -- 172.21.15.29:0/3868793139 wait complete. 2024-09-21T02:03:07.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.574+0000 7f185025b740 20 client.28372 trim_cache size 0 max 16384 2024-09-21T02:03:07.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 Processor -- start 2024-09-21T02:03:07.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 -- start start 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d8d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f182400e410 con 0x5618039a51b0 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f182400ff50 con 0x5618039ce070 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f185025b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f182400eae0 con 0x5618039ad390 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f183affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f18437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f183affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45620/0 (socket says 172.21.15.29:45620) 2024-09-21T02:03:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f183affd640 1 -- 172.21.15.29:0/2599193558 learned_addr learned my addr 172.21.15.29:0/2599193558 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.595+0000 7f1843fff640 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d8d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 -- 172.21.15.29:0/2599193558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 msgr2=0x561803d8d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d8d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 -- 172.21.15.29:0/2599193558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 msgr2=0x561803d7fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d7fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 -- 172.21.15.29:0/2599193558 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561803d93b70 con 0x5618039ad390 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f183affd640 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f182c0e4f60 tx=0x7f182c0a9d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f8c23a474fbf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f18397fa640 1 -- 172.21.15.29:0/2599193558 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0acd60 con 0x5618039ad390 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f18397fa640 1 -- 172.21.15.29:0/2599193558 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f182c0ad100 con 0x5618039ad390 2024-09-21T02:03:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.596+0000 7f18397fa640 1 -- 172.21.15.29:0/2599193558 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0b7010 con 0x5618039ad390 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 -- 172.21.15.29:0/2599193558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 msgr2=0x561803d78eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f182c0e4f60 tx=0x7f182c0a9d60 comp rx=0 tx=0).stop 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 -- 172.21.15.29:0/2599193558 shutdown_connections 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039ce070 0x561803d8d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d78eb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 --2- 172.21.15.29:0/2599193558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039a51b0 0x561803d7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 -- 172.21.15.29:0/2599193558 >> 172.21.15.29:0/2599193558 conn(0x5618039c2460 msgr2=0x561803d787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 -- 172.21.15.29:0/2599193558 shutdown_connections 2024-09-21T02:03:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.597+0000 7f185025b740 1 -- 172.21.15.29:0/2599193558 wait complete. 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.598+0000 7f185025b740 1 -- start start 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.598+0000 7f185025b740 1 Processor -- start 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039a51b0 0x561803d89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 0x561803d85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f18437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039a51b0 0x561803d89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f183affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f183affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45626/0 (socket says 172.21.15.29:45626) 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f183affd640 1 -- 172.21.15.29:0/609437740 learned_addr learned my addr 172.21.15.29:0/609437740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f1843fff640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 0x561803d85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f182400f880 con 0x5618039ce070 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f182400f1b0 con 0x5618039a51b0 2024-09-21T02:03:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f182400c3f0 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.599+0000 7f185025b740 10 client.0 _ll_register_callbacks cb 0x7f183c011800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 -- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039a51b0 msgr2=0x561803d89630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039a51b0 0x561803d89630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 -- 172.21.15.29:0/609437740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 msgr2=0x561803d85020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 0x561803d85020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561803d85560 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f1843fff640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 0x561803d85020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f183affd640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f182c0e4fb0 tx=0x7f182c0a9a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46373f89917d08de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f1822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0048c0 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f182c0ac850 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.600+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f182c0ad100 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f182c0adcf0 con 0x5618039ad390 2024-09-21T02:03:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 10 client.28378 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561803dfdc10 con 0x5618039ad390 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f182c0ad100 con 0x5618039ad390 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f182c0a8f40 con 0x5618039ad390 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 10 client.28378 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 10 client.28378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 20 client.28378 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561803961da0 0x561803e00380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 4 client.28378 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561803ce98a0 con 0x561803961da0 2024-09-21T02:03:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.601+0000 7f18437fe640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561803961da0 0x561803e00380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f1822ffd640 10 client.28378 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f18437fe640 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561803961da0 0x561803e00380 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d67eeaa0f592c server_cookie=f9eee4a57e152b6 in_seq=0 out_seq=0 2024-09-21T02:03:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f1822ffd640 1 -- 172.21.15.29:0/609437740 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561803ce98a0 con 0x561803961da0 2024-09-21T02:03:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f1822ffd640 10 client.28378 handle_command_reply: tid=0 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f185025b740 1 client.28378 shutdown 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 msgr2=0x561803d84ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.602+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f182c0e4fb0 tx=0x7f182c0a9a90 comp rx=0 tx=0).stop 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 shutdown_connections 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x561803961da0 0x561803e00380 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618039ce070 0x561803d85020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618039ad390 0x561803d84ae0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f1822ffd640 0 client.28378 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 --2- 172.21.15.29:0/609437740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618039a51b0 0x561803d89630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 >> 172.21.15.29:0/609437740 conn(0x5618039c2460 msgr2=0x561803d72ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 shutdown_connections 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 1 -- 172.21.15.29:0/609437740 wait complete. 2024-09-21T02:03:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f185025b740 20 client.28378 trim_cache size 0 max 16384 2024-09-21T02:03:07.608 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 msgr2=0x7f18300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 0x7f18300b8aa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f183c00f3c0 tx=0x7f183c011530 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 msgr2=0x7f1844104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.603+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e9c0 tx=0x7f182c00ee80 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 shutdown_connections 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f18300b6590 0x7f18300b8aa0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1844108cf0 0x7f1844104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18441083a0 0x7f1844104790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 --2- 172.21.15.29:0/36391438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1844102a60 0x7f1844104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 >> 172.21.15.29:0/36391438 conn(0x7f1844079730 msgr2=0x7f18441019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 shutdown_connections 2024-09-21T02:03:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:07.604+0000 7f1849ce7640 1 -- 172.21.15.29:0/36391438 wait complete. 2024-09-21T02:03:07.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 870.0 s, 1582.8 tps, lat 64.178 ms stddev 67.854 2024-09-21T02:03:08.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:07 smithi029 ceph-mon[31228]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-09-21T02:03:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:07 smithi190 ceph-mon[34368]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-09-21T02:03:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:07 smithi066 ceph-mon[35001]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-09-21T02:03:09.393 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:03:09.394 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:03:09.811 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:10.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:09 smithi029 ceph-mon[31228]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 MiB/s wr, 564 op/s 2024-09-21T02:03:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:09 smithi190 ceph-mon[34368]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 MiB/s wr, 564 op/s 2024-09-21T02:03:10.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:09 smithi066 ceph-mon[35001]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 MiB/s wr, 564 op/s 2024-09-21T02:03:10.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 -- 172.21.15.29:0/3570987057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c581094c0 msgr2=0x7f0c581098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/3570987057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c581094c0 0x7f0c581098d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009860 tx=0x7f0c4c02a340 comp rx=0 tx=0).stop 2024-09-21T02:03:10.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 -- 172.21.15.29:0/3570987057 shutdown_connections 2024-09-21T02:03:10.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/3570987057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58109e10 0x7f0c58110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/3570987057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c581094c0 0x7f0c581098d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/3570987057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c58103b80 0x7f0c58103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 -- 172.21.15.29:0/3570987057 >> 172.21.15.29:0/3570987057 conn(0x7f0c580ff610 msgr2=0x7f0c58101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 -- 172.21.15.29:0/3570987057 shutdown_connections 2024-09-21T02:03:10.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.461+0000 7f0c5d01e640 1 -- 172.21.15.29:0/3570987057 wait complete. 2024-09-21T02:03:10.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.462+0000 7f0c5d01e640 1 Processor -- start 2024-09-21T02:03:10.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 -- start start 2024-09-21T02:03:10.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 0x7f0c58074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c581094c0 0x7f0c5806e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58113ad0 con 0x7f0c581094c0 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58113950 con 0x7f0c58103b80 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5d01e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58113c50 con 0x7f0c58109e10 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c56d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 0x7f0c58074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45644/0 (socket says 172.21.15.29:45644) 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 -- 172.21.15.29:0/2279411118 learned_addr learned my addr 172.21.15.29:0/2279411118 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5659c640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c581094c0 0x7f0c5806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 -- 172.21.15.29:0/2279411118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 msgr2=0x7f0c58074e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 0x7f0c58074e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 -- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c581094c0 msgr2=0x7f0c5806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c581094c0 0x7f0c5806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c5759e640 1 -- 172.21.15.29:0/2279411118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5806ef10 con 0x7f0c58109e10 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.463+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 0x7f0c58074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c5759e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000eb10 tx=0x7f0c5000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c3ffff640 1 -- 172.21.15.29:0/2279411118 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000ce20 con 0x7f0c58109e10 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c3ffff640 1 -- 172.21.15.29:0/2279411118 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c50005020 con 0x7f0c58109e10 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c58073bb0 con 0x7f0c58109e10 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c3ffff640 1 -- 172.21.15.29:0/2279411118 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50010690 con 0x7f0c58109e10 2024-09-21T02:03:10.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.464+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c58074060 con 0x7f0c58109e10 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.468+0000 7f0c63592740 1 Processor -- start 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.468+0000 7f0c3ffff640 1 -- 172.21.15.29:0/2279411118 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0c50005510 con 0x7f0c58109e10 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.468+0000 7f0c63592740 1 -- start start 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597540490 0x55f597919180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f5979196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f597886e60 con 0x55f597540490 2024-09-21T02:03:10.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f597876290 con 0x55f597548670 2024-09-21T02:03:10.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c63592740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f59787a0a0 con 0x55f597569350 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45658/0 (socket says 172.21.15.29:45658) 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c5659c640 1 -- 172.21.15.29:0/3744434816 learned_addr learned my addr 172.21.15.29:0/3744434816 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.469+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f5979196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 -- 172.21.15.29:0/3744434816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 msgr2=0x55f5979196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f5979196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 -- 172.21.15.29:0/3744434816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597540490 msgr2=0x55f597919180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597540490 0x55f597919180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 -- 172.21.15.29:0/3744434816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5975d4930 con 0x55f597569350 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c3ffff640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 0x7f0c400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.470+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02a930 tx=0x7f0c4c005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820ca4aa0fece815 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.471+0000 7f0c3ffff640 1 -- 172.21.15.29:0/2279411118 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c500e78b0 con 0x7f0c58109e10 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.471+0000 7f0c3d7fa640 1 -- 172.21.15.29:0/3744434816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c037040 con 0x55f597569350 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.471+0000 7f0c3d7fa640 1 -- 172.21.15.29:0/3744434816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c030450 con 0x55f597569350 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.471+0000 7f0c3d7fa640 1 -- 172.21.15.29:0/3744434816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c003850 con 0x55f597569350 2024-09-21T02:03:10.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.471+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 0x7f0c400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 -- 172.21.15.29:0/3744434816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 msgr2=0x55f597918c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02a930 tx=0x7f0c4c005780 comp rx=0 tx=0).stop 2024-09-21T02:03:10.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 -- 172.21.15.29:0/3744434816 shutdown_connections 2024-09-21T02:03:10.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f5979196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597540490 0x55f597919180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 --2- 172.21.15.29:0/3744434816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597569350 0x55f597918c40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 -- 172.21.15.29:0/3744434816 >> 172.21.15.29:0/3744434816 conn(0x55f59755d740 msgr2=0x55f597916b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 -- 172.21.15.29:0/3744434816 shutdown_connections 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 0x7f0c400b8aa0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55f5979178c0 tx=0x7f0c48012cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.472+0000 7f0c63592740 1 -- 172.21.15.29:0/3744434816 wait complete. 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 -- start start 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 Processor -- start 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f5979248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 0x55f59792a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5976d7890 con 0x55f597569350 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f597810d40 con 0x55f597548670 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5979273c0 con 0x55f597540490 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:39920/0 (socket says 172.21.15.29:39920) 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c5659c640 1 -- 172.21.15.29:0/1057195049 learned_addr learned my addr 172.21.15.29:0/1057195049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c63592740 10 client.0 _ll_register_callbacks cb 0x55f597874c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.473+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f5979248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5759e640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 0x55f59792a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 -- 172.21.15.29:0/1057195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 msgr2=0x55f5979248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f5979248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 -- 172.21.15.29:0/1057195049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 msgr2=0x55f59792a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 0x55f59792a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 -- 172.21.15.29:0/1057195049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59792acf0 con 0x55f597548670 2024-09-21T02:03:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5759e640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 0x55f59792a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c5659c640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c035040 tx=0x7f0c4c02ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad2310e6fdd48ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c051060 con 0x55f597548670 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.474+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c030450 con 0x55f597548670 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c4c030800 con 0x55f597548670 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0c4c04c480 con 0x55f597548670 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 10 client.28421 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:10.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f5979336c0 con 0x55f597548670 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c030450 con 0x55f597548670 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0c4c0405a0 con 0x55f597548670 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 10 client.28421 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 10 client.28421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 20 client.28421 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.475+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f597527cd0 0x55f597936a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.476+0000 7f0c63592740 4 client.28421 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.476+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f597888db0 con 0x55f597527cd0 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.476+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f597527cd0 0x55f597936a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.476+0000 7f0c3aff5640 10 client.28421 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.476+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f597527cd0 0x55f597936a20 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2649434a0e9cfbf6 server_cookie=893cfe9761eb769c in_seq=0 out_seq=0 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.477+0000 7f0c3aff5640 1 -- 172.21.15.29:0/1057195049 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f597888db0 con 0x55f597527cd0 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.477+0000 7f0c3aff5640 10 client.28421 handle_command_reply: tid=0 2024-09-21T02:03:10.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.477+0000 7f0c63592740 1 client.28421 shutdown 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 msgr2=0x55f597924e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c035040 tx=0x7f0c4c02ceb0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 shutdown_connections 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f597527cd0 0x55f597936a20 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597569350 0x55f59792a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597548670 0x55f597924e80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 --2- 172.21.15.29:0/1057195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f5979248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 >> 172.21.15.29:0/1057195049 conn(0x55f59755d740 msgr2=0x55f59791be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c3aff5640 0 client.28421 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 shutdown_connections 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 1 -- 172.21.15.29:0/1057195049 wait complete. 2024-09-21T02:03:10.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.478+0000 7f0c63592740 20 client.28421 trim_cache size 0 max 16384 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.501+0000 7f0c63592740 1 Processor -- start 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.501+0000 7f0c63592740 1 -- start start 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597924f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f597923f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b58c0 con 0x55f597548670 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b7400 con 0x55f597569350 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c63592740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b5f90 con 0x55f597540490 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c56d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597924f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45618/0 (socket says 172.21.15.29:45618) 2024-09-21T02:03:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 -- 172.21.15.29:0/142991747 learned_addr learned my addr 172.21.15.29:0/142991747 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:10.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c56d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597924f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45680/0 (socket says 172.21.15.29:45680) 2024-09-21T02:03:10.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5759e640 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f597923f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 -- 172.21.15.29:0/142991747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 msgr2=0x55f597924f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597924f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 -- 172.21.15.29:0/142991747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 msgr2=0x55f597923f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f597923f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5659c640 1 -- 172.21.15.29:0/142991747 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f597926c60 con 0x55f597548670 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.502+0000 7f0c5759e640 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f597923f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.503+0000 7f0c5659c640 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c001eb0 tx=0x7f0c4c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53baccdfcad7353c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.503+0000 7f0c3cff9640 1 -- 172.21.15.29:0/142991747 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c05c020 con 0x55f597548670 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.503+0000 7f0c3cff9640 1 -- 172.21.15.29:0/142991747 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c030900 con 0x55f597548670 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.503+0000 7f0c3cff9640 1 -- 172.21.15.29:0/142991747 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c000e60 con 0x55f597548670 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 -- 172.21.15.29:0/142991747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 msgr2=0x55f59792a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c001eb0 tx=0x7f0c4c02a340 comp rx=0 tx=0).stop 2024-09-21T02:03:10.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 -- 172.21.15.29:0/142991747 shutdown_connections 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f597923f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792a5f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 --2- 172.21.15.29:0/142991747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597924f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 -- 172.21.15.29:0/142991747 >> 172.21.15.29:0/142991747 conn(0x55f59755d740 msgr2=0x55f597924880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 -- 172.21.15.29:0/142991747 shutdown_connections 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.504+0000 7f0c63592740 1 -- 172.21.15.29:0/142991747 wait complete. 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.505+0000 7f0c63592740 1 -- start start 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.505+0000 7f0c63592740 1 Processor -- start 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.505+0000 7f0c63592740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.505+0000 7f0c63592740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c63592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f59792e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c63592740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b6d30 con 0x55f597548670 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:45692/0 (socket says 172.21.15.29:45692) 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c63592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b6660 con 0x55f597569350 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c5659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45620/0 (socket says 172.21.15.29:45620) 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b38a0 con 0x55f597540490 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 -- 172.21.15.29:0/3284648711 learned_addr learned my addr 172.21.15.29:0/3284648711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c5759e640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f59792e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c63592740 10 client.0 _ll_register_callbacks cb 0x7f0c500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 -- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 msgr2=0x55f59792e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f59792e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 -- 172.21.15.29:0/3284648711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 msgr2=0x55f59792e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:10.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.506+0000 7f0c56d9d640 1 -- 172.21.15.29:0/3284648711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59792f160 con 0x55f597540490 2024-09-21T02:03:10.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.507+0000 7f0c56d9d640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c48004380 tx=0x7f0c48015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ba580cf8edca53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.507+0000 7f0c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.507+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c480118b0 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.507+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c030900 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.507+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f0c48011a50 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0c48011c50 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 10 client.28396 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f5979a10e0 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c030900 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0c4803b3f0 con 0x55f597540490 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 10 client.28396 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 10 client.28396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 20 client.28396 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f5974fd080 0x55f5979a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 4 client.28396 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f597884860 con 0x55f5974fd080 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.508+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f5974fd080 0x55f5979a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.509+0000 7f0c5659c640 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f5974fd080 0x55f5979a3880 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8c8e9d6b9d2a39 server_cookie=9843a2d99fe508bf in_seq=0 out_seq=0 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.509+0000 7f0c3aff5640 10 client.28396 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.510+0000 7f0c3aff5640 1 -- 172.21.15.29:0/3284648711 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f597884860 con 0x55f5974fd080 2024-09-21T02:03:10.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.510+0000 7f0c3aff5640 10 client.28396 handle_command_reply: tid=0 2024-09-21T02:03:10.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.512+0000 7f0c63592740 1 client.28396 shutdown 2024-09-21T02:03:10.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 msgr2=0x55f597930fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c48004380 tx=0x7f0c48015000 comp rx=0 tx=0).stop 2024-09-21T02:03:10.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 shutdown_connections 2024-09-21T02:03:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55f5974fd080 0x55f5979a3880 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f597569350 0x55f59792e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f597548670 0x55f59792e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c3aff5640 0 client.28396 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 --2- 172.21.15.29:0/3284648711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f597540490 0x55f597930fe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 >> 172.21.15.29:0/3284648711 conn(0x55f59755d740 msgr2=0x55f59791dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 shutdown_connections 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 1 -- 172.21.15.29:0/3284648711 wait complete. 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.513+0000 7f0c63592740 20 client.28396 trim_cache size 0 max 16384 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stdout: "return_code": 0, 2024-09-21T02:03:10.518 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_tag": "be119495-8eb8-47dd-82fd-e7b815bff586", 2024-09-21T02:03:10.519 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": "asynchronous" 2024-09-21T02:03:10.519 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 msgr2=0x7f0c400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 0x7f0c400b8aa0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55f5979178c0 tx=0x7f0c48012cf0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 msgr2=0x7f0c5806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000eb10 tx=0x7f0c5000ef10 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 shutdown_connections 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f0c400b6590 0x7f0c400b8aa0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c58109e10 0x7f0c5806e6f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c581094c0 0x7f0c5806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 --2- 172.21.15.29:0/2279411118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c58103b80 0x7f0c58074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 >> 172.21.15.29:0/2279411118 conn(0x7f0c580ff610 msgr2=0x7f0c58101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.514+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 shutdown_connections 2024-09-21T02:03:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:10.515+0000 7f0c5d01e640 1 -- 172.21.15.29:0/2279411118 wait complete. 2024-09-21T02:03:11.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:03:11.742 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:12.078 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub queued for path: / 2024-09-21T02:03:12.078 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.078 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:03:12.078 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 629 op/s 2024-09-21T02:03:12.078 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: active paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.079 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:11 smithi029 ceph-mon[31228]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub queued for path: / 2024-09-21T02:03:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:03:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 629 op/s 2024-09-21T02:03:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: active paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:11 smithi190 ceph-mon[34368]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub queued for path: / 2024-09-21T02:03:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 629 op/s 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: active paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:11 smithi066 ceph-mon[35001]: scrub summary: idle+waiting paths [/] 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 -- 172.21.15.29:0/1544819385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 msgr2=0x7fee8410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 --2- 172.21.15.29:0/1544819385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8410f520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009ca0 tx=0x7fee7c02a8e0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 -- 172.21.15.29:0/1544819385 shutdown_connections 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 --2- 172.21.15.29:0/1544819385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8410f520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 --2- 172.21.15.29:0/1544819385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee841083a0 0x7fee841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 --2- 172.21.15.29:0/1544819385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84102a60 0x7fee84102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 -- 172.21.15.29:0/1544819385 >> 172.21.15.29:0/1544819385 conn(0x7fee84079730 msgr2=0x7fee84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 -- 172.21.15.29:0/1544819385 shutdown_connections 2024-09-21T02:03:12.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.385+0000 7fee8b81c640 1 -- 172.21.15.29:0/1544819385 wait complete. 2024-09-21T02:03:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.387+0000 7fee8b81c640 1 Processor -- start 2024-09-21T02:03:12.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.387+0000 7fee8b81c640 1 -- start start 2024-09-21T02:03:12.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.388+0000 7fee8b81c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 0x7fee8419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.388+0000 7fee895b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 0x7fee8419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.388+0000 7fee895b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 0x7fee8419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46480/0 (socket says 172.21.15.29:46480) 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee895b8640 1 -- 172.21.15.29:0/3966017535 learned_addr learned my addr 172.21.15.29:0/3966017535 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 0x7fee8419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee88db7640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 0x7fee8419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee89db9640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee841129b0 con 0x7fee84102a60 2024-09-21T02:03:12.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee84112830 con 0x7fee84108cf0 2024-09-21T02:03:12.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.389+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee84112b30 con 0x7fee841083a0 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 -- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 msgr2=0x7fee8419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 0x7fee8419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 -- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 msgr2=0x7fee8419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 0x7fee8419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee841a0380 con 0x7fee84108cf0 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee88db7640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 0x7fee8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee89db9640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8419fc50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fee7c005e30 tx=0x7fee7c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee7a7fc640 1 -- 172.21.15.29:0/3966017535 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c04a070 con 0x7fee84108cf0 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.390+0000 7fee7a7fc640 1 -- 172.21.15.29:0/3966017535 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c0305e0 con 0x7fee84108cf0 2024-09-21T02:03:12.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.391+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841a0610 con 0x7fee84108cf0 2024-09-21T02:03:12.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.391+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841094f0 con 0x7fee84108cf0 2024-09-21T02:03:12.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.392+0000 7fee7a7fc640 1 -- 172.21.15.29:0/3966017535 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c03caa0 con 0x7fee84108cf0 2024-09-21T02:03:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.395+0000 7fee91d90740 1 Processor -- start 2024-09-21T02:03:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee7a7fc640 1 -- 172.21.15.29:0/3966017535 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fee7c04d070 con 0x7fee84108cf0 2024-09-21T02:03:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 -- start start 2024-09-21T02:03:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb40390 0x55a29bf08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a29be7ee50 con 0x55a29bb381b0 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a29be6e280 con 0x55a29bb61070 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee91d90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a29be72090 con 0x55a29bb40390 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.396+0000 7fee88db7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59204/0 (socket says 172.21.15.29:59204) 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 -- 172.21.15.29:0/2511870275 learned_addr learned my addr 172.21.15.29:0/2511870275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb40390 0x55a29bf08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee895b8640 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 -- 172.21.15.29:0/2511870275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb40390 msgr2=0x55a29bf08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb40390 0x55a29bf08cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 -- 172.21.15.29:0/2511870275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 msgr2=0x55a29bf08780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf08780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 -- 172.21.15.29:0/2511870275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29bbcc650 con 0x55a29bb61070 2024-09-21T02:03:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee7a7fc640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 0x7fee700b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee88db7640 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee74011bb0 tx=0x7fee74004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a912f9ea1561b642 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee895b8640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 0x7fee700b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.397+0000 7fee7a7fc640 1 -- 172.21.15.29:0/3966017535 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee7c10b660 con 0x7fee84108cf0 2024-09-21T02:03:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.398+0000 7fee6f7fe640 1 -- 172.21.15.29:0/2511870275 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7400fc50 con 0x55a29bb61070 2024-09-21T02:03:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.398+0000 7fee6f7fe640 1 -- 172.21.15.29:0/2511870275 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7400fdf0 con 0x55a29bb61070 2024-09-21T02:03:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.398+0000 7fee6f7fe640 1 -- 172.21.15.29:0/2511870275 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74003a10 con 0x55a29bb61070 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee895b8640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 0x7fee700b8aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55a29bf06ec0 tx=0x7fee80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 -- 172.21.15.29:0/2511870275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 msgr2=0x55a29bf08240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee74011bb0 tx=0x7fee74004730 comp rx=0 tx=0).stop 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 -- 172.21.15.29:0/2511870275 shutdown_connections 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb40390 0x55a29bf08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf08780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 --2- 172.21.15.29:0/2511870275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb61070 0x55a29bf08240 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 -- 172.21.15.29:0/2511870275 >> 172.21.15.29:0/2511870275 conn(0x55a29bb55460 msgr2=0x55a29bf06130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 -- 172.21.15.29:0/2511870275 shutdown_connections 2024-09-21T02:03:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.399+0000 7fee91d90740 1 -- 172.21.15.29:0/2511870275 wait complete. 2024-09-21T02:03:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.400+0000 7fee91d90740 1 -- start start 2024-09-21T02:03:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.401+0000 7fee91d90740 1 Processor -- start 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf13a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb61070 0x55a29bf1fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a29bccf700 con 0x55a29bb381b0 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a29be09110 con 0x55a29bb40390 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a29bf16a70 con 0x55a29bb61070 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee91d90740 10 client.0 _ll_register_callbacks cb 0x55a29be22990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee88db7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee88db7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59212/0 (socket says 172.21.15.29:59212) 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee88db7640 1 -- 172.21.15.29:0/781290116 learned_addr learned my addr 172.21.15.29:0/781290116 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.402+0000 7fee895b8640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf13a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 -- 172.21.15.29:0/781290116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb61070 msgr2=0x55a29bf1fd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb61070 0x55a29bf1fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 -- 172.21.15.29:0/781290116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 msgr2=0x55a29bf13a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf13a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 -- 172.21.15.29:0/781290116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29be36c40 con 0x55a29bb40390 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee88db7640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fee7401f9f0 tx=0x7fee7401fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4841227a3779b5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.403+0000 7fee6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74011240 con 0x55a29bb40390 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee74028440 con 0x55a29bb40390 2024-09-21T02:03:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7400fdf0 con 0x55a29bb40390 2024-09-21T02:03:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fee740113e0 con 0x55a29bb40390 2024-09-21T02:03:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee91d90740 10 client.28439 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.404+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a29bf22830 con 0x55a29bb40390 2024-09-21T02:03:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee74028af0 con 0x55a29bb40390 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fee74026020 con 0x55a29bb40390 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 10 client.28439 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 10 client.28439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 20 client.28439 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb1f9f0 0x55a29bf25b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 4 client.28439 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.405+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee7400fdf0 con 0x55a29bb1f9f0 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.406+0000 7fee895b8640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb1f9f0 0x55a29bf25b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.407+0000 7fee895b8640 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb1f9f0 0x55a29bf25b90 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bab989beeebd0d server_cookie=3f499daf76f92772 in_seq=0 out_seq=0 2024-09-21T02:03:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.407+0000 7fee6cff9640 10 client.28439 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee6cff9640 1 -- 172.21.15.29:0/781290116 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fee7400fdf0 con 0x55a29bb1f9f0 2024-09-21T02:03:12.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee6cff9640 10 client.28439 handle_command_reply: tid=0 2024-09-21T02:03:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee91d90740 1 client.28439 shutdown 2024-09-21T02:03:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 msgr2=0x55a29bf13f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fee7401f9f0 tx=0x7fee7401fa20 comp rx=0 tx=0).stop 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.408+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 shutdown_connections 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb1f9f0 0x55a29bf25b90 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb61070 0x55a29bf1fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee6cff9640 0 client.28439 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf13f90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 --2- 172.21.15.29:0/781290116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf13a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 >> 172.21.15.29:0/781290116 conn(0x55a29bb55460 msgr2=0x55a29bf0b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 shutdown_connections 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 1 -- 172.21.15.29:0/781290116 wait complete. 2024-09-21T02:03:12.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.409+0000 7fee91d90740 20 client.28439 trim_cache size 0 max 16384 2024-09-21T02:03:12.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 Processor -- start 2024-09-21T02:03:12.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 -- start start 2024-09-21T02:03:12.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb1f9f0 0x55a29bf0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf1fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ad0a0 con 0x55a29bb381b0 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee841aebe0 con 0x55a29bb40390 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.433+0000 7fee91d90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ad770 con 0x55a29bb1f9f0 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee88db7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf1fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59218/0 (socket says 172.21.15.29:59218) 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 -- 172.21.15.29:0/399401880 learned_addr learned my addr 172.21.15.29:0/399401880 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee895b8640 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb1f9f0 0x55a29bf0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 -- 172.21.15.29:0/399401880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb1f9f0 msgr2=0x55a29bf0ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb1f9f0 0x55a29bf0ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 -- 172.21.15.29:0/399401880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 msgr2=0x55a29bf1fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf1fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 -- 172.21.15.29:0/399401880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29bf14d10 con 0x55a29bb40390 2024-09-21T02:03:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee89db9640 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cde80 tx=0x7fee7c030e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c991fe67fc35b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.434+0000 7fee6effd640 1 -- 172.21.15.29:0/399401880 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0d13c0 con 0x55a29bb40390 2024-09-21T02:03:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee6effd640 1 -- 172.21.15.29:0/399401880 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c036040 con 0x55a29bb40390 2024-09-21T02:03:12.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee6effd640 1 -- 172.21.15.29:0/399401880 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c03cc40 con 0x55a29bb40390 2024-09-21T02:03:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 -- 172.21.15.29:0/399401880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 msgr2=0x55a29bf10c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cde80 tx=0x7fee7c030e90 comp rx=0 tx=0).stop 2024-09-21T02:03:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 -- 172.21.15.29:0/399401880 shutdown_connections 2024-09-21T02:03:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf10c40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb381b0 0x55a29bf1fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.435+0000 7fee91d90740 1 --2- 172.21.15.29:0/399401880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb1f9f0 0x55a29bf0ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.436+0000 7fee91d90740 1 -- 172.21.15.29:0/399401880 >> 172.21.15.29:0/399401880 conn(0x55a29bb55460 msgr2=0x55a29bf0b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.436+0000 7fee91d90740 1 -- 172.21.15.29:0/399401880 shutdown_connections 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.436+0000 7fee91d90740 1 -- 172.21.15.29:0/399401880 wait complete. 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.436+0000 7fee91d90740 1 -- start start 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.436+0000 7fee91d90740 1 Processor -- start 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee91d90740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb1f9f0 0x55a29bf17cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee91d90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 0x55a29bf18200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee91d90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee91d90740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ae510 con 0x55a29bb1f9f0 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee89db9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee89db9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf18740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59230/0 (socket says 172.21.15.29:59230) 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee89db9640 1 -- 172.21.15.29:0/1527916059 learned_addr learned my addr 172.21.15.29:0/1527916059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:12.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee88db7640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 0x55a29bf18200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.437+0000 7fee895b8640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb1f9f0 0x55a29bf17cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ade40 con 0x55a29bb40390 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ab0a0 con 0x55a29bb381b0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee91d90740 10 client.0 _ll_register_callbacks cb 0x7fee74011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee88db7640 1 -- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 msgr2=0x55a29bf18740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee88db7640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf18740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee88db7640 1 -- 172.21.15.29:0/1527916059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb1f9f0 msgr2=0x55a29bf17cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee88db7640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb1f9f0 0x55a29bf17cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.438+0000 7fee88db7640 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29bf8f330 con 0x55a29bb381b0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee88db7640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 0x55a29bf18200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee7400fc20 tx=0x7fee74010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f329d3edb18d96b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:12.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740113e0 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fee74010c20 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7c036040 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fee7401fa90 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee91d90740 10 client.28411 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.439+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a29bf90500 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c036040 con 0x55a29bb381b0 2024-09-21T02:03:12.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fee74004900 con 0x55a29bb381b0 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 10 client.28411 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 10 client.28411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 20 client.28411 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb61070 0x55a29bf92c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 4 client.28411 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:03:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a29be80da0 con 0x55a29bb61070 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.440+0000 7fee895b8640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb61070 0x55a29bf92c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee6cff9640 10 client.28411 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee895b8640 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb61070 0x55a29bf92c20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c3f1693b2f8a95 server_cookie=b456c7918937eabb in_seq=0 out_seq=0 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee6cff9640 1 -- 172.21.15.29:0/1527916059 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x55a29be80da0 con 0x55a29bb61070 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee6cff9640 10 client.28411 handle_command_reply: tid=0 2024-09-21T02:03:12.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 client.28411 shutdown 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 msgr2=0x55a29bf18200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 0x55a29bf18200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee7400fc20 tx=0x7fee74010660 comp rx=0 tx=0).stop 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 shutdown_connections 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55a29bb61070 0x55a29bf92c20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a29bb40390 0x55a29bf18740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a29bb381b0 0x55a29bf18200 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 --2- 172.21.15.29:0/1527916059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a29bb1f9f0 0x55a29bf17cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.441+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 >> 172.21.15.29:0/1527916059 conn(0x55a29bb55460 msgr2=0x55a29bf0bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.442+0000 7fee6cff9640 0 client.28411 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.442+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 shutdown_connections 2024-09-21T02:03:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.444+0000 7fee91d90740 1 -- 172.21.15.29:0/1527916059 wait complete. 2024-09-21T02:03:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.444+0000 7fee91d90740 20 client.28411 trim_cache size 0 max 16384 2024-09-21T02:03:12.448 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:03:12.448 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": { 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: "be119495-8eb8-47dd-82fd-e7b815bff586": { 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tag": "be119495-8eb8-47dd-82fd-e7b815bff586", 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: "options": "recursive,force" 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:03:12.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "be119495-8eb8-47dd-82fd-e7b815bff586": { 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:03:10.508986+0000", 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.450 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "c48dc890-9260-4f8f-b08e-1fd9d06fa732": { 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:02:20.247270+0000", 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.452 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.453 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.454 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:03:12.455 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.456 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.457 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:03:12.458 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.460 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:03:12.461 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 msgr2=0x7fee700b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 0x7fee700b8aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55a29bf06ec0 tx=0x7fee80005f90 comp rx=0 tx=0).stop 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 msgr2=0x7fee8419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8419fc50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fee7c005e30 tx=0x7fee7c003f10 comp rx=0 tx=0).stop 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 shutdown_connections 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fee700b6590 0x7fee700b8aa0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee84108cf0 0x7fee8419fc50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee841083a0 0x7fee8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 --2- 172.21.15.29:0/3966017535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee84102a60 0x7fee8419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:12.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 >> 172.21.15.29:0/3966017535 conn(0x7fee84079730 msgr2=0x7fee84101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:12.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 shutdown_connections 2024-09-21T02:03:12.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:12.445+0000 7fee8b81c640 1 -- 172.21.15.29:0/3966017535 wait complete. 2024-09-21T02:03:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 875.0 s, 1466.8 tps, lat 67.972 ms stddev 73.466 2024-09-21T02:03:13.341 INFO:tasks.cephfs.filesystem:scrub status for tag:be119495-8eb8-47dd-82fd-e7b815bff586 - {'path': '/', 'tag': 'be119495-8eb8-47dd-82fd-e7b815bff586', 'options': 'recursive,force'} 2024-09-21T02:03:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.783+0000 7f5fba9da640 1 -- 172.21.15.29:0/3971040961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 msgr2=0x7f5fb411ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.783+0000 7f5fba9da640 1 --2- 172.21.15.29:0/3971040961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb411ab90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009900 tx=0x7f5fa402a440 comp rx=0 tx=0).stop 2024-09-21T02:03:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.790+0000 7f5fba9da640 1 -- 172.21.15.29:0/3971040961 shutdown_connections 2024-09-21T02:03:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.790+0000 7f5fba9da640 1 --2- 172.21.15.29:0/3971040961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb411ab90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.790+0000 7f5fba9da640 1 --2- 172.21.15.29:0/3971040961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb410cdb0 0x7f5fb410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.790+0000 7f5fba9da640 1 --2- 172.21.15.29:0/3971040961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb410c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.790+0000 7f5fba9da640 1 -- 172.21.15.29:0/3971040961 >> 172.21.15.29:0/3971040961 conn(0x7f5fb4070810 msgr2=0x7f5fb4072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.792+0000 7f5fba9da640 1 -- 172.21.15.29:0/3971040961 shutdown_connections 2024-09-21T02:03:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.793+0000 7f5fba9da640 1 -- 172.21.15.29:0/3971040961 wait complete. 2024-09-21T02:03:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.794+0000 7f5fba9da640 1 Processor -- start 2024-09-21T02:03:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 -- start start 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb410cdb0 0x7f5fb4118e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb4119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4112c80 con 0x7f5fb410a3c0 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4112b00 con 0x7f5fb410cdb0 2024-09-21T02:03:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.797+0000 7f5fba9da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4112e00 con 0x7f5fb410f9d0 2024-09-21T02:03:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.800+0000 7f5fb8f77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb4119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.800+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.800+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46534/0 (socket says 172.21.15.29:46534) 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.800+0000 7f5fb3fff640 1 -- 172.21.15.29:0/1115109689 learned_addr learned my addr 172.21.15.29:0/1115109689 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.801+0000 7f5fb8f77640 1 -- 172.21.15.29:0/1115109689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb410cdb0 msgr2=0x7f5fb4118e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.801+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb410cdb0 0x7f5fb4118e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.801+0000 7f5fb8f77640 1 -- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 msgr2=0x7f5fb411a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.801+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.801+0000 7f5fb8f77640 1 -- 172.21.15.29:0/1115109689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb41198f0 con 0x7f5fb410f9d0 2024-09-21T02:03:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.803+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.809+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb4119380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402aa30 tx=0x7f5fa4003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.814+0000 7f5fb17fa640 1 -- 172.21.15.29:0/1115109689 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa403f070 con 0x7f5fb410f9d0 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.814+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb4119b80 con 0x7f5fb410f9d0 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.814+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41b97e0 con 0x7f5fb410f9d0 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.814+0000 7f5fb17fa640 1 -- 172.21.15.29:0/1115109689 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa4031dd0 con 0x7f5fb410f9d0 2024-09-21T02:03:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.814+0000 7f5fb17fa640 1 -- 172.21.15.29:0/1115109689 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4035030 con 0x7f5fb410f9d0 2024-09-21T02:03:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.821+0000 7f5fc1a21740 1 Processor -- start 2024-09-21T02:03:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.821+0000 7f5fc1a21740 1 -- start start 2024-09-21T02:03:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9282f0 0x560b6ccf94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6ccfa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560b6cc6b6f0 con 0x560b6c9304d0 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b6cc5ab20 con 0x560b6c9511b0 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.824+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b6cc5e930 con 0x560b6c9282f0 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.827+0000 7f5fb17fa640 1 -- 172.21.15.29:0/1115109689 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5fa4042070 con 0x7f5fb410f9d0 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.827+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.827+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59248/0 (socket says 172.21.15.29:59248) 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.827+0000 7f5fb37fe640 1 -- 172.21.15.29:0/3096603742 learned_addr learned my addr 172.21.15.29:0/3096603742 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.827+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6ccfa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.828+0000 7f5fb37fe640 1 -- 172.21.15.29:0/3096603742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9282f0 msgr2=0x560b6ccf94c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:03:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.828+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9282f0 0x560b6ccf94c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.828+0000 7f5fb37fe640 1 -- 172.21.15.29:0/3096603742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 msgr2=0x560b6ccfa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.828+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6ccfa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.828+0000 7f5fb37fe640 1 -- 172.21.15.29:0/3096603742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6cc20530 con 0x560b6c9511b0 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.829+0000 7f5fb17fa640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 0x7f5f940b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.829+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bc30 tx=0x7f5f9c00b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbc684a81d7776 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.829+0000 7f5fb17fa640 1 -- 172.21.15.29:0/1115109689 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5fa4107c10 con 0x7f5fb410f9d0 2024-09-21T02:03:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.830+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 0x7f5f940b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.830+0000 7f5faa7fc640 1 -- 172.21.15.29:0/3096603742 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01b070 con 0x560b6c9511b0 2024-09-21T02:03:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.830+0000 7f5faa7fc640 1 -- 172.21.15.29:0/3096603742 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c0060d0 con 0x560b6c9511b0 2024-09-21T02:03:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 -- 172.21.15.29:0/3096603742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 msgr2=0x560b6ccf8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bc30 tx=0x7f5f9c00b2c0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 -- 172.21.15.29:0/3096603742 shutdown_connections 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6ccfa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9282f0 0x560b6ccf94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/3096603742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6ccf8e50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.831+0000 7f5fc1a21740 1 -- 172.21.15.29:0/3096603742 >> 172.21.15.29:0/3096603742 conn(0x560b6c945300 msgr2=0x560b6ccf48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.835+0000 7f5fc1a21740 1 -- 172.21.15.29:0/3096603742 shutdown_connections 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.835+0000 7f5fc1a21740 1 -- 172.21.15.29:0/3096603742 wait complete. 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.836+0000 7f5fc1a21740 1 -- start start 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.836+0000 7f5fc1a21740 1 Processor -- start 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd02720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd02de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560b6cb92ab0 con 0x560b6c9511b0 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b6cc00730 con 0x560b6c9282f0 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b6ccfab80 con 0x560b6c9304d0 2024-09-21T02:03:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.837+0000 7f5fc1a21740 10 client.0 _ll_register_callbacks cb 0x560b6cc546b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46566/0 (socket says 172.21.15.29:46566) 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 -- 172.21.15.29:0/301260720 learned_addr learned my addr 172.21.15.29:0/301260720 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 -- 172.21.15.29:0/301260720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 msgr2=0x560b6cd02de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd02de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 -- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 msgr2=0x560b6cd02720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd02720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.838+0000 7f5fb8f77640 1 -- 172.21.15.29:0/301260720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6c9bc790 con 0x560b6c9511b0 2024-09-21T02:03:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.839+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 0x7f5f940b8c30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5fac007fd0 tx=0x7f5fac007620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.839+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40cdce0 tx=0x7f5fa40cd580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2bc7316b973d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.839+0000 7f5fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.839+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4049040 con 0x560b6c9511b0 2024-09-21T02:03:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.839+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c0060d0 con 0x560b6c9511b0 2024-09-21T02:03:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5fa40d74e0 con 0x560b6c9511b0 2024-09-21T02:03:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5fa40d7770 con 0x560b6c9511b0 2024-09-21T02:03:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fc1a21740 10 client.18645 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560b6ca35620 con 0x560b6c9511b0 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c0060d0 con 0x560b6c9511b0 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.840+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa4105b80 con 0x560b6c9511b0 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 10 client.18645 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 20 client.18645 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c90fd80 0x560b6cd32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 4 client.18645 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b6cc6d640 con 0x560b6c90fd80 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c90fd80 0x560b6cd32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fa37fe640 10 client.18645 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.841+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c90fd80 0x560b6cd32210 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e314f7a8dece8b45 server_cookie=a389530e0b34e0fc in_seq=0 out_seq=0 2024-09-21T02:03:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.842+0000 7f5fa37fe640 1 -- 172.21.15.29:0/301260720 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b6cc6d640 con 0x560b6c90fd80 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.842+0000 7f5fa37fe640 10 client.18645 handle_command_reply: tid=0 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.842+0000 7f5fc1a21740 1 client.18645 shutdown 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.842+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 msgr2=0x560b6cd03950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.842+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40cdce0 tx=0x7f5fa40cd580 comp rx=0 tx=0).stop 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 shutdown_connections 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c90fd80 0x560b6cd32210 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9511b0 0x560b6cd03950 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd02de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/301260720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd02720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 >> 172.21.15.29:0/301260720 conn(0x560b6c945300 msgr2=0x560b6ccf2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fa37fe640 0 client.18645 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 shutdown_connections 2024-09-21T02:03:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 1 -- 172.21.15.29:0/301260720 wait complete. 2024-09-21T02:03:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.843+0000 7f5fc1a21740 20 client.18645 trim_cache size 0 max 16384 2024-09-21T02:03:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.863+0000 7f5fc1a21740 1 Processor -- start 2024-09-21T02:03:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.863+0000 7f5fc1a21740 1 -- start start 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9282f0 0x560b6ccfd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6cd32160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800e420 con 0x560b6c9282f0 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800ff60 con 0x560b6c9511b0 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800eaf0 con 0x560b6c9304d0 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:32850/0 (socket says 172.21.15.29:32850) 2024-09-21T02:03:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.864+0000 7f5fb37fe640 1 -- 172.21.15.29:0/1813022671 learned_addr learned my addr 172.21.15.29:0/1813022671 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 -- 172.21.15.29:0/1813022671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 msgr2=0x560b6cd32160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6cd32160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 -- 172.21.15.29:0/1813022671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9282f0 msgr2=0x560b6ccfd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9282f0 0x560b6ccfd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 -- 172.21.15.29:0/1813022671 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6cd024b0 con 0x560b6c9304d0 2024-09-21T02:03:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fb37fe640 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bc00 tx=0x7f5f9c00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef26bd9e743eb2c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.865+0000 7f5fa9ffb640 1 -- 172.21.15.29:0/1813022671 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01b070 con 0x560b6c9304d0 2024-09-21T02:03:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fa9ffb640 1 -- 172.21.15.29:0/1813022671 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c00dda0 con 0x560b6c9304d0 2024-09-21T02:03:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fa9ffb640 1 -- 172.21.15.29:0/1813022671 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c006650 con 0x560b6c9304d0 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fc1a21740 1 -- 172.21.15.29:0/1813022671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 msgr2=0x560b6cd01110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bc00 tx=0x7f5f9c00dc00 comp rx=0 tx=0).stop 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fc1a21740 1 -- 172.21.15.29:0/1813022671 shutdown_connections 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9511b0 0x560b6cd32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.866+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9304d0 0x560b6cd01110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/1813022671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9282f0 0x560b6ccfd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 -- 172.21.15.29:0/1813022671 >> 172.21.15.29:0/1813022671 conn(0x560b6c945300 msgr2=0x560b6ccef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 -- 172.21.15.29:0/1813022671 shutdown_connections 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 -- 172.21.15.29:0/1813022671 wait complete. 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 -- start start 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.867+0000 7f5fc1a21740 1 Processor -- start 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6cd07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 0x560b6cd08700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800f890 con 0x560b6c9304d0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800f1c0 con 0x560b6c9282f0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800c400 con 0x560b6c9511b0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:59270/0 (socket says 172.21.15.29:59270) 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6cd07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fb3fff640 1 -- 172.21.15.29:0/4153711825 learned_addr learned my addr 172.21.15.29:0/4153711825 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fc1a21740 10 client.0 _ll_register_callbacks cb 0x7f5fa4003ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.868+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 0x560b6cd08700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 -- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 msgr2=0x560b6cd074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 -- 172.21.15.29:0/4153711825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 msgr2=0x560b6cd07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6cd07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 -- 172.21.15.29:0/4153711825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6cd08e60 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fb8f77640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 0x560b6cd08700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009850 tx=0x7f5fa40b9230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7b82307352a74d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4049070 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.869+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5fa407d0e0 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c00dda0 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5fa40cdb20 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 10 client.28423 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560b6cd0a1a0 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c00dda0 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa40d72e0 con 0x560b6c9511b0 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 10 client.28423 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 10 client.28423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 20 client.28423 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.870+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c8e51c0 0x560b6cd9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.871+0000 7f5fc1a21740 4 client.28423 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.871+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b6cc690f0 con 0x560b6c8e51c0 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.871+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c8e51c0 0x560b6cd9ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.871+0000 7f5fa37fe640 10 client.28423 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.871+0000 7f5fb3fff640 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c8e51c0 0x560b6cd9ec20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb186b3633ef006c server_cookie=bbb0864695bff33d in_seq=0 out_seq=0 2024-09-21T02:03:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.879+0000 7f5fa37fe640 1 -- 172.21.15.29:0/4153711825 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b6cc690f0 con 0x560b6c8e51c0 2024-09-21T02:03:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fa37fe640 10 client.28423 handle_command_reply: tid=0 2024-09-21T02:03:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 client.28423 shutdown 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 msgr2=0x560b6cd08700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 0x560b6cd08700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009850 tx=0x7f5fa40b9230 comp rx=0 tx=0).stop 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 shutdown_connections 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x560b6c8e51c0 0x560b6cd9ec20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b6c9511b0 0x560b6cd08700 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560b6c9304d0 0x560b6cd07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 --2- 172.21.15.29:0/4153711825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b6c9282f0 0x560b6cd074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 >> 172.21.15.29:0/4153711825 conn(0x560b6c945300 msgr2=0x560b6cd32160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.880+0000 7f5fa37fe640 0 client.28423 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 shutdown_connections 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fc1a21740 1 -- 172.21.15.29:0/4153711825 wait complete. 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fc1a21740 20 client.28423 trim_cache size 0 max 16384 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:03:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 msgr2=0x7f5f940b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 0x7f5f940b8c30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5fac007fd0 tx=0x7f5fac007620 comp rx=0 tx=0).stop 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 msgr2=0x7f5fb4119380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.881+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb4119380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402aa30 tx=0x7f5fa4003d60 comp rx=0 tx=0).stop 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 shutdown_connections 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5f940b6720 0x7f5f940b8c30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb410f9d0 0x7f5fb4119380 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb410cdb0 0x7f5fb4118e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 --2- 172.21.15.29:0/1115109689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fb410a3c0 0x7f5fb411a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 >> 172.21.15.29:0/1115109689 conn(0x7f5fb4070810 msgr2=0x7f5fb410b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 shutdown_connections 2024-09-21T02:03:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:13.882+0000 7f5fba9da640 1 -- 172.21.15.29:0/1115109689 wait complete. 2024-09-21T02:03:14.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:13 smithi029 ceph-mon[31228]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-21T02:03:14.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:13 smithi190 ceph-mon[34368]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-21T02:03:14.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:13 smithi066 ceph-mon[35001]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-21T02:03:16.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:15 smithi029 ceph-mon[31228]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-21T02:03:16.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:15 smithi190 ceph-mon[34368]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-21T02:03:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:15 smithi066 ceph-mon[35001]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-21T02:03:17.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 880.0 s, 1789.6 tps, lat 56.261 ms stddev 65.151 2024-09-21T02:03:18.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:17 smithi029 ceph-mon[31228]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 739 op/s 2024-09-21T02:03:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:17 smithi190 ceph-mon[34368]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 739 op/s 2024-09-21T02:03:18.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:17 smithi066 ceph-mon[35001]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 739 op/s 2024-09-21T02:03:20.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:19 smithi029 ceph-mon[31228]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 633 op/s 2024-09-21T02:03:20.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:19 smithi190 ceph-mon[34368]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 633 op/s 2024-09-21T02:03:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:19 smithi066 ceph-mon[35001]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 633 op/s 2024-09-21T02:03:22.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:21 smithi029 ceph-mon[31228]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.9 MiB/s wr, 712 op/s 2024-09-21T02:03:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:21 smithi190 ceph-mon[34368]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.9 MiB/s wr, 712 op/s 2024-09-21T02:03:22.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:21 smithi066 ceph-mon[35001]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.9 MiB/s wr, 712 op/s 2024-09-21T02:03:22.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 885.0 s, 1678.4 tps, lat 59.089 ms stddev 68.925 2024-09-21T02:03:24.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:23 smithi029 ceph-mon[31228]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 737 op/s 2024-09-21T02:03:24.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:23 smithi190 ceph-mon[34368]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 737 op/s 2024-09-21T02:03:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:23 smithi066 ceph-mon[35001]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 737 op/s 2024-09-21T02:03:26.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:25 smithi029 ceph-mon[31228]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-09-21T02:03:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:03:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:03:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:03:26.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:25 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:03:26.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:25 smithi066 ceph-mon[35001]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-09-21T02:03:26.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:03:26.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:03:26.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:03:26.102 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:25 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:03:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:25 smithi190 ceph-mon[34368]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-09-21T02:03:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:03:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:03:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:03:26.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:25 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 -- 172.21.15.29:0/3071907682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e0103650 msgr2=0x7fe4e0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3071907682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e0103650 0x7fe4e0103a60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009860 tx=0x7fe4d002a360 comp rx=0 tx=0).stop 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 -- 172.21.15.29:0/3071907682 shutdown_connections 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3071907682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103ff0 0x7fe4e010a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3071907682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e0103650 0x7fe4e0103a60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3071907682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e01025d0 0x7fe4e0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 -- 172.21.15.29:0/3071907682 >> 172.21.15.29:0/3071907682 conn(0x7fe4e00fdf30 msgr2=0x7fe4e01003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 -- 172.21.15.29:0/3071907682 shutdown_connections 2024-09-21T02:03:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.021+0000 7fe4e8425640 1 -- 172.21.15.29:0/3071907682 wait complete. 2024-09-21T02:03:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 Processor -- start 2024-09-21T02:03:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 -- start start 2024-09-21T02:03:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e01025d0 0x7fe4e0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103650 0x7fe4e01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010d150 con 0x7fe4e0103ff0 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010cfd0 con 0x7fe4e0103650 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e8425640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010d2d0 con 0x7fe4e01025d0 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103650 0x7fe4e01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51446/0 (socket says 172.21.15.29:51446) 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3267977088 learned_addr learned my addr 172.21.15.29:0/3267977088 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e01025d0 0x7fe4e0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3267977088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e01025d0 msgr2=0x7fe4e0197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e01025d0 0x7fe4e0197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3267977088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103650 msgr2=0x7fe4e01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.022+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103650 0x7fe4e01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3267977088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e0199780 con 0x7fe4e0103ff0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800eb10 tx=0x7fe4d800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4d77fe640 1 -- 172.21.15.29:0/3267977088 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800ce20 con 0x7fe4e0103ff0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4d77fe640 1 -- 172.21.15.29:0/3267977088 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d8005020 con 0x7fe4e0103ff0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e0199a70 con 0x7fe4e0103ff0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4d77fe640 1 -- 172.21.15.29:0/3267977088 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8010690 con 0x7fe4e0103ff0 2024-09-21T02:03:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.023+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01a0160 con 0x7fe4e0103ff0 2024-09-21T02:03:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.026+0000 7fe4ef472740 1 Processor -- start 2024-09-21T02:03:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.027+0000 7fe4d77fe640 1 -- 172.21.15.29:0/3267977088 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe4d8010830 con 0x7fe4e0103ff0 2024-09-21T02:03:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.027+0000 7fe4ef472740 1 -- start start 2024-09-21T02:03:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591d9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559d591426f0 con 0x559d58dff2f0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559d59131b20 con 0x559d58e074d0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4ef472740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559d59135930 con 0x559d58e281b0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e61c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48050/0 (socket says 172.21.15.29:48050) 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 -- 172.21.15.29:0/3375263250 learned_addr learned my addr 172.21.15.29:0/3375263250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591d9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 -- 172.21.15.29:0/3375263250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 msgr2=0x559d591d9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591d9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 -- 172.21.15.29:0/3375263250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 msgr2=0x559d591d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591d8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e59c0640 1 -- 172.21.15.29:0/3375263250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d590f7530 con 0x559d58e281b0 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.028+0000 7fe4d77fe640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 0x7fe4cc0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.029+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe4d002cfd0 tx=0x7fe4d003c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b1c3509456012a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.029+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 0x7fe4cc0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.029+0000 7fe4d77fe640 1 -- 172.21.15.29:0/3267977088 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d80e7910 con 0x7fe4e0103ff0 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.029+0000 7fe4d4bfe640 1 -- 172.21.15.29:0/3375263250 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0037040 con 0x559d58e281b0 2024-09-21T02:03:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.029+0000 7fe4d4bfe640 1 -- 172.21.15.29:0/3375263250 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0030c30 con 0x559d58e281b0 2024-09-21T02:03:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4d4bfe640 1 -- 172.21.15.29:0/3375263250 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0005090 con 0x559d58e281b0 2024-09-21T02:03:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 -- 172.21.15.29:0/3375263250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 msgr2=0x559d591d8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 0x7fe4cc0b8a90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005fd0 tx=0x7fe4dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe4d002cfd0 tx=0x7fe4d003c040 comp rx=0 tx=0).stop 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 -- 172.21.15.29:0/3375263250 shutdown_connections 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3375263250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591d8660 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 -- 172.21.15.29:0/3375263250 >> 172.21.15.29:0/3375263250 conn(0x559d58e1c300 msgr2=0x559d591cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 -- 172.21.15.29:0/3375263250 shutdown_connections 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.030+0000 7fe4ef472740 1 -- 172.21.15.29:0/3375263250 wait complete. 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 -- start start 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 Processor -- start 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559d59069ab0 con 0x559d58dff2f0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559d590d7730 con 0x559d58e074d0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4e61c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4ef472740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559d591da380 con 0x559d58e281b0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4e61c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51464/0 (socket says 172.21.15.29:51464) 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4e61c1640 1 -- 172.21.15.29:0/376293157 learned_addr learned my addr 172.21.15.29:0/376293157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.031+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4ef472740 10 client.0 _ll_register_callbacks cb 0x559d5912b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 -- 172.21.15.29:0/376293157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 msgr2=0x559d591e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 -- 172.21.15.29:0/376293157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 msgr2=0x559d591e1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 -- 172.21.15.29:0/376293157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d58e93790 con 0x559d58e074d0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591e24b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0035040 tx=0x7fe4d00303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f071c249b75732d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0037050 con 0x559d58e074d0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d0030550 con 0x559d58e074d0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d0030c30 con 0x559d58e074d0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0030750 con 0x559d58e074d0 2024-09-21T02:03:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.032+0000 7fe4ef472740 10 client.28463 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.033+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559d58f0c620 con 0x559d58e074d0 2024-09-21T02:03:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.033+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0030c30 con 0x559d58e074d0 2024-09-21T02:03:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.033+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d0004a30 con 0x559d58e074d0 2024-09-21T02:03:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.034+0000 7fe4ef472740 10 client.28463 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.034+0000 7fe4ef472740 10 client.28463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.034+0000 7fe4ef472740 20 client.28463 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.034+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58de6d80 0x559d591ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.035+0000 7fe4ef472740 4 client.28463 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.035+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d59144640 con 0x559d58de6d80 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.035+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58de6d80 0x559d591ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.036+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58de6d80 0x559d591ef0c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe892395c11fcb server_cookie=43fd7d14de48f1c3 in_seq=0 out_seq=0 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.036+0000 7fe4c9ffb640 10 client.28463 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.036+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/376293157 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d59144640 con 0x559d58de6d80 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.036+0000 7fe4c9ffb640 10 client.28463 handle_command_reply: tid=0 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.036+0000 7fe4ef472740 1 client.28463 shutdown 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 msgr2=0x559d591e24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591e24b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0035040 tx=0x7fe4d00303b0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 shutdown_connections 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58de6d80 0x559d591ef0c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58e281b0 0x559d591e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4c9ffb640 0 client.28463 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e074d0 0x559d591e24b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 --2- 172.21.15.29:0/376293157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58dff2f0 0x559d591e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 >> 172.21.15.29:0/376293157 conn(0x559d58e1c300 msgr2=0x559d591c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 shutdown_connections 2024-09-21T02:03:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 1 -- 172.21.15.29:0/376293157 wait complete. 2024-09-21T02:03:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.037+0000 7fe4ef472740 20 client.28463 trim_cache size 0 max 16384 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 Processor -- start 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 -- start start 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e1c300 0x559d591ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58de6d80 0x559d591d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01a8cd0 con 0x559d58de6d80 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01aa810 con 0x559d58e1c300 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.056+0000 7fe4ef472740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01a93a0 con 0x559d58dff2f0 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58de6d80 0x559d591d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48064/0 (socket says 172.21.15.29:48064) 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3390271097 learned_addr learned my addr 172.21.15.29:0/3390271097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e1c300 0x559d591ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3390271097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e1c300 msgr2=0x559d591ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e1c300 0x559d591ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3390271097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58de6d80 msgr2=0x559d591d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58de6d80 0x559d591d94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.057+0000 7fe4e69c2640 1 -- 172.21.15.29:0/3390271097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d591e0e00 con 0x559d58dff2f0 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.058+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c760 tx=0x7fe4d8005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f059beb6e68c8e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.058+0000 7fe4cbfff640 1 -- 172.21.15.29:0/3390271097 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80b2070 con 0x559d58dff2f0 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.058+0000 7fe4cbfff640 1 -- 172.21.15.29:0/3390271097 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d80d10e0 con 0x559d58dff2f0 2024-09-21T02:03:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.058+0000 7fe4cbfff640 1 -- 172.21.15.29:0/3390271097 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80a9cc0 con 0x559d58dff2f0 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 -- 172.21.15.29:0/3390271097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 msgr2=0x559d591dc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c760 tx=0x7fe4d8005480 comp rx=0 tx=0).stop 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 -- 172.21.15.29:0/3390271097 shutdown_connections 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58dff2f0 0x559d591dc770 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58de6d80 0x559d591d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 --2- 172.21.15.29:0/3390271097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58e1c300 0x559d591ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 -- 172.21.15.29:0/3390271097 >> 172.21.15.29:0/3390271097 conn(0x559d591d8f30 msgr2=0x559d591e1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 -- 172.21.15.29:0/3390271097 shutdown_connections 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.059+0000 7fe4ef472740 1 -- 172.21.15.29:0/3390271097 wait complete. 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 -- start start 2024-09-21T02:03:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 Processor -- start 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 0x559d591e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58e1c300 0x559d591e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01aa140 con 0x559d58e1c300 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01a9a70 con 0x559d58dff2f0 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01a6cb0 con 0x559d58de6d80 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4ef472740 10 client.0 _ll_register_callbacks cb 0x7fe4dc0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e61c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 0x559d591e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e59c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:53404/0 (socket says 172.21.15.29:53404) 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e61c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 0x559d591e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:48066/0 (socket says 172.21.15.29:48066) 2024-09-21T02:03:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e59c0640 1 -- 172.21.15.29:0/618381866 learned_addr learned my addr 172.21.15.29:0/618381866 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.060+0000 7fe4e69c2640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58e1c300 0x559d591e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 -- 172.21.15.29:0/618381866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 msgr2=0x559d591e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 0x559d591e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 -- 172.21.15.29:0/618381866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58e1c300 msgr2=0x559d591e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58e1c300 0x559d591e8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 -- 172.21.15.29:0/618381866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d591e8dd0 con 0x559d58dff2f0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4e59c0640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0004650 tx=0x7fe4d0030e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e2d4004411ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0037040 con 0x559d58dff2f0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d80d10e0 con 0x559d58dff2f0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d00046c0 con 0x559d58dff2f0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe4d00096b0 con 0x559d58dff2f0 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4ef472740 10 client.28469 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559d591e7c60 con 0x559d58dff2f0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.061+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d80d10e0 con 0x559d58dff2f0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d00035f0 con 0x559d58dff2f0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 10 client.28469 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 10 client.28469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 20 client.28469 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58e074d0 0x559d5925beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 4 client.28469 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d591400f0 con 0x559d58e074d0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58e074d0 0x559d5925beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4e61c1640 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58e074d0 0x559d5925beb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b727014c66f34eac server_cookie=c05f54166398e62c in_seq=0 out_seq=0 2024-09-21T02:03:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.062+0000 7fe4c9ffb640 10 client.28469 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.070+0000 7fe4c9ffb640 1 -- 172.21.15.29:0/618381866 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d591400f0 con 0x559d58e074d0 2024-09-21T02:03:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.070+0000 7fe4c9ffb640 10 client.28469 handle_command_reply: tid=0 2024-09-21T02:03:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.070+0000 7fe4ef472740 1 client.28469 shutdown 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 msgr2=0x559d591e9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0004650 tx=0x7fe4d0030e90 comp rx=0 tx=0).stop 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 shutdown_connections 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x559d58e074d0 0x559d5925beb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4c9ffb640 0 client.28469 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559d58e1c300 0x559d591e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d58dff2f0 0x559d591e9e70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 --2- 172.21.15.29:0/618381866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d58de6d80 0x559d591e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 >> 172.21.15.29:0/618381866 conn(0x559d591d8f30 msgr2=0x559d591d94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 shutdown_connections 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 1 -- 172.21.15.29:0/618381866 wait complete. 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.071+0000 7fe4ef472740 20 client.28469 trim_cache size 0 max 16384 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:03:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 msgr2=0x7fe4cc0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 0x7fe4cc0b8a90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005fd0 tx=0x7fe4dc009260 comp rx=0 tx=0).stop 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 msgr2=0x7fe4e0198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800eb10 tx=0x7fe4d800ef10 comp rx=0 tx=0).stop 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 shutdown_connections 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe4cc0b6580 0x7fe4cc0b8a90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e0103ff0 0x7fe4e0198f60 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4e0103650 0x7fe4e01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 --2- 172.21.15.29:0/3267977088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e01025d0 0x7fe4e0197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 >> 172.21.15.29:0/3267977088 conn(0x7fe4e00fdf30 msgr2=0x7fe4e0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 shutdown_connections 2024-09-21T02:03:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:27.072+0000 7fe4e8425640 1 -- 172.21.15.29:0/3267977088 wait complete. 2024-09-21T02:03:27.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:26 smithi029 ceph-mon[31228]: scrub summary: idle 2024-09-21T02:03:27.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:26 smithi190 ceph-mon[34368]: scrub summary: idle 2024-09-21T02:03:27.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:26 smithi066 ceph-mon[35001]: scrub summary: idle 2024-09-21T02:03:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 890.0 s, 1438.0 tps, lat 70.033 ms stddev 73.649 2024-09-21T02:03:28.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:27 smithi190 ceph-mon[34368]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-09-21T02:03:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:27 smithi066 ceph-mon[35001]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-09-21T02:03:28.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:27 smithi029 ceph-mon[31228]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-09-21T02:03:30.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:29 smithi190 ceph-mon[34368]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-09-21T02:03:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:29 smithi066 ceph-mon[35001]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-09-21T02:03:30.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:29 smithi029 ceph-mon[31228]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-09-21T02:03:32.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:31 smithi190 ceph-mon[34368]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-09-21T02:03:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:31 smithi066 ceph-mon[35001]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-09-21T02:03:32.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:31 smithi029 ceph-mon[31228]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-09-21T02:03:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 895.0 s, 1579.6 tps, lat 62.611 ms stddev 73.369 2024-09-21T02:03:34.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:33 smithi190 ceph-mon[34368]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-21T02:03:34.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:33 smithi066 ceph-mon[35001]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-21T02:03:34.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:33 smithi029 ceph-mon[31228]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-21T02:03:36.146 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:35 smithi029 ceph-mon[31228]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-09-21T02:03:36.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:35 smithi190 ceph-mon[34368]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-09-21T02:03:36.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:35 smithi066 ceph-mon[35001]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-09-21T02:03:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:progress: 900.0 s, 1331.4 tps, lat 75.769 ms stddev 82.338 2024-09-21T02:03:38.186 INFO:teuthology.orchestra.run.smithi029.stdout:transaction type: 2024-09-21T02:03:38.186 INFO:teuthology.orchestra.run.smithi029.stdout:scaling factor: 32 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:query mode: simple 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:number of clients: 100 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:number of threads: 4 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:duration: 900 s 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:number of transactions actually processed: 2346927 2024-09-21T02:03:38.187 INFO:teuthology.orchestra.run.smithi029.stdout:latency average = 38.346 ms 2024-09-21T02:03:38.188 INFO:teuthology.orchestra.run.smithi029.stdout:latency stddev = 99.277 ms 2024-09-21T02:03:38.188 INFO:teuthology.orchestra.run.smithi029.stdout:tps = 2607.064600 (including connections establishing) 2024-09-21T02:03:38.188 INFO:teuthology.orchestra.run.smithi029.stdout:tps = 2607.071858 (excluding connections establishing) 2024-09-21T02:03:38.188 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:37 smithi029 ceph-mon[31228]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 689 op/s 2024-09-21T02:03:38.190 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-21T02:03:38.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:37 smithi190 ceph-mon[34368]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 689 op/s 2024-09-21T02:03:38.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:37 smithi066 ceph-mon[35001]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 689 op/s 2024-09-21T02:03:40.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:39 smithi190 ceph-mon[34368]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 559 op/s 2024-09-21T02:03:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.235+0000 7f308cab6640 1 -- 172.21.15.29:0/198660296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810aff0 msgr2=0x7f308810b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.235+0000 7f308cab6640 1 --2- 172.21.15.29:0/198660296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810aff0 0x7f308810b4c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3078008570 tx=0x7f307802e3b0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.242+0000 7f308cab6640 1 -- 172.21.15.29:0/198660296 shutdown_connections 2024-09-21T02:03:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.242+0000 7f308cab6640 1 --2- 172.21.15.29:0/198660296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308810bc00 0x7f3088112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.242+0000 7f308cab6640 1 --2- 172.21.15.29:0/198660296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810aff0 0x7f308810b4c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.242+0000 7f308cab6640 1 --2- 172.21.15.29:0/198660296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810a5a0 0x7f308810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.242+0000 7f308cab6640 1 -- 172.21.15.29:0/198660296 >> 172.21.15.29:0/198660296 conn(0x7f3088070c70 msgr2=0x7f3088073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.251+0000 7f308cab6640 1 -- 172.21.15.29:0/198660296 shutdown_connections 2024-09-21T02:03:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.254+0000 7f308cab6640 1 -- 172.21.15.29:0/198660296 wait complete. 2024-09-21T02:03:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.255+0000 7f308cab6640 1 Processor -- start 2024-09-21T02:03:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 -- start start 2024-09-21T02:03:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810a5a0 0x7f30881a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308810bc00 0x7f30881aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30881159c0 con 0x7f308810a5a0 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f308cab6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3088115840 con 0x7f308810bc00 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f3085d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f3085d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52548/0 (socket says 172.21.15.29:52548) 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.259+0000 7f3085d9b640 1 -- 172.21.15.29:0/619546197 learned_addr learned my addr 172.21.15.29:0/619546197 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3088115b40 con 0x7f308810aff0 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f308659c640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810a5a0 0x7f30881a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 -- 172.21.15.29:0/619546197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308810bc00 msgr2=0x7f30881aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308810bc00 0x7f30881aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 -- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810a5a0 msgr2=0x7f30881a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810a5a0 0x7f30881a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 -- 172.21.15.29:0/619546197 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30881aa890 con 0x7f308810aff0 2024-09-21T02:03:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.260+0000 7f3085d9b640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f307802e920 tx=0x7f307800f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.261+0000 7f30777fe640 1 -- 172.21.15.29:0/619546197 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780053d0 con 0x7f308810aff0 2024-09-21T02:03:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.261+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30881aab20 con 0x7f308810aff0 2024-09-21T02:03:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.261+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30881a04e0 con 0x7f308810aff0 2024-09-21T02:03:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.264+0000 7f3093a21740 1 Processor -- start 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 -- start start 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c65010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5561b4bd66f0 con 0x5561b48932f0 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561b4bc5b20 con 0x5561b489b4d0 2024-09-21T02:03:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.265+0000 7f3093a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561b4bc9930 con 0x5561b48bc1b0 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:52556/0 (socket says 172.21.15.29:52556) 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 -- 172.21.15.29:0/380381541 learned_addr learned my addr 172.21.15.29:0/380381541 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f30777fe640 1 -- 172.21.15.29:0/619546197 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307803a030 con 0x7f308810aff0 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 -- 172.21.15.29:0/380381541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 msgr2=0x5561b4c65010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c65010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 -- 172.21.15.29:0/380381541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 msgr2=0x5561b4c644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f3086d9d640 1 -- 172.21.15.29:0/380381541 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b4b8b530 con 0x5561b48bc1b0 2024-09-21T02:03:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.266+0000 7f30777fe640 1 -- 172.21.15.29:0/619546197 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078036440 con 0x7f308810aff0 2024-09-21T02:03:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.270+0000 7f30777fe640 1 -- 172.21.15.29:0/619546197 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f30780365e0 con 0x7f308810aff0 2024-09-21T02:03:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.270+0000 7f3086d9d640 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f307c009760 tx=0x7f307c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a6cef5321fcd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.271+0000 7f30777fe640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 0x7f30700b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.271+0000 7f30777fe640 1 -- 172.21.15.29:0/619546197 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f30781075b0 con 0x7f308810aff0 2024-09-21T02:03:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.272+0000 7f3074ff9640 1 -- 172.21.15.29:0/380381541 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c015020 con 0x5561b48bc1b0 2024-09-21T02:03:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.272+0000 7f3074ff9640 1 -- 172.21.15.29:0/380381541 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c011420 con 0x5561b48bc1b0 2024-09-21T02:03:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.272+0000 7f308659c640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 0x7f30700b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.273+0000 7f308659c640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 0x7f30700b8b60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3080005fd0 tx=0x7f3080009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.274+0000 7f3074ff9640 1 -- 172.21.15.29:0/380381541 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0105c0 con 0x5561b48bc1b0 2024-09-21T02:03:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 -- 172.21.15.29:0/380381541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 msgr2=0x5561b4c63e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f307c009760 tx=0x7f307c009c20 comp rx=0 tx=0).stop 2024-09-21T02:03:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 -- 172.21.15.29:0/380381541 shutdown_connections 2024-09-21T02:03:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c65010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/380381541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c63e50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.279+0000 7f3093a21740 1 -- 172.21.15.29:0/380381541 >> 172.21.15.29:0/380381541 conn(0x5561b48b0300 msgr2=0x5561b4c5f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.283+0000 7f3093a21740 1 -- 172.21.15.29:0/380381541 shutdown_connections 2024-09-21T02:03:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.283+0000 7f3093a21740 1 -- 172.21.15.29:0/380381541 wait complete. 2024-09-21T02:03:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.283+0000 7f3093a21740 1 -- start start 2024-09-21T02:03:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.283+0000 7f3093a21740 1 Processor -- start 2024-09-21T02:03:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.284+0000 7f3093a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.284+0000 7f3093a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c6dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.284+0000 7f3093a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.284+0000 7f3093a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5561b4afdab0 con 0x5561b48932f0 2024-09-21T02:03:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3093a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5561b4b6b730 con 0x5561b48bc1b0 2024-09-21T02:03:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3093a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561b4c65b30 con 0x5561b489b4d0 2024-09-21T02:03:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3093a21740 10 client.0 _ll_register_callbacks cb 0x5561b4bbf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3086d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3086d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49464/0 (socket says 172.21.15.29:49464) 2024-09-21T02:03:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3086d9d640 1 -- 172.21.15.29:0/619108842 learned_addr learned my addr 172.21.15.29:0/619108842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.285+0000 7f3085d9b640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c6dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 -- 172.21.15.29:0/619108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 msgr2=0x5561b4c6dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c6dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 -- 172.21.15.29:0/619108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 msgr2=0x5561b4c6d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 -- 172.21.15.29:0/619108842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b4927790 con 0x5561b48bc1b0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.286+0000 7f3086d9d640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f307c006ac0 tx=0x7f307c011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae4636744e32011 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.287+0000 7f306dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.287+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00bce0 con 0x5561b48bc1b0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.287+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c011420 con 0x5561b48bc1b0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.287+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f307c004020 con 0x5561b48bc1b0 2024-09-21T02:03:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.288+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f307c00be80 con 0x5561b48bc1b0 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.288+0000 7f3093a21740 10 client.28472 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.288+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5561b49a0620 con 0x5561b48bc1b0 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.288+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c011420 con 0x5561b48bc1b0 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f307c003c20 con 0x5561b48bc1b0 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 10 client.28472 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 10 client.28472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 20 client.28472 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b487ad80 0x5561b4c7b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 4 client.28472 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561b4bd8640 con 0x5561b487ad80 2024-09-21T02:03:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.289+0000 7f308659c640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b487ad80 0x5561b4c7b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:39 smithi066 ceph-mon[35001]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 559 op/s 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.294+0000 7f308659c640 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b487ad80 0x5561b4c7b1c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac90b1bbecf04e server_cookie=2df8fa07b1eacf1f in_seq=0 out_seq=0 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.294+0000 7f306dffb640 10 client.28472 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f306dffb640 1 -- 172.21.15.29:0/619108842 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561b4bd8640 con 0x5561b487ad80 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f306dffb640 10 client.28472 handle_command_reply: tid=0 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 client.28472 shutdown 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 msgr2=0x5561b4c6e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f307c006ac0 tx=0x7f307c011800 comp rx=0 tx=0).stop 2024-09-21T02:03:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 shutdown_connections 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b487ad80 0x5561b4c7b1c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48bc1b0 0x5561b4c6e8e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c6dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 --2- 172.21.15.29:0/619108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c6d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 >> 172.21.15.29:0/619108842 conn(0x5561b48b0300 msgr2=0x5561b4c5dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f306dffb640 0 client.28472 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.295+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 shutdown_connections 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.296+0000 7f3093a21740 1 -- 172.21.15.29:0/619108842 wait complete. 2024-09-21T02:03:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.298+0000 7f3093a21740 20 client.28472 trim_cache size 0 max 16384 2024-09-21T02:03:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.329+0000 7f3093a21740 1 Processor -- start 2024-09-21T02:03:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.329+0000 7f3093a21740 1 -- start start 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c6c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c6c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f306800e420 con 0x5561b48932f0 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f306800ff60 con 0x5561b489b4d0 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3093a21740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f306800eaf0 con 0x5561b48bc1b0 2024-09-21T02:03:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3085d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c6c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3085d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c6c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49466/0 (socket says 172.21.15.29:49466) 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3085d9b640 1 -- 172.21.15.29:0/3268455675 learned_addr learned my addr 172.21.15.29:0/3268455675 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.330+0000 7f3086d9d640 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c6c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f308659c640 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 -- 172.21.15.29:0/3268455675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 msgr2=0x5561b4c6c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c6c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 -- 172.21.15.29:0/3268455675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 msgr2=0x5561b4c7b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c7b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 -- 172.21.15.29:0/3268455675 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b487afb0 con 0x5561b48bc1b0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.332+0000 7f3086d9d640 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c6c8b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f307c001f70 tx=0x7f307c009270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f7bec64c42928 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.333+0000 7f306ffff640 1 -- 172.21.15.29:0/3268455675 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c015030 con 0x5561b48bc1b0 2024-09-21T02:03:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.333+0000 7f306ffff640 1 -- 172.21.15.29:0/3268455675 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c01f070 con 0x5561b48bc1b0 2024-09-21T02:03:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.334+0000 7f3093a21740 1 -- 172.21.15.29:0/3268455675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 msgr2=0x5561b4c6c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.334+0000 7f3093a21740 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c6c8b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f307c001f70 tx=0x7f307c009270 comp rx=0 tx=0).stop 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 -- 172.21.15.29:0/3268455675 shutdown_connections 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b48bc1b0 0x5561b4c6c8b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b489b4d0 0x5561b4c6c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 --2- 172.21.15.29:0/3268455675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48932f0 0x5561b4c7b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 -- 172.21.15.29:0/3268455675 >> 172.21.15.29:0/3268455675 conn(0x5561b48b0300 msgr2=0x5561b4c6e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 -- 172.21.15.29:0/3268455675 shutdown_connections 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.335+0000 7f3093a21740 1 -- 172.21.15.29:0/3268455675 wait complete. 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.336+0000 7f3093a21740 1 -- start start 2024-09-21T02:03:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.336+0000 7f3093a21740 1 Processor -- start 2024-09-21T02:03:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.336+0000 7f3093a21740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f3093a21740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c72170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f308659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f308659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:49480/0 (socket says 172.21.15.29:49480) 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f308659c640 1 -- 172.21.15.29:0/3983427286 learned_addr learned my addr 172.21.15.29:0/3983427286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f3085d9b640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c72170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48bc1b0 0x5561b4c72c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.337+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f306800f890 con 0x5561b48bc1b0 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.338+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f306800f1c0 con 0x5561b48932f0 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.338+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f306800c400 con 0x5561b489b4d0 2024-09-21T02:03:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.338+0000 7f3093a21740 10 client.0 _ll_register_callbacks cb 0x7f307c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.339+0000 7f308659c640 1 -- 172.21.15.29:0/3983427286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 msgr2=0x5561b4c72170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.339+0000 7f308659c640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c72170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.339+0000 7f308659c640 1 -- 172.21.15.29:0/3983427286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48bc1b0 msgr2=0x5561b4c72c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.339+0000 7f308659c640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48bc1b0 0x5561b4c72c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.339+0000 7f308659c640 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b4c733d0 con 0x5561b48932f0 2024-09-21T02:03:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.340+0000 7f308659c640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f308000d010 tx=0x7f3080018c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00fb50e6e33837a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.340+0000 7f306dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.340+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080030060 con 0x5561b48932f0 2024-09-21T02:03:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.340+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c01f070 con 0x5561b48932f0 2024-09-21T02:03:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.341+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3080016d90 con 0x5561b48932f0 2024-09-21T02:03:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.341+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3080018e60 con 0x5561b48932f0 2024-09-21T02:03:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.341+0000 7f3093a21740 10 client.28478 fetch_fsmap learned FSMap version 49 2024-09-21T02:03:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.341+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5561b4c747e0 con 0x5561b48932f0 2024-09-21T02:03:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.341+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c01f070 con 0x5561b48932f0 2024-09-21T02:03:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f3080025400 con 0x5561b48932f0 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 10 client.28478 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 10 client.28478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 20 client.28478 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b48501c0 0x5561b4c751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 4 client.28478 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.343+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561b4bd40f0 con 0x5561b48501c0 2024-09-21T02:03:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.344+0000 7f3085d9b640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b48501c0 0x5561b4c751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:39 smithi029 ceph-mon[31228]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 559 op/s 2024-09-21T02:03:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.348+0000 7f3085d9b640 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b48501c0 0x5561b4c751e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29ed016a6671289 server_cookie=ed9dac445ec1b30 in_seq=0 out_seq=0 2024-09-21T02:03:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:40.348+0000 7f306dffb640 10 client.28478 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:42.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:41 smithi190 ceph-mon[34368]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-09-21T02:03:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f306dffb640 1 -- 172.21.15.29:0/3983427286 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561b4bd40f0 con 0x5561b48501c0 2024-09-21T02:03:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f306dffb640 10 client.28478 handle_command_reply: tid=0 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 client.28478 shutdown 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 msgr2=0x5561b4c71ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f308000d010 tx=0x7f3080018c70 comp rx=0 tx=0).stop 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 shutdown_connections 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5561b48501c0 0x5561b4c751e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5561b48bc1b0 0x5561b4c72c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561b489b4d0 0x5561b4c72170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 --2- 172.21.15.29:0/3983427286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561b48932f0 0x5561b4c71ab0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.279+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 >> 172.21.15.29:0/3983427286 conn(0x5561b48b0300 msgr2=0x5561b4c6e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.280+0000 7f306dffb640 0 client.28478 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.280+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 shutdown_connections 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.280+0000 7f3093a21740 1 -- 172.21.15.29:0/3983427286 wait complete. 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.280+0000 7f3093a21740 20 client.28478 trim_cache size 0 max 16384 2024-09-21T02:03:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:{ 2024-09-21T02:03:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "message": "", 2024-09-21T02:03:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout: "return_code": 0 2024-09-21T02:03:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stdout:} 2024-09-21T02:03:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 msgr2=0x7f30700b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 0x7f30700b8b60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3080005fd0 tx=0x7f3080009260 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 msgr2=0x7f30881a94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f307802e920 tx=0x7f307800f310 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 shutdown_connections 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f30700b6650 0x7f30700b8b60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308810bc00 0x7f30881aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f308810aff0 0x7f30881a94c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 --2- 172.21.15.29:0/619546197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f308810a5a0 0x7f30881a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 >> 172.21.15.29:0/619546197 conn(0x7f3088070c70 msgr2=0x7f308810e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 shutdown_connections 2024-09-21T02:03:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi029.front.sepia.ceph.com.smithi029.stderr:2024-09-21T02:03:42.281+0000 7f308cab6640 1 -- 172.21.15.29:0/619546197 wait complete. 2024-09-21T02:03:42.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:41 smithi066 ceph-mon[35001]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-09-21T02:03:42.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:41 smithi029 ceph-mon[31228]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-09-21T02:03:43.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:42 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:43.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:42 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:43.342 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:03:43.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:42 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:43.881 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:44.227 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:43 smithi029 ceph-mon[31228]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-09-21T02:03:44.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:43 smithi190 ceph-mon[34368]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-09-21T02:03:44.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:43 smithi066 ceph-mon[35001]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.554+0000 7fe06d390640 1 -- 172.21.15.29:0/1680662159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06811cba0 msgr2=0x7fe06811efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.554+0000 7fe06d390640 1 --2- 172.21.15.29:0/1680662159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06811cba0 0x7fe06811efc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe05802a420 comp rx=0 tx=0).stop 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.556+0000 7fe06d390640 1 -- 172.21.15.29:0/1680662159 shutdown_connections 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.556+0000 7fe06d390640 1 --2- 172.21.15.29:0/1680662159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06811cba0 0x7fe06811efc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.556+0000 7fe06d390640 1 --2- 172.21.15.29:0/1680662159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06810ab80 0x7fe06810af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.556+0000 7fe06d390640 1 --2- 172.21.15.29:0/1680662159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe068111d40 0x7fe06810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.556+0000 7fe06d390640 1 -- 172.21.15.29:0/1680662159 >> 172.21.15.29:0/1680662159 conn(0x7fe06806d950 msgr2=0x7fe06806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.559+0000 7fe06d390640 1 -- 172.21.15.29:0/1680662159 shutdown_connections 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.561+0000 7fe06d390640 1 -- 172.21.15.29:0/1680662159 wait complete. 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.562+0000 7fe06d390640 1 Processor -- start 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.563+0000 7fe06d390640 1 -- start start 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.563+0000 7fe06d390640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06810ab80 0x7fe06811c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.563+0000 7fe06d390640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe068111d40 0x7fe06811c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.564+0000 7fe06d390640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.564+0000 7fe06d390640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe068112850 con 0x7fe06810ab80 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.564+0000 7fe06d390640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681126d0 con 0x7fe068111d40 2024-09-21T02:03:44.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.564+0000 7fe06d390640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681129d0 con 0x7fe06811cba0 2024-09-21T02:03:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.565+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06810ab80 0x7fe06811c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.565+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.565+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:43412/0 (socket says 172.21.15.29:43412) 2024-09-21T02:03:44.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.565+0000 7fe0677fe640 1 -- 172.21.15.29:0/1376503774 learned_addr learned my addr 172.21.15.29:0/1376503774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.566+0000 7fe0677fe640 1 -- 172.21.15.29:0/1376503774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe068111d40 msgr2=0x7fe06811c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.566+0000 7fe0677fe640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe068111d40 0x7fe06811c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.566+0000 7fe0677fe640 1 -- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06810ab80 msgr2=0x7fe06811c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.566+0000 7fe0677fe640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06810ab80 0x7fe06811c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.566+0000 7fe0677fe640 1 -- 172.21.15.29:0/1376503774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe068116d10 con 0x7fe06811cba0 2024-09-21T02:03:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.567+0000 7fe0677fe640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe05802a990 tx=0x7fe058003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.568+0000 7fe04ffff640 1 -- 172.21.15.29:0/1376503774 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05803f070 con 0x7fe06811cba0 2024-09-21T02:03:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.568+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe068116fa0 con 0x7fe06811cba0 2024-09-21T02:03:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.568+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe068128460 con 0x7fe06811cba0 2024-09-21T02:03:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.568+0000 7fe04ffff640 1 -- 172.21.15.29:0/1376503774 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe058003e40 con 0x7fe06811cba0 2024-09-21T02:03:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.568+0000 7fe04ffff640 1 -- 172.21.15.29:0/1376503774 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058035030 con 0x7fe06811cba0 2024-09-21T02:03:44.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.573+0000 7fe073904740 1 Processor -- start 2024-09-21T02:03:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.575+0000 7fe04ffff640 1 -- 172.21.15.29:0/1376503774 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe058042070 con 0x7fe06811cba0 2024-09-21T02:03:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.576+0000 7fe04ffff640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 0x7fe0500b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.576+0000 7fe04ffff640 1 -- 172.21.15.29:0/1376503774 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe0581054f0 con 0x7fe06811cba0 2024-09-21T02:03:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 -- start start 2024-09-21T02:03:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 0x5603d1676d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 0x5603d1677290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5603d15ece50 con 0x5603d12a61b0 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603d15dc280 con 0x5603d12cf070 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe073904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5603d15e0090 con 0x5603d12ae390 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 0x5603d1677290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40768/0 (socket says 172.21.15.29:40768) 2024-09-21T02:03:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.577+0000 7fe066ffd640 1 -- 172.21.15.29:0/2360624355 learned_addr learned my addr 172.21.15.29:0/2360624355 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe0667fc640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 0x7fe0500b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe0667fc640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 0x5603d1676d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe066ffd640 1 -- 172.21.15.29:0/2360624355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 msgr2=0x5603d1677290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe066ffd640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 0x5603d1677290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe066ffd640 1 -- 172.21.15.29:0/2360624355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 msgr2=0x5603d1676d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe066ffd640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 0x5603d1676d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.579+0000 7fe066ffd640 1 -- 172.21.15.29:0/2360624355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d133a650 con 0x5603d12cf070 2024-09-21T02:03:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.580+0000 7fe0677fe640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 0x5603d1677290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.580+0000 7fe0667fc640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 0x5603d1676d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.580+0000 7fe066ffd640 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d60 tx=0x7fe06000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12fddb4ae49919d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.583+0000 7fe04d7fa640 1 -- 172.21.15.29:0/2360624355 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06001a480 con 0x5603d12cf070 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.583+0000 7fe04d7fa640 1 -- 172.21.15.29:0/2360624355 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0600118f0 con 0x5603d12cf070 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 -- 172.21.15.29:0/2360624355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 msgr2=0x5603d1676810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d60 tx=0x7fe06000f730 comp rx=0 tx=0).stop 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe0667fc640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 0x7fe0500b8bb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5603d1675490 tx=0x7fe05c005cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 -- 172.21.15.29:0/2360624355 shutdown_connections 2024-09-21T02:03:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12ae390 0x5603d1677290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12a61b0 0x5603d1676d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 --2- 172.21.15.29:0/2360624355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12cf070 0x5603d1676810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 -- 172.21.15.29:0/2360624355 >> 172.21.15.29:0/2360624355 conn(0x5603d12c3460 msgr2=0x5603d1674700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 -- 172.21.15.29:0/2360624355 shutdown_connections 2024-09-21T02:03:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.584+0000 7fe073904740 1 -- 172.21.15.29:0/2360624355 wait complete. 2024-09-21T02:03:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.585+0000 7fe073904740 1 -- start start 2024-09-21T02:03:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.585+0000 7fe073904740 1 Processor -- start 2024-09-21T02:03:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1684650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1681b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5603d143d700 con 0x5603d12ae390 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603d1577110 con 0x5603d128d9f0 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5603d1685040 con 0x5603d12a61b0 2024-09-21T02:03:44.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.586+0000 7fe073904740 10 client.0 _ll_register_callbacks cb 0x5603d1590990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:44.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.587+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.587+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40780/0 (socket says 172.21.15.29:40780) 2024-09-21T02:03:44.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.587+0000 7fe0677fe640 1 -- 172.21.15.29:0/15328017 learned_addr learned my addr 172.21.15.29:0/15328017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:44.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.588+0000 7fe066ffd640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1684650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.588+0000 7fe0667fc640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1681b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.589+0000 7fe0677fe640 1 -- 172.21.15.29:0/15328017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 msgr2=0x5603d1684650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.589+0000 7fe0677fe640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1684650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.589+0000 7fe0677fe640 1 -- 172.21.15.29:0/15328017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 msgr2=0x5603d1681b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.589+0000 7fe0677fe640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1681b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.589+0000 7fe0677fe640 1 -- 172.21.15.29:0/15328017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d15a4c40 con 0x5603d128d9f0 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.590+0000 7fe0677fe640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0580caae0 tx=0x7fe0580c8100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392d191c9ae395af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.590+0000 7fe04aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580ca260 con 0x5603d128d9f0 2024-09-21T02:03:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603d15eeda0 con 0x5603d128d9f0 2024-09-21T02:03:44.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe0580d2120 con 0x5603d128d9f0 2024-09-21T02:03:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe0580ca400 con 0x5603d128d9f0 2024-09-21T02:03:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe073904740 10 client.28484 fetch_fsmap learned FSMap version 50 2024-09-21T02:03:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.592+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5603d1690a40 con 0x5603d128d9f0 2024-09-21T02:03:44.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.593+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603d15eeda0 con 0x5603d128d9f0 2024-09-21T02:03:44.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.594+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fe0580b8030 con 0x5603d128d9f0 2024-09-21T02:03:44.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.594+0000 7fe073904740 10 client.28484 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T02:03:44.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.594+0000 7fe073904740 10 client.28484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:44.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.594+0000 7fe073904740 20 client.28484 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:44.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.595+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d1262da0 0x5603d16940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.595+0000 7fe073904740 4 client.28484 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:44.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.595+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d15ea850 con 0x5603d1262da0 2024-09-21T02:03:44.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.599+0000 7fe066ffd640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d1262da0 0x5603d16940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.599+0000 7fe066ffd640 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d1262da0 0x5603d16940f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecff2a4994699afb server_cookie=3b280956965af83 in_seq=0 out_seq=0 2024-09-21T02:03:44.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.599+0000 7fe04aff5640 10 client.28484 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:44.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.600+0000 7fe04aff5640 1 -- 172.21.15.29:0/15328017 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603d15ea850 con 0x5603d1262da0 2024-09-21T02:03:44.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.600+0000 7fe04aff5640 10 client.28484 handle_command_reply: tid=0 2024-09-21T02:03:44.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 client.28484 shutdown 2024-09-21T02:03:44.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 msgr2=0x5603d1682270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0580caae0 tx=0x7fe0580c8100 comp rx=0 tx=0).stop 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 shutdown_connections 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d1262da0 0x5603d16940f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d1682270 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1681b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 --2- 172.21.15.29:0/15328017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1684650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 >> 172.21.15.29:0/15328017 conn(0x5603d12c3460 msgr2=0x5603d16799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe04aff5640 0 client.28484 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.601+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 shutdown_connections 2024-09-21T02:03:44.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.602+0000 7fe073904740 1 -- 172.21.15.29:0/15328017 wait complete. 2024-09-21T02:03:44.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.604+0000 7fe073904740 20 client.28484 trim_cache size 0 max 16384 2024-09-21T02:03:44.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.641+0000 7fe073904740 1 Processor -- start 2024-09-21T02:03:44.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe073904740 1 -- start start 2024-09-21T02:03:44.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe073904740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d128d9f0 0x5603d167d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe073904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe073904740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12ae390 0x5603d16998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe073904740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe05400e530 con 0x5603d128d9f0 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.642+0000 7fe0667fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.29:43444/0 (socket says 172.21.15.29:43444) 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 -- 172.21.15.29:0/3677963687 learned_addr learned my addr 172.21.15.29:0/3677963687 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0677fe640 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12ae390 0x5603d16998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe054010070 con 0x5603d12ae390 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe05400ec00 con 0x5603d12a61b0 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe066ffd640 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d128d9f0 0x5603d167d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 -- 172.21.15.29:0/3677963687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12ae390 msgr2=0x5603d16998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12ae390 0x5603d16998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 -- 172.21.15.29:0/3677963687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d128d9f0 msgr2=0x5603d167d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d128d9f0 0x5603d167d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.643+0000 7fe0667fc640 1 -- 172.21.15.29:0/3677963687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d16827d0 con 0x5603d12a61b0 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.644+0000 7fe0667fc640 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00edc0 tx=0x7fe05c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869cfd6e3f12e16b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.646+0000 7fe04cff9640 1 -- 172.21.15.29:0/3677963687 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c015040 con 0x5603d12a61b0 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.646+0000 7fe04cff9640 1 -- 172.21.15.29:0/3677963687 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c00edf0 con 0x5603d12a61b0 2024-09-21T02:03:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.646+0000 7fe04cff9640 1 -- 172.21.15.29:0/3677963687 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c01e480 con 0x5603d12a61b0 2024-09-21T02:03:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.647+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 msgr2=0x5603d1683750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.647+0000 7fe073904740 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00edc0 tx=0x7fe05c011000 comp rx=0 tx=0).stop 2024-09-21T02:03:44.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.651+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 shutdown_connections 2024-09-21T02:03:44.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.651+0000 7fe073904740 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d12ae390 0x5603d16998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.651+0000 7fe073904740 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1683750 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.651+0000 7fe073904740 1 --2- 172.21.15.29:0/3677963687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d128d9f0 0x5603d167d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.651+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 >> 172.21.15.29:0/3677963687 conn(0x5603d12c3460 msgr2=0x5603d1683040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.656+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 shutdown_connections 2024-09-21T02:03:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.656+0000 7fe073904740 1 -- 172.21.15.29:0/3677963687 wait complete. 2024-09-21T02:03:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.656+0000 7fe073904740 1 -- start start 2024-09-21T02:03:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.657+0000 7fe073904740 1 Processor -- start 2024-09-21T02:03:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.657+0000 7fe073904740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d168aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.658+0000 7fe073904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1687ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.658+0000 7fe073904740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1688430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.658+0000 7fe073904740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe05400f9a0 con 0x5603d12ae390 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.658+0000 7fe073904740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe05400f2d0 con 0x5603d128d9f0 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.658+0000 7fe073904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe05400c510 con 0x5603d12a61b0 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.659+0000 7fe073904740 10 client.0 _ll_register_callbacks cb 0x7fe058092270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.659+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d168aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.659+0000 7fe0667fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1687ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.659+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d168aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40804/0 (socket says 172.21.15.29:40804) 2024-09-21T02:03:44.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.659+0000 7fe066ffd640 1 -- 172.21.15.29:0/4171064347 learned_addr learned my addr 172.21.15.29:0/4171064347 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.661+0000 7fe0667fc640 1 -- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 msgr2=0x5603d168aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.661+0000 7fe0667fc640 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d168aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.661+0000 7fe0667fc640 1 -- 172.21.15.29:0/4171064347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 msgr2=0x5603d1688430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.661+0000 7fe0667fc640 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1688430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.661+0000 7fe0667fc640 1 -- 172.21.15.29:0/4171064347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d1688aa0 con 0x5603d12a61b0 2024-09-21T02:03:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.663+0000 7fe0667fc640 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1687ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0059b0 tx=0x7fe05c00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129b348dccc22fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.663+0000 7fe04aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.663+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c015070 con 0x5603d12a61b0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.664+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe05c003bd0 con 0x5603d12a61b0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.664+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c00edf0 con 0x5603d12a61b0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.664+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe05c003df0 con 0x5603d12a61b0 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.664+0000 7fe073904740 10 client.28477 fetch_fsmap learned FSMap version 50 2024-09-21T02:03:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.664+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5603d16fe8f0 con 0x5603d12a61b0 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.665+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c004440 con 0x5603d12a61b0 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.666+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fe05c01b050 con 0x5603d12a61b0 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.666+0000 7fe073904740 10 client.28477 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.666+0000 7fe073904740 10 client.28477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.666+0000 7fe073904740 20 client.28477 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.666+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d16fef70 0x5603d1701390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.667+0000 7fe073904740 4 client.28477 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:03:44.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.667+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe05c00edf0 con 0x5603d16fef70 2024-09-21T02:03:44.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.668+0000 7fe066ffd640 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d16fef70 0x5603d1701390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.669+0000 7fe066ffd640 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d16fef70 0x5603d1701390 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471ed602566e3220 server_cookie=e5d1f5acdf6f1b1 in_seq=0 out_seq=0 2024-09-21T02:03:44.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.669+0000 7fe04aff5640 10 client.28477 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:44.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.669+0000 7fe04aff5640 1 -- 172.21.15.29:0/4171064347 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x7fe05c00edf0 con 0x5603d16fef70 2024-09-21T02:03:44.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.669+0000 7fe04aff5640 10 client.28477 handle_command_reply: tid=0 2024-09-21T02:03:44.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.669+0000 7fe073904740 1 client.28477 shutdown 2024-09-21T02:03:44.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 msgr2=0x5603d1687ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1687ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0059b0 tx=0x7fe05c00eba0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 shutdown_connections 2024-09-21T02:03:44.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x5603d16fef70 0x5603d1701390 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5603d12ae390 0x5603d1688430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe04aff5640 0 client.28477 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603d12a61b0 0x5603d1687ef0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 --2- 172.21.15.29:0/4171064347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603d128d9f0 0x5603d168aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 >> 172.21.15.29:0/4171064347 conn(0x5603d12c3460 msgr2=0x5603d1682f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.674+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 shutdown_connections 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.675+0000 7fe073904740 1 -- 172.21.15.29:0/4171064347 wait complete. 2024-09-21T02:03:44.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.675+0000 7fe073904740 20 client.28477 trim_cache size 0 max 16384 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "status": "no active scrubs running", 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "scrubs": {}, 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_stats": { 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "be119495-8eb8-47dd-82fd-e7b815bff586": { 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:03:10.508986+0000", 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.679 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "c48dc890-9260-4f8f-b08e-1fd9d06fa732": { 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:02:20.247270+0000", 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.680 INFO:teuthology.orchestra.run.smithi029.stdout: "5aa0003d-8663-4473-83a4-1d4c3aa57a8a": { 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:59:44.166167+0000", 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "49a9abae-6e84-4db7-9902-1340c785ba9b": { 2024-09-21T02:03:44.681 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:00:37.380859+0000", 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "fab85dd2-3e0e-4f9f-9c88-39d29141ba22": { 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:49:16.190285+0000", 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "fd9b75aa-5267-413c-8378-05fbd6a25987": { 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:05.966516+0000", 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "8a12d9b0-8fa5-4a08-844f-7ed03505aeb3": { 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:48:24.046653+0000", 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.683 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "91217b03-65df-4a37-8426-4ace79f8bff9": { 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:50:13.861880+0000", 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "90602b42-4b81-4751-a9a5-c37e2d79d262": { 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:51:58.477151+0000", 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.684 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "5313c304-821b-4232-9273-31da1c2b71bf": { 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T02:01:28.547564+0000", 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "6fc36252-80c6-4cdc-b5bd-92ba0ba9585d": { 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:52:51.843780+0000", 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.685 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "3657cfac-3db2-43e1-a67e-58a77fb4d032": { 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:54:34.549069+0000", 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.686 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "7e20e03c-6c73-4734-b7ca-ec40b4b4216e": { 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:56:17.969398+0000", 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.687 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "4700a670-a6ec-4427-bbc4-e0c61de06cab": { 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:53:42.319428+0000", 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "cfaef6f1-6b63-4cba-acca-61015f7b83fc": { 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:55:26.323726+0000", 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.688 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "399d6a80-0726-417c-8b65-467fd131cefd": { 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:08.149091+0000", 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "f7547296-cdb5-408c-b734-10250cb2b816": { 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:57:59.384777+0000", 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "40467f92-f1af-4b93-ac87-06d10971b4d5": { 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "start_time": "2024-09-21T01:58:52.533467+0000", 2024-09-21T02:03:44.690 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/", 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_passed": 0, 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_failed": 0, 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_skipped": 0 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:03:44.691 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.676+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 msgr2=0x7fe0500b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.676+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 0x7fe0500b8bb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5603d1675490 tx=0x7fe05c005cd0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.676+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 msgr2=0x7fe0681164f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe05802a990 tx=0x7fe058003cd0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 shutdown_connections 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0500b66a0 0x7fe0500b8bb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06811cba0 0x7fe0681164f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe068111d40 0x7fe06811c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 --2- 172.21.15.29:0/1376503774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe06810ab80 0x7fe06811c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 >> 172.21.15.29:0/1376503774 conn(0x7fe06806d950 msgr2=0x7fe06811d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 shutdown_connections 2024-09-21T02:03:44.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:44.677+0000 7fe06d390640 1 -- 172.21.15.29:0/1376503774 wait complete. 2024-09-21T02:03:45.688 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:03:45.689 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:03:46.023 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:46.172 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:45 smithi029 ceph-mon[31228]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 711 op/s 2024-09-21T02:03:46.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:45 smithi190 ceph-mon[34368]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 711 op/s 2024-09-21T02:03:46.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:45 smithi066 ceph-mon[35001]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 711 op/s 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 -- 172.21.15.29:0/3597682874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 msgr2=0x7f9d7010bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 --2- 172.21.15.29:0/3597682874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d7010bf00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c370 tx=0x7f9d60029f90 comp rx=0 tx=0).stop 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 -- 172.21.15.29:0/3597682874 shutdown_connections 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 --2- 172.21.15.29:0/3597682874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d70110490 0x7f9d7007b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 --2- 172.21.15.29:0/3597682874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d7010fb40 0x7f9d7010ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 --2- 172.21.15.29:0/3597682874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d7010bf00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 -- 172.21.15.29:0/3597682874 >> 172.21.15.29:0/3597682874 conn(0x7f9d70070ec0 msgr2=0x7f9d700712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:46.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 -- 172.21.15.29:0/3597682874 shutdown_connections 2024-09-21T02:03:46.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.642+0000 7f9d763be640 1 -- 172.21.15.29:0/3597682874 wait complete. 2024-09-21T02:03:46.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.643+0000 7f9d763be640 1 Processor -- start 2024-09-21T02:03:46.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.643+0000 7f9d763be640 1 -- start start 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d7010fb40 0x7f9d70076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d70110490 0x7f9d701c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7007e2a0 con 0x7f9d70110490 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7007e120 con 0x7f9d7010baf0 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.644+0000 7f9d763be640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7007e420 con 0x7f9d7010fb40 2024-09-21T02:03:46.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.645+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.645+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40816/0 (socket says 172.21.15.29:40816) 2024-09-21T02:03:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.645+0000 7f9d6ffff640 1 -- 172.21.15.29:0/1879232610 learned_addr learned my addr 172.21.15.29:0/1879232610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6f7fe640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d7010fb40 0x7f9d70076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d7495b640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d70110490 0x7f9d701c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6ffff640 1 -- 172.21.15.29:0/1879232610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d7010fb40 msgr2=0x7f9d70076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6ffff640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d7010fb40 0x7f9d70076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6ffff640 1 -- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d70110490 msgr2=0x7f9d701c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6ffff640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d70110490 0x7f9d701c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.646+0000 7f9d6ffff640 1 -- 172.21.15.29:0/1879232610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701c66e0 con 0x7f9d7010baf0 2024-09-21T02:03:46.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.647+0000 7f9d6ffff640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d6000a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.647+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60040070 con 0x7f9d7010baf0 2024-09-21T02:03:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.647+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701c68b0 con 0x7f9d7010baf0 2024-09-21T02:03:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.648+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701c6d60 con 0x7f9d7010baf0 2024-09-21T02:03:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.648+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6000a990 con 0x7f9d7010baf0 2024-09-21T02:03:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.648+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60004880 con 0x7f9d7010baf0 2024-09-21T02:03:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.654+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9d6000a4e0 con 0x7f9d7010baf0 2024-09-21T02:03:46.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.656+0000 7f9d6d7fa640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 0x7f9d580b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.656+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9d60103a00 con 0x7f9d7010baf0 2024-09-21T02:03:46.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.657+0000 7f9d6f7fe640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 0x7f9d580b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.660+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c0052c0 con 0x7f9d7010baf0 2024-09-21T02:03:46.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.669+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9d60038070 con 0x7f9d7010baf0 2024-09-21T02:03:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:46.670+0000 7f9d6f7fe640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 0x7f9d580b8ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d7010abc0 tx=0x7f9d68009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:47.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.144+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5c005880 con 0x7f9d7010baf0 2024-09-21T02:03:47.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.146+0000 7f9d6d7fa640 1 -- 172.21.15.29:0/1879232610 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11977 (secure 0 0 0) 0x7f9d60004460 con 0x7f9d7010baf0 2024-09-21T02:03:47.149 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:47.149 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"btime":"2024-09-21T02:03:46:906549+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:03:46.192378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24644,24640,24625,24620]},"id":1}]} 2024-09-21T02:03:47.149 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 msgr2=0x7f9d580b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 0x7f9d580b8ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d7010abc0 tx=0x7f9d68009500 comp rx=0 tx=0).stop 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 msgr2=0x7f9d70075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d6000a370 comp rx=0 tx=0).stop 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 shutdown_connections 2024-09-21T02:03:47.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9d580b6690 0x7f9d580b8ba0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d70110490 0x7f9d701c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d7010fb40 0x7f9d70076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 --2- 172.21.15.29:0/1879232610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d7010baf0 0x7f9d70075ce0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 >> 172.21.15.29:0/1879232610 conn(0x7f9d70070ec0 msgr2=0x7f9d7010e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 shutdown_connections 2024-09-21T02:03:47.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:47.151+0000 7f9d763be640 1 -- 172.21.15.29:0/1879232610 wait complete. 2024-09-21T02:03:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:47 smithi190 ceph-mon[34368]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 997 op/s 2024-09-21T02:03:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:47 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:47 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1879232610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:48.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:47 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:47 smithi066 ceph-mon[35001]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 997 op/s 2024-09-21T02:03:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:47 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:47 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1879232610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:48.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:47 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:48.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:47 smithi029 ceph-mon[31228]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 997 op/s 2024-09-21T02:03:48.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:47 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:48.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:47 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1879232610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:48.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:47 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:49.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:03:49.334 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 -- 172.21.15.29:0/1659892573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c108cf0 msgr2=0x7f393c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 --2- 172.21.15.29:0/1659892573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c108cf0 0x7f393c10f520 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 -- 172.21.15.29:0/1659892573 shutdown_connections 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 --2- 172.21.15.29:0/1659892573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c108cf0 0x7f393c10f520 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 --2- 172.21.15.29:0/1659892573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c1083a0 0x7f393c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 --2- 172.21.15.29:0/1659892573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c102a60 0x7f393c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 -- 172.21.15.29:0/1659892573 >> 172.21.15.29:0/1659892573 conn(0x7f393c079730 msgr2=0x7f393c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 -- 172.21.15.29:0/1659892573 shutdown_connections 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.840+0000 7f3944532640 1 -- 172.21.15.29:0/1659892573 wait complete. 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 Processor -- start 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 -- start start 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 0x7f393c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c108cf0 0x7f393c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f393c113a80 con 0x7f393c1083a0 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f393c113900 con 0x7f393c102a60 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.841+0000 7f3944532640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f393c113c00 con 0x7f393c108cf0 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 0x7f393c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40654/0 (socket says 172.21.15.29:40654) 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3941acd640 1 -- 172.21.15.29:0/2499653240 learned_addr learned my addr 172.21.15.29:0/2499653240 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3942acf640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c108cf0 0x7f393c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 -- 172.21.15.29:0/2499653240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c108cf0 msgr2=0x7f393c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c108cf0 0x7f393c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 -- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 msgr2=0x7f393c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 -- 172.21.15.29:0/2499653240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c1a0340 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3941acd640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f39422ce640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 0x7f393c19a3d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f392c009fd0 tx=0x7f392c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f393b7fe640 1 -- 172.21.15.29:0/2499653240 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c004b20 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f393b7fe640 1 -- 172.21.15.29:0/2499653240 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c004cc0 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c1a0630 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f393b7fe640 1 -- 172.21.15.29:0/2499653240 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c010720 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.842+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c1a6d90 con 0x7f393c102a60 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 Processor -- start 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 -- start start 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f393b7fe640 1 -- 172.21.15.29:0/2499653240 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f392c005020 con 0x7f393c102a60 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d211b0 0x55ad270f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad270f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ad27067ea0 con 0x55ad26d4a070 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ad270572d0 con 0x55ad26d211b0 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.845+0000 7f394aaa6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ad2705b0e0 con 0x55ad26d29390 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40666/0 (socket says 172.21.15.29:40666) 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 -- 172.21.15.29:0/1263883449 learned_addr learned my addr 172.21.15.29:0/1263883449 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3941acd640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad270f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f39422ce640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d211b0 0x55ad270f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 -- 172.21.15.29:0/1263883449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 msgr2=0x55ad270f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad270f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 -- 172.21.15.29:0/1263883449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d211b0 msgr2=0x55ad270f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d211b0 0x55ad270f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 -- 172.21.15.29:0/1263883449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad26db5650 con 0x55ad26d4a070 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3941acd640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad270f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3942acf640 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f3934032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52de3b41ac910108 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f393b7fe640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 0x7f39240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.846+0000 7f3938ff9640 1 -- 172.21.15.29:0/1263883449 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393403f070 con 0x55ad26d4a070 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.847+0000 7f3938ff9640 1 -- 172.21.15.29:0/1263883449 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3934004030 con 0x55ad26d4a070 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.847+0000 7f3938ff9640 1 -- 172.21.15.29:0/1263883449 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035070 con 0x55ad26d4a070 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.847+0000 7f39422ce640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 0x7f39240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.847+0000 7f393b7fe640 1 -- 172.21.15.29:0/2499653240 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f392c0e7ba0 con 0x7f393c102a60 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- 172.21.15.29:0/1263883449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 msgr2=0x55ad270f12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f3934032bb0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f39422ce640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 0x7f39240b8a20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ad9e0 tx=0x7f392c095040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- 172.21.15.29:0/1263883449 shutdown_connections 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad270f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d211b0 0x55ad270f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 --2- 172.21.15.29:0/1263883449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad270f12c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- 172.21.15.29:0/1263883449 >> 172.21.15.29:0/1263883449 conn(0x55ad26d3e460 msgr2=0x55ad270ef1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- 172.21.15.29:0/1263883449 shutdown_connections 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- 172.21.15.29:0/1263883449 wait complete. 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 -- start start 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.848+0000 7f394aaa6740 1 Processor -- start 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d211b0 0x55ad270fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad270fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ad26eb8700 con 0x55ad26d4a070 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ad26ff2160 con 0x55ad26d29390 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ad270ffa40 con 0x55ad26d211b0 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3942acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f394aaa6740 10 client.0 _ll_register_callbacks cb 0x55ad2700b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3942acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40670/0 (socket says 172.21.15.29:40670) 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3942acf640 1 -- 172.21.15.29:0/402447471 learned_addr learned my addr 172.21.15.29:0/402447471 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3942acf640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d211b0 0x55ad270fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad270fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 -- 172.21.15.29:0/402447471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d211b0 msgr2=0x55ad270fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d211b0 0x55ad270fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 -- 172.21.15.29:0/402447471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 msgr2=0x55ad27108d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 -- 172.21.15.29:0/402447471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2701fc90 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3942acf640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3941acd640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad270fcf60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f393c107060 tx=0x7f393000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619a5d1f34477452 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.849+0000 7f3921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930004030 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f39300041d0 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3934004030 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f39300043d0 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 10 client.28508 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ad2710b820 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3934004030 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f393001c460 con 0x55ad26d29390 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 10 client.28508 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 10 client.28508 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:03:49.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 20 client.28508 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d089f0 0x55ad2710edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 4 client.28508 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.850+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad27069df0 con 0x55ad26d089f0 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.851+0000 7f3942acf640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d089f0 0x55ad2710edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.851+0000 7f3921ffb640 10 client.28508 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.851+0000 7f3942acf640 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d089f0 0x55ad2710edf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3682772884eb9ff server_cookie=41daa2096a1ebbbf in_seq=0 out_seq=0 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f3921ffb640 1 -- 172.21.15.29:0/402447471 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad27069df0 con 0x55ad26d089f0 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f3921ffb640 10 client.28508 handle_command_reply: tid=0 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 client.28508 shutdown 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 msgr2=0x55ad270fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad270fcf60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f393c107060 tx=0x7f393000f790 comp rx=0 tx=0).stop 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 shutdown_connections 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d089f0 0x55ad2710edf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d4a070 0x55ad27108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f3921ffb640 0 client.28508 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad270fcf60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 --2- 172.21.15.29:0/402447471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d211b0 0x55ad270fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 >> 172.21.15.29:0/402447471 conn(0x55ad26d3e460 msgr2=0x55ad270f44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 shutdown_connections 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 1 -- 172.21.15.29:0/402447471 wait complete. 2024-09-21T02:03:49.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.852+0000 7f394aaa6740 20 client.28508 trim_cache size 0 max 16384 2024-09-21T02:03:49.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.866+0000 7f394aaa6740 1 Processor -- start 2024-09-21T02:03:49.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.866+0000 7f394aaa6740 1 -- start start 2024-09-21T02:03:49.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d089f0 0x55ad270ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad271091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1af260 con 0x55ad26d211b0 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1b0da0 con 0x55ad26d29390 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f394aaa6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1af930 con 0x55ad26d089f0 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40672/0 (socket says 172.21.15.29:40672) 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 -- 172.21.15.29:0/2509682765 learned_addr learned my addr 172.21.15.29:0/2509682765 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3942acf640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d089f0 0x55ad270ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3942acf640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad271091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 -- 172.21.15.29:0/2509682765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d089f0 msgr2=0x55ad270ef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d089f0 0x55ad270ef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 -- 172.21.15.29:0/2509682765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 msgr2=0x55ad271091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad271091a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 -- 172.21.15.29:0/2509682765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad270fdce0 con 0x55ad26d211b0 2024-09-21T02:03:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3942acf640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad271091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.867+0000 7f3941acd640 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f39300059f0 tx=0x7f393000fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620f47982a0f26c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.868+0000 7f3923fff640 1 -- 172.21.15.29:0/2509682765 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393001c460 con 0x55ad26d211b0 2024-09-21T02:03:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.868+0000 7f3923fff640 1 -- 172.21.15.29:0/2509682765 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393002b030 con 0x55ad26d211b0 2024-09-21T02:03:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.868+0000 7f3923fff640 1 -- 172.21.15.29:0/2509682765 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930005580 con 0x55ad26d211b0 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- 172.21.15.29:0/2509682765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 msgr2=0x55ad27108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f39300059f0 tx=0x7f393000fcd0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- 172.21.15.29:0/2509682765 shutdown_connections 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d29390 0x55ad271091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27108c60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2509682765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d089f0 0x55ad270ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- 172.21.15.29:0/2509682765 >> 172.21.15.29:0/2509682765 conn(0x55ad26d3e460 msgr2=0x55ad270eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- 172.21.15.29:0/2509682765 shutdown_connections 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- 172.21.15.29:0/2509682765 wait complete. 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- start start 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 Processor -- start 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d089f0 0x55ad27100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad27178790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1b06d0 con 0x55ad26d211b0 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1b0000 con 0x55ad26d089f0 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1ad240 con 0x55ad26d29390 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.869+0000 7f394aaa6740 10 client.0 _ll_register_callbacks cb 0x7f3930009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:49.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40686/0 (socket says 172.21.15.29:40686) 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 -- 172.21.15.29:0/2026125742 learned_addr learned my addr 172.21.15.29:0/2026125742 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3942acf640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad27178790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3942acf640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d089f0 0x55ad27100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 -- 172.21.15.29:0/2026125742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 msgr2=0x55ad27178790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad27178790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 -- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d089f0 msgr2=0x55ad27100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d089f0 0x55ad27100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 -- 172.21.15.29:0/2026125742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad27101c20 con 0x55ad26d211b0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3941acd640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3930005fd0 tx=0x7f39300127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80119a9e076c6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930035060 con 0x55ad26d211b0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f393001caf0 con 0x55ad26d211b0 2024-09-21T02:03:49.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.870+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f393002b030 con 0x55ad26d211b0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f393001ccf0 con 0x55ad26d211b0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 10 client.18705 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ad27179c80 con 0x55ad26d211b0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39300054f0 con 0x55ad26d211b0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f39300049b0 con 0x55ad26d211b0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 10 client.18705 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 10 client.18705 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 20 client.18705 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d4a070 0x55ad2717c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 4 client.18705 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 --> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f393002b030 con 0x55ad26d4a070 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.871+0000 7f3942acf640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d4a070 0x55ad2717c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f3921ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.66:6834/2455557806 2024-09-21T02:03:49.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f3942acf640 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d4a070 0x55ad2717c2c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f13a745aa48a4c5 server_cookie=c5aaeee21baa0e93 in_seq=0 out_seq=0 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f3921ffb640 1 -- 172.21.15.29:0/2026125742 <== mds.4 v2:172.21.15.66:6834/2455557806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f393002b030 con 0x55ad26d4a070 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f3921ffb640 10 client.18705 handle_command_reply: tid=0 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 client.18705 shutdown 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 msgr2=0x55ad27101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3930005fd0 tx=0x7f39300127f0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 shutdown_connections 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:6834/2455557806,v1:172.21.15.66:6835/2455557806] conn(0x55ad26d4a070 0x55ad2717c2c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad26d29390 0x55ad27178790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad26d211b0 0x55ad27101430 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 --2- 172.21.15.29:0/2026125742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad26d089f0 0x55ad27100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f3921ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.66:6834/2455557806 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 >> 172.21.15.29:0/2026125742 conn(0x55ad26d3e460 msgr2=0x55ad270f43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 shutdown_connections 2024-09-21T02:03:49.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 1 -- 172.21.15.29:0/2026125742 wait complete. 2024-09-21T02:03:49.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.872+0000 7f394aaa6740 20 client.18705 trim_cache size 0 max 16384 2024-09-21T02:03:49.879 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:49.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 msgr2=0x7f39240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 0x7f39240b8a20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ad9e0 tx=0x7f392c095040 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 msgr2=0x7f393c19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 0x7f393c19a3d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f392c009fd0 tx=0x7f392c00ef90 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 shutdown_connections 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f39240b6510 0x7f39240b8a20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393c108cf0 0x7f393c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f393c1083a0 0x7f393c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 --2- 172.21.15.29:0/2499653240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c102a60 0x7f393c19a3d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 >> 172.21.15.29:0/2499653240 conn(0x7f393c079730 msgr2=0x7f393c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 shutdown_connections 2024-09-21T02:03:49.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:49.873+0000 7f3944532640 1 -- 172.21.15.29:0/2499653240 wait complete. 2024-09-21T02:03:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:49 smithi029 ceph-mon[31228]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-09-21T02:03:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:50.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:50.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:49 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:03:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:49 smithi190 ceph-mon[34368]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-09-21T02:03:50.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:50.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:50.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:49 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:03:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:49 smithi066 ceph-mon[35001]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-09-21T02:03:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:50.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:49 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' 2024-09-21T02:03:50.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:03:50.975 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 -- 172.21.15.29:0/130361542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 msgr2=0x7f711c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 --2- 172.21.15.29:0/130361542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c102e70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f710c009900 tx=0x7f710c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 -- 172.21.15.29:0/130361542 shutdown_connections 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 --2- 172.21.15.29:0/130361542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c108cf0 0x7f711c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 --2- 172.21.15.29:0/130361542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c1083a0 0x7f711c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 --2- 172.21.15.29:0/130361542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c102e70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 -- 172.21.15.29:0/130361542 >> 172.21.15.29:0/130361542 conn(0x7f711c079720 msgr2=0x7f711c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 -- 172.21.15.29:0/130361542 shutdown_connections 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.730+0000 7f7122828640 1 -- 172.21.15.29:0/130361542 wait complete. 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 Processor -- start 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 -- start start 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c1083a0 0x7f711c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c108cf0 0x7f711c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f711c112960 con 0x7f711c102a60 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1127e0 con 0x7f711c108cf0 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f7122828640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f711c112ae0 con 0x7f711c1083a0 2024-09-21T02:03:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f711bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f711bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40702/0 (socket says 172.21.15.29:40702) 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.731+0000 7f711bfff640 1 -- 172.21.15.29:0/882872247 learned_addr learned my addr 172.21.15.29:0/882872247 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7113fff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c1083a0 0x7f711c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7120dc5640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c108cf0 0x7f711c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 -- 172.21.15.29:0/882872247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c1083a0 msgr2=0x7f711c19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c1083a0 0x7f711c19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 -- 172.21.15.29:0/882872247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c108cf0 msgr2=0x7f711c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c108cf0 0x7f711c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 -- 172.21.15.29:0/882872247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c1a6500 con 0x7f711c102a60 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f711bfff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f710c005940 tx=0x7f710c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7119ffb640 1 -- 172.21.15.29:0/882872247 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c03f030 con 0x7f711c102a60 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7119ffb640 1 -- 172.21.15.29:0/882872247 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710c003aa0 con 0x7f711c102a60 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c1a6790 con 0x7f711c102a60 2024-09-21T02:03:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7119ffb640 1 -- 172.21.15.29:0/882872247 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c035070 con 0x7f711c102a60 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.732+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1a6c70 con 0x7f711c102a60 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 Processor -- start 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 -- start start 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7119ffb640 1 -- 172.21.15.29:0/882872247 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f710c042030 con 0x7f711c102a60 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933346be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933347120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5599332bcea0 con 0x559932f9f070 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599332ac2d0 con 0x559932f761b0 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.735+0000 7f7128d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599332b00e0 con 0x559932f7e390 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f711bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933346be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40716/0 (socket says 172.21.15.29:40716) 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 -- 172.21.15.29:0/3787810017 learned_addr learned my addr 172.21.15.29:0/3787810017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7113fff640 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933347120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 -- 172.21.15.29:0/3787810017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 msgr2=0x559933347120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933347120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 -- 172.21.15.29:0/3787810017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 msgr2=0x559933346be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933346be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 -- 172.21.15.29:0/3787810017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993300a650 con 0x559932f9f070 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f711bfff640 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933346be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7120dc5640 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7114009fd0 tx=0x7f7114010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a989b881fb3c96ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f7119ffb640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 0x7f71080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f71127fc640 1 -- 172.21.15.29:0/3787810017 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114016070 con 0x559932f9f070 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f711bfff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 0x7f71080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f71127fc640 1 -- 172.21.15.29:0/3787810017 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7114012bc0 con 0x559932f9f070 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.736+0000 7f71127fc640 1 -- 172.21.15.29:0/3787810017 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114011b20 con 0x559932f9f070 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7119ffb640 1 -- 172.21.15.29:0/882872247 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f710c1079b0 con 0x7f711c102a60 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 -- 172.21.15.29:0/3787810017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 msgr2=0x5599333466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f711bfff640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 0x7f71080b8a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559933345320 tx=0x7f710c10d4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7114009fd0 tx=0x7f7114010800 comp rx=0 tx=0).stop 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 -- 172.21.15.29:0/3787810017 shutdown_connections 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933347120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933346be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 --2- 172.21.15.29:0/3787810017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f9f070 0x5599333466a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 -- 172.21.15.29:0/3787810017 >> 172.21.15.29:0/3787810017 conn(0x559932f93460 msgr2=0x559933344590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 -- 172.21.15.29:0/3787810017 shutdown_connections 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.737+0000 7f7128d9c740 1 -- 172.21.15.29:0/3787810017 wait complete. 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 -- start start 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 Processor -- start 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933351e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f9f070 0x55993335e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55993310d700 con 0x559932f7e390 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559933247160 con 0x559932f761b0 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559933354e80 con 0x559932f9f070 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7128d9c740 10 client.0 _ll_register_callbacks cb 0x5599332609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7113fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7113fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40732/0 (socket says 172.21.15.29:40732) 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7113fff640 1 -- 172.21.15.29:0/2093322937 learned_addr learned my addr 172.21.15.29:0/2093322937 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.738+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933351e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f9f070 0x55993335e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 -- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f9f070 msgr2=0x55993335e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f9f070 0x55993335e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 -- 172.21.15.29:0/2093322937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 msgr2=0x559933351e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933351e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 -- 172.21.15.29:0/2093322937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559933274c90 con 0x559932f7e390 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7113fff640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7104012700 tx=0x7f7104012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92da3a8f9f0ebe28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f70fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104004050 con 0x559932f7e390 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f71040041f0 con 0x559932f7e390 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7114012bc0 con 0x559932f7e390 2024-09-21T02:03:51.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f71040043f0 con 0x559932f7e390 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7128d9c740 10 client.18717 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559933360c80 con 0x559932f7e390 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.739+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7114012bc0 con 0x559932f7e390 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f7104003b50 con 0x559932f7e390 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 10 client.18717 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 10 client.18717 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 20 client.18717 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f5d9f0 0x559933364270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 4 client.18717 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599332bedf0 con 0x559932f5d9f0 2024-09-21T02:03:51.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.740+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f5d9f0 0x559933364270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.741+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f5d9f0 0x559933364270 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46488717ee19a3a1 server_cookie=63d63af9fc1ca8eb in_seq=0 out_seq=0 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.741+0000 7f70fb7fe640 10 client.18717 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.741+0000 7f70fb7fe640 1 -- 172.21.15.29:0/2093322937 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599332bedf0 con 0x559932f5d9f0 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.741+0000 7f70fb7fe640 10 client.18717 handle_command_reply: tid=0 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.741+0000 7f7128d9c740 1 client.18717 shutdown 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 msgr2=0x559933352340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7104012700 tx=0x7f7104012bc0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 shutdown_connections 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f5d9f0 0x559933364270 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f9f070 0x55993335e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f7e390 0x559933352340 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f70fb7fe640 0 client.18717 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:51.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2093322937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f761b0 0x559933351e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 >> 172.21.15.29:0/2093322937 conn(0x559932f93460 msgr2=0x559933349880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 shutdown_connections 2024-09-21T02:03:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 1 -- 172.21.15.29:0/2093322937 wait complete. 2024-09-21T02:03:51.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.742+0000 7f7128d9c740 20 client.18717 trim_cache size 0 max 16384 2024-09-21T02:03:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 Processor -- start 2024-09-21T02:03:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 -- start start 2024-09-21T02:03:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f5d9f0 0x559933350cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f7e390 0x55993335e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.755+0000 7f7128d9c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1af300 con 0x559932f761b0 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7128d9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1b0e40 con 0x559932f7e390 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7128d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1af9d0 con 0x559932f5d9f0 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40742/0 (socket says 172.21.15.29:40742) 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 -- 172.21.15.29:0/2154618190 learned_addr learned my addr 172.21.15.29:0/2154618190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f5d9f0 0x559933350cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7120dc5640 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f7e390 0x55993335e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 -- 172.21.15.29:0/2154618190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f5d9f0 msgr2=0x559933350cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f5d9f0 0x559933350cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 -- 172.21.15.29:0/2154618190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f7e390 msgr2=0x55993335e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f7e390 0x55993335e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 -- 172.21.15.29:0/2154618190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559933348c80 con 0x559932f761b0 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7113fff640 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f710400a010 tx=0x7f7104010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f88adf14eaf6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7111ffb640 1 -- 172.21.15.29:0/2154618190 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104004000 con 0x559932f761b0 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7111ffb640 1 -- 172.21.15.29:0/2154618190 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7104010e20 con 0x559932f761b0 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.756+0000 7f7111ffb640 1 -- 172.21.15.29:0/2154618190 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104005040 con 0x559932f761b0 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 -- 172.21.15.29:0/2154618190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 msgr2=0x5599333445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f710400a010 tx=0x7f7104010800 comp rx=0 tx=0).stop 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 -- 172.21.15.29:0/2154618190 shutdown_connections 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f7e390 0x55993335e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333445c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 --2- 172.21.15.29:0/2154618190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f5d9f0 0x559933350cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 -- 172.21.15.29:0/2154618190 >> 172.21.15.29:0/2154618190 conn(0x559932f93460 msgr2=0x559933343eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 -- 172.21.15.29:0/2154618190 shutdown_connections 2024-09-21T02:03:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.757+0000 7f7128d9c740 1 -- 172.21.15.29:0/2154618190 wait complete. 2024-09-21T02:03:51.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 -- start start 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 Processor -- start 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933356c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1b0770 con 0x559932f761b0 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1b00a0 con 0x559932f5d9f0 2024-09-21T02:03:51.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.758+0000 7f7128d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1ad2e0 con 0x559932f7e390 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7128d9c740 10 client.0 _ll_register_callbacks cb 0x7f71040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7113fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.29:40920/0 (socket says 172.21.15.29:40920) 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 -- 172.21.15.29:0/752845011 learned_addr learned my addr 172.21.15.29:0/752845011 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933356c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 -- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 msgr2=0x559933356c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933356c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 -- 172.21.15.29:0/752845011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 msgr2=0x5599333566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 -- 172.21.15.29:0/752845011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599333cdce0 con 0x559932f5d9f0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7113fff640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7120dc5640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f71140115d0 tx=0x7f7104009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d9a34f2f7c4224 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f70fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114016040 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7104010e20 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.759+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f7114011e40 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7114003a10 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 10 client.28532 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5599333cef60 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7104010e20 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f711402c040 con 0x559932f5d9f0 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 10 client.28532 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 10 client.28532 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 20 client.28532 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f9f070 0x5599333d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 4 client.28532 mds_command: new command op to 24625 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.760+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 --> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599332ba8a0 con 0x559932f9f070 2024-09-21T02:03:51.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7113fff640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f9f070 0x5599333d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f70fb7fe640 10 client.28532 ms_handle_connect on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7113fff640 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f9f070 0x5599333d14f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9aaf59c271de6f9 server_cookie=f62915582dd043b8 in_seq=0 out_seq=0 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f70fb7fe640 1 -- 172.21.15.29:0/752845011 <== mds.3 v2:172.21.15.190:6836/2047991561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599332ba8a0 con 0x559932f9f070 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f70fb7fe640 10 client.28532 handle_command_reply: tid=0 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7128d9c740 1 client.28532 shutdown 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 msgr2=0x559933356180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f71140115d0 tx=0x7f7104009ca0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.761+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 shutdown_connections 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:6836/2047991561,v1:172.21.15.190:6837/2047991561] conn(0x559932f9f070 0x5599333d14f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559932f7e390 0x559933356c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559932f761b0 0x5599333566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f70fb7fe640 0 client.28532 ms_handle_reset on v2:172.21.15.190:6836/2047991561 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 --2- 172.21.15.29:0/752845011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559932f5d9f0 0x559933356180 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 >> 172.21.15.29:0/752845011 conn(0x559932f93460 msgr2=0x5599333497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 shutdown_connections 2024-09-21T02:03:51.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 1 -- 172.21.15.29:0/752845011 wait complete. 2024-09-21T02:03:51.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7128d9c740 20 client.28532 trim_cache size 0 max 16384 2024-09-21T02:03:51.769 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 msgr2=0x7f71080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 0x7f71080b8a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559933345320 tx=0x7f710c10d4f0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 msgr2=0x7f711c19b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f710c005940 tx=0x7f710c004270 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 shutdown_connections 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f71080b6510 0x7f71080b8a20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711c108cf0 0x7f711c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c1083a0 0x7f711c19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 --2- 172.21.15.29:0/882872247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f711c102a60 0x7f711c19b1f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 >> 172.21.15.29:0/882872247 conn(0x7f711c079720 msgr2=0x7f711c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.762+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 shutdown_connections 2024-09-21T02:03:51.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:51.763+0000 7f7122828640 1 -- 172.21.15.29:0/882872247 wait complete. 2024-09-21T02:03:52.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:51 smithi190 ceph-mon[34368]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-09-21T02:03:52.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:51 smithi066 ceph-mon[35001]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-09-21T02:03:52.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:51 smithi029 ceph-mon[31228]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-09-21T02:03:52.381 DEBUG:teuthology.orchestra.run.smithi029:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/: 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 data 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data: 2024-09-21T02:03:52.463 INFO:teuthology.orchestra.run.smithi029.stdout:total 36 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 . 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 .. 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 base 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 21 01:48 current_logfiles 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 01:48 global 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 log 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_commit_ts 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_dynshmem 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 01:48 pg_hba.conf 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 01:48 pg_ident.conf 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 pg_logical 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 pg_multixact 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_notify 2024-09-21T02:03:52.464 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_replslot 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_serial 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_snapshots 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 pg_stat 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 02:03 pg_stat_tmp 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 02:03 pg_subtrans 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_tblspc 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pg_twophase 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 02:03 pg_wal 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 pg_xact 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 01:48 postgresql.auto.conf 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 01:48 postgresql.conf 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 21 01:48 postmaster.opts 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.465 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 . 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 1 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 13493 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 01:51 13494 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 pgsql_tmp 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:total 7777 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 . 2024-09-21T02:03:52.466 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 .. 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 01:48 1259 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T02:03:52.467 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T02:03:52.468 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T02:03:52.469 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T02:03:52.470 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T02:03:52.471 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T02:03:52.472 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 01:48 2619 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2619_fsm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T02:03:52.473 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T02:03:52.474 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T02:03:52.475 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2696 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T02:03:52.476 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T02:03:52.477 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T02:03:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T02:03:52.479 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T02:03:52.480 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T02:03:52.481 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T02:03:52.482 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T02:03:52.483 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T02:03:52.484 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T02:03:52.485 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T02:03:52.485 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T02:03:52.486 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T02:03:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:total 7777 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 01:48 . 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 .. 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T02:03:52.488 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 01:48 1259 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T02:03:52.489 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T02:03:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T02:03:52.491 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T02:03:52.492 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T02:03:52.493 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 01:48 2619 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2619_fsm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T02:03:52.494 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T02:03:52.495 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T02:03:52.496 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2696 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T02:03:52.497 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T02:03:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T02:03:52.499 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T02:03:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T02:03:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T02:03:52.502 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T02:03:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T02:03:52.504 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T02:03:52.505 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T02:03:52.506 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T02:03:52.507 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T02:03:52.508 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:total 639934 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 01:51 . 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 .. 2024-09-21T02:03:52.509 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 112 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 113 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 1247 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1247_fsm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1247_vm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 01:48 1249 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1249_fsm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1249_vm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 01:48 1255 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1255_fsm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1255_vm 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 02:03 1259 2024-09-21T02:03:52.510 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1259_fsm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1259_vm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 01:48 13344 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13344_fsm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13344_vm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13346 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13348 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13349_fsm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13349_vm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13351 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13353 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13354_fsm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13354_vm 2024-09-21T02:03:52.511 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13356 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13358 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 13359_fsm 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13359_vm 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 13361 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 13363 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1417 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 1418 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 447209472 Sep 21 02:03 16396 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 21 02:03 16396_fsm 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:51 16396_vm 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2768896 Sep 21 02:03 16397 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 02:03 16397_fsm 2024-09-21T02:03:52.512 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 02:03 16397_vm 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2220032 Sep 21 02:03 16399 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 02:03 16399_fsm 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 02:03 16399_vm 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 02:03 16400 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 02:03 16402 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 21 02:03 16404 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122699776 Sep 21 02:03 16406 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 02:03 16406_fsm 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 02:02 16406_vm 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 174 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 175 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2187 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2224 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2328 2024-09-21T02:03:52.513 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2336 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2337 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2579 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2600 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2600_fsm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2600_vm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2601_fsm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2601_vm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2602 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2602_fsm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2602_vm 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2603 2024-09-21T02:03:52.514 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2603_fsm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2603_vm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2604 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2605 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2605_fsm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2605_vm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2606_fsm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2606_vm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2607 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2607_fsm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2607_vm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 01:48 2608 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2608_fsm 2024-09-21T02:03:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2608_vm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 01:48 2609 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2609_fsm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2609_vm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2610 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2610_fsm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2610_vm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2611 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2612_fsm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2612_vm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2613 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2615_fsm 2024-09-21T02:03:52.516 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2615_vm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2616 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2616_fsm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2616_vm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 01:48 2617 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2617_fsm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2617_vm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 01:48 2618 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2618_fsm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2618_vm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 21 02:03 2619 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:50 2619_fsm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2619_vm 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2620 2024-09-21T02:03:52.517 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2650 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2651 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2652 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2653 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2654 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2655 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2656 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2657 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 01:48 2658 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 01:48 2659 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2660 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2661 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2662 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2663 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2664 2024-09-21T02:03:52.518 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2665 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2666 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2667 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2668 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2669 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2670 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 01:48 2673 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 01:48 2674 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 01:48 2675 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2678 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2679 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2680 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2681 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2682 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2683 2024-09-21T02:03:52.519 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2684 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2685 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2686 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2687 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2688 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 01:48 2689 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 01:48 2690 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 01:48 2691 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2692 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2693 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:54 2696 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2699 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2701 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2702 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2703 2024-09-21T02:03:52.520 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 2704 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2753 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2753_fsm 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2753_vm 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2754 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2755 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2756 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 2757 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2830 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2831 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2832 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2833 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2834 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2835 2024-09-21T02:03:52.521 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2836 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2837 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 01:48 2838 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2838_fsm 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2838_vm 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2839 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2840_fsm 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2840_vm 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2841 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2995 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2996 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3079_fsm 2024-09-21T02:03:52.522 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3079_vm 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3080 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3081 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3085 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3118 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3119 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 01:48 3164 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3256 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3257 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3258 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3350 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3351 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3379 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3380 2024-09-21T02:03:52.523 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3381 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3394 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3394_fsm 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3394_vm 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3395 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3429 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3430 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3431 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3433 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3439 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3440 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 01:48 3455 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 01:48 3456 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3456_fsm 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3456_vm 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3466 2024-09-21T02:03:52.524 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3467 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3468 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3501 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3502 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3503 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3534 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3541_fsm 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3541_vm 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3542 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3574 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3575 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3576 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3596 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3597 2024-09-21T02:03:52.525 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3598 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3599 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3600_fsm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3600_vm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3601_fsm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3601_vm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3602_fsm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3602_vm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3603_fsm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3603_vm 2024-09-21T02:03:52.526 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3604 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3605 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3606 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3607 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3608 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 01:48 3609 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3712 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 3764_fsm 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3764_vm 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3766 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 3767 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3997 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4143 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4144 2024-09-21T02:03:52.527 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4145 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4146 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4147 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4148 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4149 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4150 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4151 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4152 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4153 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4154 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4155 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4156 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4157 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4158 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4159 2024-09-21T02:03:52.528 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4160 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4161 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4162 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4163 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4164 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4165 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4166 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4167 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4168 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4169 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4170 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4171 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4172 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4173 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4174 2024-09-21T02:03:52.529 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 5002 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 548 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 549 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6102 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6104 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6106 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6110 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6111 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6112 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6113 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6117 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 826 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 827 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 828 2024-09-21T02:03:52.530 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 21 01:48 pg_internal.init 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 01:48 PG_VERSION 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 01:48 .. 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:total 560 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 01:48 . 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1213 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1213_fsm 2024-09-21T02:03:52.531 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1213_vm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1214 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1214_fsm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1214_vm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 1232 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 1233 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1260 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1260_fsm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1260_vm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1261 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1261_fsm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1261_vm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1262 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 1262_fsm 2024-09-21T02:03:52.532 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 1262_vm 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2396 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 01:48 2396_fsm 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2396_vm 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2397 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2671 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2672 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2676 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2677 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2694 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2695 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2697 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 01:48 2698 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2846 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2847 2024-09-21T02:03:52.533 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2964 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2965 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 2966 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 2967 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 3592 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 3593 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4060 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4061 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4175 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4176 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4177 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4178 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4181 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4182 2024-09-21T02:03:52.534 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4183 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4184 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 4185 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 4186 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6000 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6001 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6002 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 6100 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6114 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 6115 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 02:03 pg_control 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 01:48 pg_filenode.map 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 21 01:48 pg_internal.init 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.535 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:total 2 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Sep 21 02:03 postgresql-Sat.log 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:total 1 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 . 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 mappings 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 02:03 replorigin_checkpoint 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 snapshots 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 .. 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T02:03:52.537 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 .. 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 . 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 members 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 offsets 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T02:03:52.538 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 .. 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:48 0000 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:total 8 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 01:48 . 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 01:48 .. 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 01:50 0000 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:total 15 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 . 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 21 02:03 db_0.stat 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 21 02:03 db_13494.stat 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 21 02:03 global.stat 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 02:03 . 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.542 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:total 216 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 02:03 . 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 02:03 0023 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.543 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:total 1048576 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 02:03 . 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:03 0000000100000000000000CD 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000CE 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:58 0000000100000000000000CF 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:58 0000000100000000000000D0 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:59 0000000100000000000000D1 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000D2 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000D3 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000D4 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000D5 2024-09-21T02:03:52.544 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000D6 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000D7 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000D8 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:59 0000000100000000000000D9 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000DA 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000DB 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000DC 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000DD 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000DE 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000DF 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000E0 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000E1 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000E2 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:58 0000000100000000000000E3 2024-09-21T02:03:52.545 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000E4 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000E5 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:57 0000000100000000000000E6 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 01:56 0000000100000000000000E7 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000E8 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:02 0000000100000000000000E9 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000EA 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 0000000100000000000000EB 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:02 0000000100000000000000EC 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000ED 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000EE 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 0000000100000000000000EF 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F0 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:02 0000000100000000000000F1 2024-09-21T02:03:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F2 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F3 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F4 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F5 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F6 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:02 0000000100000000000000F7 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000F8 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 0000000100000000000000F9 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 0000000100000000000000FA 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 0000000100000000000000FB 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 0000000100000000000000FC 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:03 0000000100000000000000FD 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:03 0000000100000000000000FE 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:02 0000000100000000000000FF 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 000000010000000100000000 2024-09-21T02:03:52.547 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 000000010000000100000001 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000002 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000003 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000004 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 000000010000000100000005 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000006 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000007 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000008 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:01 000000010000000100000009 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:03 00000001000000010000000A 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 00000001000000010000000B 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 02:00 00000001000000010000000C 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 archive_status 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.548 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:total 0 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 01:48 . 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 02:03 .. 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:total 576 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 02:03 . 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 02:03 .. 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 21 01:53 0000 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 21 02:03 0001 2024-09-21T02:03:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 02:03 0002 2024-09-21T02:03:52.549 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T02:03:52.560 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T02:03:52.568 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T02:03:52.577 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T02:03:53.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:03:53.371 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:54.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:53 smithi190 ceph-mon[34368]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2024-09-21T02:03:54.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:53 smithi066 ceph-mon[35001]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2024-09-21T02:03:54.322 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:53 smithi029 ceph-mon[31228]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2024-09-21T02:03:54.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 -- 172.21.15.29:0/2787086017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 msgr2=0x7f975c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 --2- 172.21.15.29:0/2787086017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c104c50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ca0 tx=0x7f974c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 -- 172.21.15.29:0/2787086017 shutdown_connections 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 --2- 172.21.15.29:0/2787086017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c10ddf0 0x7f975c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 --2- 172.21.15.29:0/2787086017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c105270 0x7f975c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 --2- 172.21.15.29:0/2787086017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c104c50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 -- 172.21.15.29:0/2787086017 >> 172.21.15.29:0/2787086017 conn(0x7f975c079730 msgr2=0x7f975c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 -- 172.21.15.29:0/2787086017 shutdown_connections 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 -- 172.21.15.29:0/2787086017 wait complete. 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.687+0000 7f9761235640 1 Processor -- start 2024-09-21T02:03:54.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 -- start start 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c105270 0x7f975c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 0x7f975c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f975c113b10 con 0x7f975c104840 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f975c113990 con 0x7f975c105270 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f9761235640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f975c113c90 con 0x7f975c10ddf0 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57070/0 (socket says 172.21.15.29:57070) 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975ad9d640 1 -- 172.21.15.29:0/1237180691 learned_addr learned my addr 172.21.15.29:0/1237180691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975b59e640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 0x7f975c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975a59c640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c105270 0x7f975c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975ad9d640 1 -- 172.21.15.29:0/1237180691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 msgr2=0x7f975c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.688+0000 7f975ad9d640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 0x7f975c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f975ad9d640 1 -- 172.21.15.29:0/1237180691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c105270 msgr2=0x7f975c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f975ad9d640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c105270 0x7f975c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f975ad9d640 1 -- 172.21.15.29:0/1237180691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c1a04b0 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f975b59e640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 0x7f975c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f975ad9d640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f974c006010 tx=0x7f974c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f9753fff640 1 -- 172.21.15.29:0/1237180691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c003bf0 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c1a0740 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f9753fff640 1 -- 172.21.15.29:0/1237180691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c003d90 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f9753fff640 1 -- 172.21.15.29:0/1237180691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c02fb60 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.689+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1a6f00 con 0x7f975c104840 2024-09-21T02:03:54.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 Processor -- start 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 -- start start 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f9753fff640 1 -- 172.21.15.29:0/1237180691 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f974c02fd00 con 0x7f975c104840 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fce71b0 0x5604800b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 0x5604800b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56048002dea0 con 0x56047fd10070 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56048001d2d0 con 0x56047fce71b0 2024-09-21T02:03:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.692+0000 7f97677a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5604800210e0 con 0x56047fcef390 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57080/0 (socket says 172.21.15.29:57080) 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 -- 172.21.15.29:0/693448534 learned_addr learned my addr 172.21.15.29:0/693448534 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975a59c640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 0x5604800b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975ad9d640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fce71b0 0x5604800b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 -- 172.21.15.29:0/693448534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 msgr2=0x5604800b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 0x5604800b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 -- 172.21.15.29:0/693448534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fce71b0 msgr2=0x5604800b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fce71b0 0x5604800b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 -- 172.21.15.29:0/693448534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047fd7b650 con 0x56047fd10070 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975a59c640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 0x5604800b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f975b59e640 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9754007800 tx=0x7f9754007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7cabc985f37f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f97517fa640 1 -- 172.21.15.29:0/693448534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400d950 con 0x56047fd10070 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f9753fff640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 0x7f97480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f97517fa640 1 -- 172.21.15.29:0/693448534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754018070 con 0x56047fd10070 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.693+0000 7f97517fa640 1 -- 172.21.15.29:0/693448534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754013d80 con 0x56047fd10070 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.694+0000 7f975ad9d640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 0x7f97480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.694+0000 7f9753fff640 1 -- 172.21.15.29:0/1237180691 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f974c108670 con 0x7f975c104840 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.694+0000 7f97677a9740 1 -- 172.21.15.29:0/693448534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 msgr2=0x5604800b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.694+0000 7f975ad9d640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 0x7f97480b8a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5604800b5f40 tx=0x7f974c003f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.694+0000 7f97677a9740 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9754007800 tx=0x7f9754007cc0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- 172.21.15.29:0/693448534 shutdown_connections 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fcef390 0x5604800b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fce71b0 0x5604800b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- 172.21.15.29:0/693448534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b72c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- 172.21.15.29:0/693448534 >> 172.21.15.29:0/693448534 conn(0x56047fd04460 msgr2=0x5604800b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- 172.21.15.29:0/693448534 shutdown_connections 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- 172.21.15.29:0/693448534 wait complete. 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- start start 2024-09-21T02:03:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 Processor -- start 2024-09-21T02:03:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fd10070 0x5604800c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56047fe7e700 con 0x56047fce71b0 2024-09-21T02:03:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56047ffb8160 con 0x56047fcef390 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.695+0000 7f97677a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5604800c5aa0 con 0x56047fd10070 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f97677a9740 10 client.0 _ll_register_callbacks cb 0x56047ffd19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57086/0 (socket says 172.21.15.29:57086) 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 -- 172.21.15.29:0/2439271412 learned_addr learned my addr 172.21.15.29:0/2439271412 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975a59c640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 -- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fd10070 msgr2=0x5604800c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fd10070 0x5604800c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 -- 172.21.15.29:0/2439271412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 msgr2=0x5604800c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 -- 172.21.15.29:0/2439271412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047ffe5c90 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975a59c640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f975b59e640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9754006f60 tx=0x7f9754013480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e32d59f5e0e183 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f97427fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754024070 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f975400d950 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.696+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754018070 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f975400db50 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 10 client.18735 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5604800d5b60 con 0x56047fce71b0 2024-09-21T02:03:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754018070 con 0x56047fce71b0 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f975401f450 con 0x56047fce71b0 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 10 client.18735 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 10 client.18735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 20 client.18735 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fcce9f0 0x5604800d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 4 client.18735 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048002fdf0 con 0x56047fcce9f0 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.697+0000 7f975a59c640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fcce9f0 0x5604800d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.698+0000 7f975a59c640 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fcce9f0 0x5604800d8f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f6a937abe4704d8 server_cookie=1fde499175e71ae6 in_seq=0 out_seq=0 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.698+0000 7f97427fc640 10 client.18735 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97427fc640 1 -- 172.21.15.29:0/2439271412 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56048002fdf0 con 0x56047fcce9f0 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97427fc640 10 client.18735 handle_command_reply: tid=0 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 client.18735 shutdown 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 msgr2=0x5604800c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9754006f60 tx=0x7f9754013480 comp rx=0 tx=0).stop 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 shutdown_connections 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fcce9f0 0x5604800d8f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fd10070 0x5604800c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 --2- 172.21.15.29:0/2439271412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fce71b0 0x5604800c2940 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 >> 172.21.15.29:0/2439271412 conn(0x56047fd04460 msgr2=0x5604800ba4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97427fc640 0 client.18735 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 shutdown_connections 2024-09-21T02:03:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 1 -- 172.21.15.29:0/2439271412 wait complete. 2024-09-21T02:03:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.699+0000 7f97677a9740 20 client.18735 trim_cache size 0 max 16384 2024-09-21T02:03:54.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.712+0000 7f97677a9740 1 Processor -- start 2024-09-21T02:03:54.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.712+0000 7f97677a9740 1 -- start start 2024-09-21T02:03:54.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ad290 con 0x56047fd10070 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1aedd0 con 0x56047fcef390 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f97677a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ad960 con 0x56047fce71b0 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57090/0 (socket says 172.21.15.29:57090) 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 -- 172.21.15.29:0/4120114182 learned_addr learned my addr 172.21.15.29:0/4120114182 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975a59c640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 -- 172.21.15.29:0/4120114182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 msgr2=0x5604800c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 -- 172.21.15.29:0/4120114182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 msgr2=0x5604800b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 -- 172.21.15.29:0/4120114182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604800ba120 con 0x56047fd10070 2024-09-21T02:03:54.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975a59c640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f975b59e640 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9754007720 tx=0x7f9754015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c561589e5e7fd29f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.713+0000 7f9750ff9640 1 -- 172.21.15.29:0/4120114182 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754024070 con 0x56047fd10070 2024-09-21T02:03:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f9750ff9640 1 -- 172.21.15.29:0/4120114182 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754018050 con 0x56047fd10070 2024-09-21T02:03:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f9750ff9640 1 -- 172.21.15.29:0/4120114182 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975401d6a0 con 0x56047fd10070 2024-09-21T02:03:54.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 -- 172.21.15.29:0/4120114182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 msgr2=0x5604800b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9754007720 tx=0x7f9754015040 comp rx=0 tx=0).stop 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 -- 172.21.15.29:0/4120114182 shutdown_connections 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fd10070 0x5604800b4ca0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fcef390 0x5604800b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 --2- 172.21.15.29:0/4120114182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.714+0000 7f97677a9740 1 -- 172.21.15.29:0/4120114182 >> 172.21.15.29:0/4120114182 conn(0x56047fd04460 msgr2=0x5604800c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- 172.21.15.29:0/4120114182 shutdown_connections 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- 172.21.15.29:0/4120114182 wait complete. 2024-09-21T02:03:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- start start 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 Processor -- start 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fd10070 0x5604800da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ae700 con 0x56047fcef390 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ae030 con 0x56047fd10070 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ab290 con 0x56047fce71b0 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f97677a9740 10 client.0 _ll_register_callbacks cb 0x7f9754007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.715+0000 7f975a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57094/0 (socket says 172.21.15.29:57094) 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 -- 172.21.15.29:0/4183329256 learned_addr learned my addr 172.21.15.29:0/4183329256 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975b59e640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fd10070 0x5604800da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975b59e640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 -- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 msgr2=0x5604800dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800dc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 -- 172.21.15.29:0/4183329256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fd10070 msgr2=0x5604800da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fd10070 0x5604800da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 -- 172.21.15.29:0/4183329256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604800b5d70 con 0x56047fcef390 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f975a59c640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f975c06ad30 tx=0x7f974400ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e213dc5183a0747 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f97427fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744011910 con 0x56047fcef390 2024-09-21T02:03:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9744011ab0 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.716+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754018050 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9744011cb0 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 10 client.18741 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5604801433e0 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754018050 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f97440243d0 con 0x56047fcef390 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 10 client.18741 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 10 client.18741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 20 client.18741 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fca3da0 0x560480145c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 4 client.18741 mds_command: new command op to 24640 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.717+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 --> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56048002b8a0 con 0x56047fca3da0 2024-09-21T02:03:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f975b59e640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fca3da0 0x560480145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97427fc640 10 client.18741 ms_handle_connect on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f975b59e640 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fca3da0 0x560480145c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bae45c28e16133 server_cookie=c4ce6c4e851d237 in_seq=0 out_seq=0 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97427fc640 1 -- 172.21.15.29:0/4183329256 <== mds.2 v2:172.21.15.190:6838/636701059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56048002b8a0 con 0x56047fca3da0 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97427fc640 10 client.18741 handle_command_reply: tid=0 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 client.18741 shutdown 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 msgr2=0x5604800d9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f975c06ad30 tx=0x7f974400ba80 comp rx=0 tx=0).stop 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 shutdown_connections 2024-09-21T02:03:54.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:6838/636701059,v1:172.21.15.190:6839/636701059] conn(0x56047fca3da0 0x560480145c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56047fd10070 0x5604800da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56047fcef390 0x5604800d9cb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 --2- 172.21.15.29:0/4183329256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56047fce71b0 0x5604800dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97427fc640 0 client.18741 ms_handle_reset on v2:172.21.15.190:6838/636701059 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.718+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 >> 172.21.15.29:0/4183329256 conn(0x56047fd04460 msgr2=0x5604800b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 shutdown_connections 2024-09-21T02:03:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f97677a9740 1 -- 172.21.15.29:0/4183329256 wait complete. 2024-09-21T02:03:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f97677a9740 20 client.18741 trim_cache size 0 max 16384 2024-09-21T02:03:54.730 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 msgr2=0x7f97480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 0x7f97480b8a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5604800b5f40 tx=0x7f974c003f40 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 msgr2=0x7f975c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f974c006010 tx=0x7f974c0319f0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 shutdown_connections 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f97480b6510 0x7f97480b8a20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c10ddf0 0x7f975c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c105270 0x7f975c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.719+0000 7f9761235640 1 --2- 172.21.15.29:0/1237180691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f975c104840 0x7f975c19a540 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.720+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 >> 172.21.15.29:0/1237180691 conn(0x7f975c079730 msgr2=0x7f975c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.720+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 shutdown_connections 2024-09-21T02:03:54.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:54.720+0000 7f9761235640 1 -- 172.21.15.29:0/1237180691 wait complete. 2024-09-21T02:03:55.614 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:03:55.876 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:56.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:55 smithi190 ceph-mon[34368]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2024-09-21T02:03:56.274 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:55 smithi029 ceph-mon[31228]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2024-09-21T02:03:56.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:55 smithi066 ceph-mon[35001]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 -- 172.21.15.29:0/3527951297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c108d20 msgr2=0x7f3e4c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/3527951297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c108d20 0x7f3e4c10f550 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009ca0 tx=0x7f3e4402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 -- 172.21.15.29:0/3527951297 shutdown_connections 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/3527951297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c108d20 0x7f3e4c10f550 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/3527951297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/3527951297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 0x7f3e4c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 -- 172.21.15.29:0/3527951297 >> 172.21.15.29:0/3527951297 conn(0x7f3e4c079760 msgr2=0x7f3e4c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 -- 172.21.15.29:0/3527951297 shutdown_connections 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 -- 172.21.15.29:0/3527951297 wait complete. 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.367+0000 7f3e52cc7640 1 Processor -- start 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 -- start start 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 0x7f3e4c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c108d20 0x7f3e4c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c112990 con 0x7f3e4c1083d0 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c112810 con 0x7f3e4c102ae0 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e52cc7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c112b10 con 0x7f3e4c108d20 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57100/0 (socket says 172.21.15.29:57100) 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 -- 172.21.15.29:0/2033365767 learned_addr learned my addr 172.21.15.29:0/2033365767 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e51264640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c108d20 0x7f3e4c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e50a63640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 0x7f3e4c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 -- 172.21.15.29:0/2033365767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c108d20 msgr2=0x7f3e4c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c108d20 0x7f3e4c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 -- 172.21.15.29:0/2033365767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 msgr2=0x7f3e4c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 0x7f3e4c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.368+0000 7f3e43fff640 1 -- 172.21.15.29:0/2033365767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1a03c0 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00d740 tx=0x7f3e3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e41ffb640 1 -- 172.21.15.29:0/2033365767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c017070 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e41ffb640 1 -- 172.21.15.29:0/2033365767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c00de50 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c1a06b0 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e41ffb640 1 -- 172.21.15.29:0/2033365767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c004ce0 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.369+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1a6e10 con 0x7f3e4c1083d0 2024-09-21T02:03:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.371+0000 7f3e5923b740 1 Processor -- start 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 -- start start 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e41ffb640 1 -- 172.21.15.29:0/2033365767 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3e3c004e80 con 0x7f3e4c1083d0 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x5601802016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180202120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560180177ea0 con 0x56017fe311b0 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5601801672d0 con 0x56017fe5a070 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e5923b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56018016b0e0 con 0x56017fe39390 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57102/0 (socket says 172.21.15.29:57102) 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 -- 172.21.15.29:0/1820369278 learned_addr learned my addr 172.21.15.29:0/1820369278 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e43fff640 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180202120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e51264640 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x5601802016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 -- 172.21.15.29:0/1820369278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 msgr2=0x560180202120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180202120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 -- 172.21.15.29:0/1820369278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 msgr2=0x5601802016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x5601802016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.372+0000 7f3e50a63640 1 -- 172.21.15.29:0/1820369278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017fec5650 con 0x56017fe311b0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e41ffb640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 0x7f3e340b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800bd40 tx=0x7f3e4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54f8d601e59c4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e51264640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 0x7f3e340b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e41ffb640 1 -- 172.21.15.29:0/2033365767 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3e3c0e6c10 con 0x7f3e4c1083d0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e32ffd640 1 -- 172.21.15.29:0/1820369278 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48015070 con 0x56017fe311b0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e32ffd640 1 -- 172.21.15.29:0/1820369278 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4800be30 con 0x56017fe311b0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.373+0000 7f3e32ffd640 1 -- 172.21.15.29:0/1820369278 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48011a30 con 0x56017fe311b0 2024-09-21T02:03:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- 172.21.15.29:0/1820369278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 msgr2=0x560180201be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e51264640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 0x7f3e340b8a20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006010 tx=0x7f3e44009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800bd40 tx=0x7f3e4800f730 comp rx=0 tx=0).stop 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- 172.21.15.29:0/1820369278 shutdown_connections 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180202120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe311b0 0x560180201be0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1820369278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x5601802016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- 172.21.15.29:0/1820369278 >> 172.21.15.29:0/1820369278 conn(0x56017fe4e460 msgr2=0x5601801ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- 172.21.15.29:0/1820369278 shutdown_connections 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- 172.21.15.29:0/1820369278 wait complete. 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.374+0000 7f3e5923b740 1 -- start start 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 Processor -- start 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x5601802110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x560180211b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56017ffc8700 con 0x56017fe39390 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560180102160 con 0x56017fe5a070 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5601802141b0 con 0x56017fe311b0 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e5923b740 10 client.0 _ll_register_callbacks cb 0x56018011b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57106/0 (socket says 172.21.15.29:57106) 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e43fff640 1 -- 172.21.15.29:0/2804440486 learned_addr learned my addr 172.21.15.29:0/2804440486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.375+0000 7f3e50a63640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x5601802110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e51264640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x560180211b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 -- 172.21.15.29:0/2804440486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 msgr2=0x5601802110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x5601802110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 -- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 msgr2=0x560180211b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x560180211b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 -- 172.21.15.29:0/2804440486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018012fc90 con 0x56017fe39390 2024-09-21T02:03:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e50a63640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x5601802110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e43fff640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0acd40 tx=0x7f3e3c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1190cbf62c61ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c020070 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3e3c0e4be0 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4800be30 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c0ac700 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e5923b740 10 client.18753 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.376+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56018021fea0 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4800be30 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f3e3c0d41f0 con 0x56017fe39390 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 10 client.18753 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 10 client.18753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 20 client.18753 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe189f0 0x560180223450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 4 client.18753 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560180179df0 con 0x56017fe189f0 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.377+0000 7f3e50a63640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe189f0 0x560180223450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.378+0000 7f3e2bfff640 10 client.18753 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.378+0000 7f3e50a63640 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe189f0 0x560180223450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40ab608ae3f21077 server_cookie=6007eae3cf7dfb02 in_seq=0 out_seq=0 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.378+0000 7f3e2bfff640 1 -- 172.21.15.29:0/2804440486 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560180179df0 con 0x56017fe189f0 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.378+0000 7f3e2bfff640 10 client.18753 handle_command_reply: tid=0 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 client.18753 shutdown 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 msgr2=0x560180211610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0acd40 tx=0x7f3e3c009e60 comp rx=0 tx=0).stop 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 shutdown_connections 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe189f0 0x560180223450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e2bfff640 0 client.18753 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe5a070 0x560180211b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x560180211610 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 --2- 172.21.15.29:0/2804440486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x5601802110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 >> 172.21.15.29:0/2804440486 conn(0x56017fe4e460 msgr2=0x560180204830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 shutdown_connections 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 1 -- 172.21.15.29:0/2804440486 wait complete. 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.379+0000 7f3e5923b740 20 client.18753 trim_cache size 0 max 16384 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.392+0000 7f3e5923b740 1 Processor -- start 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 -- start start 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 0x5601802233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x560180210e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1ad170 con 0x56017fe39390 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1aecb0 con 0x56017fe189f0 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e5923b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1ad840 con 0x56017fe311b0 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57118/0 (socket says 172.21.15.29:57118) 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 -- 172.21.15.29:0/3198923705 learned_addr learned my addr 172.21.15.29:0/3198923705 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e43fff640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x560180210e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e50a63640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 0x5601802233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 -- 172.21.15.29:0/3198923705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 msgr2=0x560180210e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x560180210e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 -- 172.21.15.29:0/3198923705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 msgr2=0x5601802233c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 0x5601802233c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e51264640 1 -- 172.21.15.29:0/3198923705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018020c960 con 0x56017fe39390 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e43fff640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x560180210e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.393+0000 7f3e50a63640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 0x5601802233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.394+0000 7f3e51264640 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3e44031ec0 tx=0x7f3e44044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fad1971fc107c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.394+0000 7f3e327fc640 1 -- 172.21.15.29:0/3198923705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44003a30 con 0x56017fe39390 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.394+0000 7f3e327fc640 1 -- 172.21.15.29:0/3198923705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44036050 con 0x56017fe39390 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.394+0000 7f3e327fc640 1 -- 172.21.15.29:0/3198923705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4402f8c0 con 0x56017fe39390 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- 172.21.15.29:0/3198923705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 msgr2=0x5601802113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3e44031ec0 tx=0x7f3e44044000 comp rx=0 tx=0).stop 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- 172.21.15.29:0/3198923705 shutdown_connections 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe39390 0x5601802113c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe311b0 0x560180210e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 --2- 172.21.15.29:0/3198923705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe189f0 0x5601802233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- 172.21.15.29:0/3198923705 >> 172.21.15.29:0/3198923705 conn(0x56017fe4e460 msgr2=0x56018020f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- 172.21.15.29:0/3198923705 shutdown_connections 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- 172.21.15.29:0/3198923705 wait complete. 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 -- start start 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.395+0000 7f3e5923b740 1 Processor -- start 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 0x5601802159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180215ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1ae5e0 con 0x56017fe189f0 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1adf10 con 0x56017fe311b0 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1ab170 con 0x56017fe39390 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e5923b740 10 client.0 _ll_register_callbacks cb 0x7f3e3c005ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57128/0 (socket says 172.21.15.29:57128) 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 -- 172.21.15.29:0/1035507767 learned_addr learned my addr 172.21.15.29:0/1035507767 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e51264640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180215ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e43fff640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 0x5601802159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 -- 172.21.15.29:0/1035507767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 msgr2=0x560180215ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180215ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 -- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 msgr2=0x5601802159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 0x5601802159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 -- 172.21.15.29:0/1035507767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018028ce80 con 0x56017fe189f0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e43fff640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 0x5601802159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e50a63640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e480115e0 tx=0x7f3e4800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0418699baec310 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.396+0000 7f3e2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48015040 con 0x56017fe189f0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f3e48004450 con 0x56017fe189f0 2024-09-21T02:03:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e44036050 con 0x56017fe189f0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3e48005040 con 0x56017fe189f0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 10 client.18759 fetch_fsmap learned FSMap version 51 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56018028e070 con 0x56017fe189f0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44036050 con 0x56017fe189f0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f3e48011bb0 con 0x56017fe189f0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 10 client.18759 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 10 client.18759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 20 client.18759 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe5a070 0x560180290600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 4 client.18759 mds_command: new command op to 24644 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.397+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 --> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601801fa9e0 con 0x56017fe5a070 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e43fff640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe5a070 0x560180290600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e43fff640 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe5a070 0x560180290600 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bedca384137bc22 server_cookie=29fc508bc0275930 in_seq=0 out_seq=0 2024-09-21T02:03:56.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e2bfff640 10 client.18759 ms_handle_connect on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e2bfff640 1 -- 172.21.15.29:0/1035507767 <== mds.1 v2:172.21.15.66:6836/1538999358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601801fa9e0 con 0x56017fe5a070 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e2bfff640 10 client.18759 handle_command_reply: tid=0 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 client.18759 shutdown 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 msgr2=0x560180215470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e480115e0 tx=0x7f3e4800b940 comp rx=0 tx=0).stop 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 shutdown_connections 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:6836/1538999358,v1:172.21.15.66:6837/1538999358] conn(0x56017fe5a070 0x560180290600 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56017fe39390 0x560180215ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56017fe311b0 0x5601802159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 --2- 172.21.15.29:0/1035507767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56017fe189f0 0x560180215470 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e2bfff640 0 client.18759 ms_handle_reset on v2:172.21.15.66:6836/1538999358 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.398+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 >> 172.21.15.29:0/1035507767 conn(0x56017fe4e460 msgr2=0x56018020f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 shutdown_connections 2024-09-21T02:03:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e5923b740 1 -- 172.21.15.29:0/1035507767 wait complete. 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e5923b740 20 client.18759 trim_cache size 0 max 16384 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 msgr2=0x7f3e340b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 0x7f3e340b8a20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006010 tx=0x7f3e44009440 comp rx=0 tx=0).stop 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 msgr2=0x7f3e4c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00d740 tx=0x7f3e3c00dc00 comp rx=0 tx=0).stop 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 shutdown_connections 2024-09-21T02:03:56.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f3e340b6510 0x7f3e340b8a20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c108d20 0x7f3e4c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e4c1083d0 0x7f3e4c19a9f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 --2- 172.21.15.29:0/2033365767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e4c102ae0 0x7f3e4c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 >> 172.21.15.29:0/2033365767 conn(0x7f3e4c079760 msgr2=0x7f3e4c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.399+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 shutdown_connections 2024-09-21T02:03:56.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:56.400+0000 7f3e52cc7640 1 -- 172.21.15.29:0/2033365767 wait complete. 2024-09-21T02:03:57.531 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:03:57.798 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:58.194 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:57 smithi029 ceph-mon[31228]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2024-09-21T02:03:58.194 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:57 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:57 smithi190 ceph-mon[34368]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2024-09-21T02:03:58.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:57 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:57 smithi066 ceph-mon[35001]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2024-09-21T02:03:58.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:57 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 -- 172.21.15.29:0/3034348502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de81083d0 msgr2=0x7f4de81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 --2- 172.21.15.29:0/3034348502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de81083d0 0x7f4de81087e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02a3f0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 -- 172.21.15.29:0/3034348502 shutdown_connections 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 --2- 172.21.15.29:0/3034348502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8108d20 0x7f4de810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 --2- 172.21.15.29:0/3034348502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de81083d0 0x7f4de81087e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 --2- 172.21.15.29:0/3034348502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de8102ae0 0x7f4de8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 -- 172.21.15.29:0/3034348502 >> 172.21.15.29:0/3034348502 conn(0x7f4de8079760 msgr2=0x7f4de8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 -- 172.21.15.29:0/3034348502 shutdown_connections 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.286+0000 7f4dee44e640 1 -- 172.21.15.29:0/3034348502 wait complete. 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 Processor -- start 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 -- start start 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 0x7f4de81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 0x7f4de8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112990 con 0x7f4de8108d20 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112810 con 0x7f4de8102ae0 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dee44e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112b10 con 0x7f4de81083d0 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dec9eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dec9eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57148/0 (socket says 172.21.15.29:57148) 2024-09-21T02:03:58.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4dec9eb640 1 -- 172.21.15.29:0/2087557027 learned_addr learned my addr 172.21.15.29:0/2087557027 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4de7fff640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 0x7f4de81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.287+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 0x7f4de8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 -- 172.21.15.29:0/2087557027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 msgr2=0x7f4de8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 0x7f4de8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 -- 172.21.15.29:0/2087557027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 msgr2=0x7f4de81993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 0x7f4de81993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 -- 172.21.15.29:0/2087557027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de819f310 con 0x7f4de8108d20 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4de7fff640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 0x7f4de81993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 0x7f4de8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009fd0 tx=0x7f4de000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4de57fa640 1 -- 172.21.15.29:0/2087557027 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0017070 con 0x7f4de8108d20 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4de57fa640 1 -- 172.21.15.29:0/2087557027 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de0003e20 con 0x7f4de8108d20 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de819f600 con 0x7f4de8108d20 2024-09-21T02:03:58.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4de57fa640 1 -- 172.21.15.29:0/2087557027 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0010460 con 0x7f4de8108d20 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.288+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a5d60 con 0x7f4de8108d20 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.291+0000 7f4df49c2740 1 Processor -- start 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de57fa640 1 -- 172.21.15.29:0/2087557027 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4de0003970 con 0x7f4de8108d20 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 -- start start 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcaef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcaefd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0bca65ea0 con 0x55e0bc71f1b0 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e0bca552d0 con 0x55e0bc727390 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4df49c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0bca590e0 con 0x55e0bc748070 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57162/0 (socket says 172.21.15.29:57162) 2024-09-21T02:03:58.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 -- 172.21.15.29:0/2670543216 learned_addr learned my addr 172.21.15.29:0/2670543216 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcaef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 -- 172.21.15.29:0/2670543216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 msgr2=0x55e0bcaef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcaef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcaefd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 -- 172.21.15.29:0/2670543216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 msgr2=0x55e0bcaefd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcaefd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de7fff640 1 -- 172.21.15.29:0/2670543216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0bc7b3650 con 0x55e0bc71f1b0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.292+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcaefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4de7fff640 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bd40 tx=0x7f4dd800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8b6ca036e44c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4de57fa640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 0x7f4dd00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4dce7fc640 1 -- 172.21.15.29:0/2670543216 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8015070 con 0x55e0bc71f1b0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 0x7f4dd00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4dce7fc640 1 -- 172.21.15.29:0/2670543216 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd800be30 con 0x55e0bc71f1b0 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4dce7fc640 1 -- 172.21.15.29:0/2670543216 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80106e0 con 0x55e0bc71f1b0 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.293+0000 7f4de57fa640 1 -- 172.21.15.29:0/2087557027 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4de00e7170 con 0x7f4de8108d20 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- 172.21.15.29:0/2670543216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 msgr2=0x55e0bcaef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bd40 tx=0x7f4dd800f730 comp rx=0 tx=0).stop 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4de77fe640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 0x7f4dd00b8a20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- 172.21.15.29:0/2670543216 shutdown_connections 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcaefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaef800 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 --2- 172.21.15.29:0/2670543216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcaef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- 172.21.15.29:0/2670543216 >> 172.21.15.29:0/2670543216 conn(0x55e0bc73c460 msgr2=0x55e0bcaed1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- 172.21.15.29:0/2670543216 shutdown_connections 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- 172.21.15.29:0/2670543216 wait complete. 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.294+0000 7f4df49c2740 1 -- start start 2024-09-21T02:03:58.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 Processor -- start 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcafd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0bc8b6700 con 0x55e0bc71f1b0 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e0bc9f0160 con 0x55e0bc727390 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0bcaffbd0 con 0x55e0bc748070 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4df49c2740 10 client.0 _ll_register_callbacks cb 0x55e0bca099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57170/0 (socket says 172.21.15.29:57170) 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4de7fff640 1 -- 172.21.15.29:0/1348208027 learned_addr learned my addr 172.21.15.29:0/1348208027 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4de77fe640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.295+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcafd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 -- 172.21.15.29:0/1348208027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 msgr2=0x55e0bcafd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcafd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 -- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 msgr2=0x55e0bcafd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 -- 172.21.15.29:0/1348208027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0bca1dc90 con 0x55e0bc71f1b0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de77fe640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4de7fff640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8011e80 tx=0x7f4dd8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23214c14318ee11d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4dc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8029060 con 0x55e0bc71f1b0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4dd8005480 con 0x55e0bc71f1b0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd800be30 con 0x55e0bc71f1b0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4dd8015050 con 0x55e0bc71f1b0 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4df49c2740 10 client.18771 fetch_fsmap learned FSMap version 52 2024-09-21T02:03:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e0bcb0b830 con 0x55e0bc71f1b0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.296+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd800be30 con 0x55e0bc71f1b0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f4dd8004b60 con 0x55e0bc71f1b0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 10 client.18771 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 20 client.18771 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc7069f0 0x55e0bcb0ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 4 client.18771 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0bca67df0 con 0x55e0bc7069f0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4de77fe640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc7069f0 0x55e0bcb0ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4dc77fe640 10 client.18771 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.297+0000 7f4de77fe640 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc7069f0 0x55e0bcb0ec40 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232b485ed08bee30 server_cookie=4344029a3efeda1c in_seq=0 out_seq=0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.298+0000 7f4dc77fe640 1 -- 172.21.15.29:0/1348208027 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0bca67df0 con 0x55e0bc7069f0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.298+0000 7f4dc77fe640 10 client.18771 handle_command_reply: tid=0 2024-09-21T02:03:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.298+0000 7f4df49c2740 1 client.18771 shutdown 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 msgr2=0x55e0bcafcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8011e80 tx=0x7f4dd8005040 comp rx=0 tx=0).stop 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 shutdown_connections 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc7069f0 0x55e0bcb0ec40 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc748070 0x55e0bcafd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 --2- 172.21.15.29:0/1348208027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcafcaf0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 >> 172.21.15.29:0/1348208027 conn(0x55e0bc73c460 msgr2=0x55e0bcaeb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4dc77fe640 0 client.18771 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 shutdown_connections 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 1 -- 172.21.15.29:0/1348208027 wait complete. 2024-09-21T02:03:58.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.299+0000 7f4df49c2740 20 client.18771 trim_cache size 0 max 16384 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.312+0000 7f4df49c2740 1 Processor -- start 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.312+0000 7f4df49c2740 1 -- start start 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.312+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.312+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaf9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.312+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4df49c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ad140 con 0x55e0bc73c460 2024-09-21T02:03:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4df49c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81aecb0 con 0x55e0bc727390 2024-09-21T02:03:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4df49c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ad810 con 0x55e0bc71f1b0 2024-09-21T02:03:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57172/0 (socket says 172.21.15.29:57172) 2024-09-21T02:03:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 -- 172.21.15.29:0/677759829 learned_addr learned my addr 172.21.15.29:0/677759829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de77fe640 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaf9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 -- 172.21.15.29:0/677759829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc71f1b0 msgr2=0x55e0bcaf9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaf9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 -- 172.21.15.29:0/677759829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 msgr2=0x55e0bcafa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 -- 172.21.15.29:0/677759829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0bcaf52b0 con 0x55e0bc73c460 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4de7fff640 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc90 tx=0x7f4dd800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fc7ca28cc5e900 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4dcdffb640 1 -- 172.21.15.29:0/677759829 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8034030 con 0x55e0bc73c460 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4dcdffb640 1 -- 172.21.15.29:0/677759829 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd8005480 con 0x55e0bc73c460 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.313+0000 7f4dcdffb640 1 -- 172.21.15.29:0/677759829 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8029070 con 0x55e0bc73c460 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 -- 172.21.15.29:0/677759829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 msgr2=0x55e0bcaf0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc90 tx=0x7f4dd800fc70 comp rx=0 tx=0).stop 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 -- 172.21.15.29:0/677759829 shutdown_connections 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcafa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc71f1b0 0x55e0bcaf9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 --2- 172.21.15.29:0/677759829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc73c460 0x55e0bcaf0a00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 -- 172.21.15.29:0/677759829 >> 172.21.15.29:0/677759829 conn(0x55e0bcae8390 msgr2=0x55e0bcaf02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 -- 172.21.15.29:0/677759829 shutdown_connections 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.314+0000 7f4df49c2740 1 -- 172.21.15.29:0/677759829 wait complete. 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 -- start start 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 Processor -- start 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcb10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc73c460 0x55e0bcb107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ae5e0 con 0x55e0bc71f1b0 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81adf10 con 0x55e0bc727390 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ab120 con 0x55e0bc73c460 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4df49c2740 10 client.0 _ll_register_callbacks cb 0x7f4dd800bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57178/0 (socket says 172.21.15.29:57178) 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4de7fff640 1 -- 172.21.15.29:0/395929451 learned_addr learned my addr 172.21.15.29:0/395929451 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4de77fe640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcb10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.315+0000 7f4dec9eb640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc73c460 0x55e0bcb107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 -- 172.21.15.29:0/395929451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc73c460 msgr2=0x55e0bcb107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc73c460 0x55e0bcb107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 -- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 msgr2=0x55e0bcb10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcb10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 -- 172.21.15.29:0/395929451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0bcb7a890 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4de7fff640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4dd801a480 tx=0x7f4dd801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24015d249b0f4248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4dc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd803d070 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4dd8004a90 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd8005480 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4dd8049050 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4df49c2740 10 client.18777 fetch_fsmap learned FSMap version 52 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.316+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e0bcb7bb10 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd8005480 con 0x55e0bc71f1b0 2024-09-21T02:03:58.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f4dd80118f0 con 0x55e0bc71f1b0 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 10 client.18777 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 20 client.18777 populate_metadata read hostname 'smithi029' 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc748070 0x55e0bcb7e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 4 client.18777 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 --> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0bca638a0 con 0x55e0bc748070 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.317+0000 7f4de77fe640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc748070 0x55e0bcb7e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4dc77fe640 10 client.18777 ms_handle_connect on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4de77fe640 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc748070 0x55e0bcb7e190 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41623792a8d55183 server_cookie=c6be8e3bf5e12db2 in_seq=0 out_seq=0 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4dc77fe640 1 -- 172.21.15.29:0/395929451 <== mds.0 v2:172.21.15.66:6838/2350524023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0bca638a0 con 0x55e0bc748070 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4dc77fe640 10 client.18777 handle_command_reply: tid=0 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 client.18777 shutdown 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 msgr2=0x55e0bcb0fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4dd801a480 tx=0x7f4dd801acd0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 shutdown_connections 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:6838/2350524023,v1:172.21.15.66:6839/2350524023] conn(0x55e0bc748070 0x55e0bcb7e190 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0bc73c460 0x55e0bcb107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0bc727390 0x55e0bcb10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 --2- 172.21.15.29:0/395929451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0bc71f1b0 0x55e0bcb0fd30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 >> 172.21.15.29:0/395929451 conn(0x55e0bcae8390 msgr2=0x55e0bcaf4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4dc77fe640 0 client.18777 ms_handle_reset on v2:172.21.15.66:6838/2350524023 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 shutdown_connections 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 1 -- 172.21.15.29:0/395929451 wait complete. 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.318+0000 7f4df49c2740 20 client.18777 trim_cache size 0 max 16384 2024-09-21T02:03:58.324 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 msgr2=0x7f4dd00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 0x7f4dd00b8a20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc031400 comp rx=0 tx=0).stop 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 msgr2=0x7f4de819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009fd0 tx=0x7f4de000b260 comp rx=0 tx=0).stop 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 shutdown_connections 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4dd00b6510 0x7f4dd00b8a20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8108d20 0x7f4de819ebe0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de81083d0 0x7f4de8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 --2- 172.21.15.29:0/2087557027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de8102ae0 0x7f4de81993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 >> 172.21.15.29:0/2087557027 conn(0x7f4de8079760 msgr2=0x7f4de8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 shutdown_connections 2024-09-21T02:03:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:58.319+0000 7f4dee44e640 1 -- 172.21.15.29:0/2087557027 wait complete. 2024-09-21T02:03:59.209 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T02:03:59.210 INFO:tasks.fwd_scrub:done joining 2024-09-21T02:03:59.210 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T02:03:59.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:59.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 -- 172.21.15.29:0/910803776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 msgr2=0x7fbc9010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 --2- 172.21.15.29:0/910803776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc9010a940 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009ca0 tx=0x7fbc8802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 -- 172.21.15.29:0/910803776 shutdown_connections 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 --2- 172.21.15.29:0/910803776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc9010a940 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 --2- 172.21.15.29:0/910803776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 0x7fbc90103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 --2- 172.21.15.29:0/910803776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc901029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 -- 172.21.15.29:0/910803776 >> 172.21.15.29:0/910803776 conn(0x7fbc900fdee0 msgr2=0x7fbc90100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 -- 172.21.15.29:0/910803776 shutdown_connections 2024-09-21T02:03:59.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.339+0000 7fbc94a69640 1 -- 172.21.15.29:0/910803776 wait complete. 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 Processor -- start 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 -- start start 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc90198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 0x7fbc90199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e1c0 con 0x7fbc90103fa0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e040 con 0x7fbc90102580 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc94a69640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e340 con 0x7fbc90103600 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57200/0 (socket says 172.21.15.29:57200) 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 -- 172.21.15.29:0/507707708 learned_addr learned my addr 172.21.15.29:0/507707708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8dd9b640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 0x7fbc90199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8e59c640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc90198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 -- 172.21.15.29:0/507707708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 msgr2=0x7fbc90199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 0x7fbc90199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 -- 172.21.15.29:0/507707708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 msgr2=0x7fbc90198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc90198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8ed9d640 1 -- 172.21.15.29:0/507707708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9019a7f0 con 0x7fbc90103fa0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.340+0000 7fbc8e59c640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc90198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc8ed9d640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbc880097b0 tx=0x7fbc88003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8804a070 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9019aa80 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc880317c0 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8802fae0 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.341+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9006bb30 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.342+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc901029f0 con 0x7fbc90103fa0 2024-09-21T02:03:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.343+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbc8804d070 con 0x7fbc90103fa0 2024-09-21T02:03:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.344+0000 7fbc7f7fe640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 0x7fbc780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.345+0000 7fbc8e59c640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 0x7fbc780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.345+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fbc88109ba0 con 0x7fbc90103fa0 2024-09-21T02:03:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.345+0000 7fbc8e59c640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 0x7fbc780b8a20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009b40 tx=0x7fbc80005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.349+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc88084710 con 0x7fbc90103fa0 2024-09-21T02:03:59.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.543+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc90109260 con 0x7fbc90103fa0 2024-09-21T02:03:59.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.545+0000 7fbc7f7fe640 1 -- 172.21.15.29:0/507707708 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7fbc880d03e0 con 0x7fbc90103fa0 2024-09-21T02:03:59.547 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:03:59.547 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 msgr2=0x7fbc780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 0x7fbc780b8a20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009b40 tx=0x7fbc80005e50 comp rx=0 tx=0).stop 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 msgr2=0x7fbc90199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbc880097b0 tx=0x7fbc88003ba0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 shutdown_connections 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fbc780b6510 0x7fbc780b8a20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc90103fa0 0x7fbc90199fd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc90103600 0x7fbc90199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 --2- 172.21.15.29:0/507707708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc90102580 0x7fbc90198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 >> 172.21.15.29:0/507707708 conn(0x7fbc900fdee0 msgr2=0x7fbc90106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 shutdown_connections 2024-09-21T02:03:59.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.547+0000 7fbc94a69640 1 -- 172.21.15.29:0/507707708 wait complete. 2024-09-21T02:03:59.561 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:41:30.465145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T01:46:06.321761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T01:46:08.562000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:03:59.561 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:03:59.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 -- 172.21.15.29:0/3140377649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 msgr2=0x7f4a4010daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3140377649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4010daf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009ca0 tx=0x7f4a3802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 -- 172.21.15.29:0/3140377649 shutdown_connections 2024-09-21T02:03:59.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3140377649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4010daf0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3140377649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40075c70 0x7f4a40076080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3140377649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40077070 0x7f4a40075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 -- 172.21.15.29:0/3140377649 >> 172.21.15.29:0/3140377649 conn(0x7f4a400ff020 msgr2=0x7f4a40101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 -- 172.21.15.29:0/3140377649 shutdown_connections 2024-09-21T02:03:59.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.676+0000 7f4a4524d640 1 -- 172.21.15.29:0/3140377649 wait complete. 2024-09-21T02:03:59.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.677+0000 7f4a4524d640 1 Processor -- start 2024-09-21T02:03:59.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.677+0000 7f4a4524d640 1 -- start start 2024-09-21T02:03:59.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40075c70 0x7f4a4019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 0x7f4a4019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401114d0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40111350 con 0x7f4a40075c70 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a4524d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40111650 con 0x7f4a40077070 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57216/0 (socket says 172.21.15.29:57216) 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 -- 172.21.15.29:0/3249231582 learned_addr learned my addr 172.21.15.29:0/3249231582 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3ed9d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40075c70 0x7f4a4019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3e59c640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 0x7f4a4019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 -- 172.21.15.29:0/3249231582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 msgr2=0x7f4a4019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 0x7f4a4019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 -- 172.21.15.29:0/3249231582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40075c70 msgr2=0x7f4a4019c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40075c70 0x7f4a4019c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 -- 172.21.15.29:0/3249231582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4019dba0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3e59c640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 0x7f4a4019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a3f59e640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009430 tx=0x7f4a38003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.678+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3802f380 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.679+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4019de30 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.679+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a380317c0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.679+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3802fcd0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.679+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a401a4580 con 0x7f4a4010b6b0 2024-09-21T02:03:59.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.680+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a40076080 con 0x7f4a4010b6b0 2024-09-21T02:03:59.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.682+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4a38004040 con 0x7f4a4010b6b0 2024-09-21T02:03:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.683+0000 7f4a27fff640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 0x7f4a280b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.683+0000 7f4a3ed9d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 0x7f4a280b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.683+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f4a38108950 con 0x7f4a4010b6b0 2024-09-21T02:03:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.684+0000 7f4a3ed9d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 0x7f4a280b8bc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005d00 tx=0x7f4a3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.686+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a380cf190 con 0x7f4a4010b6b0 2024-09-21T02:03:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.875+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4a40063fc0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.876+0000 7f4a27fff640 1 -- 172.21.15.29:0/3249231582 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f4a380098b0 con 0x7f4a4010b6b0 2024-09-21T02:03:59.877 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 msgr2=0x7f4a280b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 0x7f4a280b8bc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005d00 tx=0x7f4a3000b410 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 msgr2=0x7f4a4019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009430 tx=0x7f4a38003ba0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 shutdown_connections 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4a280b66b0 0x7f4a280b8bc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4010b6b0 0x7f4a4019d380 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a40077070 0x7f4a4019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 --2- 172.21.15.29:0/3249231582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a40075c70 0x7f4a4019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 >> 172.21.15.29:0/3249231582 conn(0x7f4a400ff020 msgr2=0x7f4a40107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 shutdown_connections 2024-09-21T02:03:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:03:59.878+0000 7f4a4524d640 1 -- 172.21.15.29:0/3249231582 wait complete. 2024-09-21T02:03:59.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 -- 172.21.15.29:0/1034780388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 msgr2=0x7f904410ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 --2- 172.21.15.29:0/1034780388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904410ba50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f903c009ca0 tx=0x7f903c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 -- 172.21.15.29:0/1034780388 shutdown_connections 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 --2- 172.21.15.29:0/1034780388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904410ba50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 --2- 172.21.15.29:0/1034780388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044104710 0x7f9044104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 --2- 172.21.15.29:0/1034780388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044103690 0x7f9044103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.007+0000 7f904bd20640 1 -- 172.21.15.29:0/1034780388 >> 172.21.15.29:0/1034780388 conn(0x7f90440fefd0 msgr2=0x7f9044101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.008+0000 7f904bd20640 1 -- 172.21.15.29:0/1034780388 shutdown_connections 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.008+0000 7f904bd20640 1 -- 172.21.15.29:0/1034780388 wait complete. 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.008+0000 7f904bd20640 1 Processor -- start 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.008+0000 7f904bd20640 1 -- start start 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.008+0000 7f904bd20640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 0x7f9044199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904bd20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044104710 0x7f904419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904bd20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904bd20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f904410f2c0 con 0x7f90441050b0 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904bd20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f904410f140 con 0x7f9044104710 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904bd20640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f904410f440 con 0x7f9044103690 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57246/0 (socket says 172.21.15.29:57246) 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 -- 172.21.15.29:0/2451914505 learned_addr learned my addr 172.21.15.29:0/2451914505 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f90492bb640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044104710 0x7f904419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f9049abc640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 0x7f9044199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 -- 172.21.15.29:0/2451914505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 msgr2=0x7f9044199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 0x7f9044199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 -- 172.21.15.29:0/2451914505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044104710 msgr2=0x7f904419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044104710 0x7f904419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 -- 172.21.15.29:0/2451914505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904419b8f0 con 0x7f90441050b0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f9049abc640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 0x7f9044199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f904a2bd640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097b0 tx=0x7f903c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.009+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c004240 con 0x7f90441050b0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.010+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904419bb80 con 0x7f90441050b0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.010+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f903c02f310 con 0x7f90441050b0 2024-09-21T02:04:00.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.010+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c02fb50 con 0x7f90441050b0 2024-09-21T02:04:00.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.010+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90441a22d0 con 0x7f90441050b0 2024-09-21T02:04:00.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.010+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9044103b00 con 0x7f90441050b0 2024-09-21T02:04:00.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.012+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f903c02fd30 con 0x7f90441050b0 2024-09-21T02:04:00.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.013+0000 7f903affd640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 0x7f90300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.013+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f903c108960 con 0x7f90441050b0 2024-09-21T02:04:00.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.013+0000 7f9049abc640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 0x7f90300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.014+0000 7f9049abc640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 0x7f90300b8a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9040009c20 tx=0x7f9040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.016+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f903c0cf200 con 0x7f90441050b0 2024-09-21T02:04:00.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.205+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9044063fc0 con 0x7f90441050b0 2024-09-21T02:04:00.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.205+0000 7f903affd640 1 -- 172.21.15.29:0/2451914505 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) ==== 104+0+11 (secure 0 0 0) 0x7f903c0d35e0 con 0x7f90441050b0 2024-09-21T02:04:00.207 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 msgr2=0x7f90300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 0x7f90300b8a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9040009c20 tx=0x7f9040005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 msgr2=0x7f904419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097b0 tx=0x7f903c003c20 comp rx=0 tx=0).stop 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 shutdown_connections 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f90300b6510 0x7f90300b8a20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90441050b0 0x7f904419b0d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9044104710 0x7f904419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 --2- 172.21.15.29:0/2451914505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9044103690 0x7f9044199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.207+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 >> 172.21.15.29:0/2451914505 conn(0x7f90440fefd0 msgr2=0x7f9044107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.208+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 shutdown_connections 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.208+0000 7f904bd20640 1 -- 172.21.15.29:0/2451914505 wait complete. 2024-09-21T02:04:00.219 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:59 smithi029 ceph-mon[31228]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T02:04:00.220 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:59 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/507707708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:00.220 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:03:59 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3249231582' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:00.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:04:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:59 smithi190 ceph-mon[34368]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T02:04:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:59 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/507707708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:00.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:03:59 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3249231582' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:59 smithi066 ceph-mon[35001]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T02:04:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:59 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/507707708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:00.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:03:59 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3249231582' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 -- 172.21.15.29:0/1656615942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 msgr2=0x7f9dc010a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/1656615942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc010a7e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009ca0 tx=0x7f9db802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 -- 172.21.15.29:0/1656615942 shutdown_connections 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/1656615942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc010a7e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/1656615942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 0x7f9dc0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/1656615942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 0x7f9dc0105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 -- 172.21.15.29:0/1656615942 >> 172.21.15.29:0/1656615942 conn(0x7f9dc00fd660 msgr2=0x7f9dc00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 -- 172.21.15.29:0/1656615942 shutdown_connections 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.335+0000 7f9dc7af9640 1 -- 172.21.15.29:0/1656615942 wait complete. 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 Processor -- start 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 -- start start 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 0x7f9dc01020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 0x7f9dc019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc010e1e0 con 0x7f9dc01082d0 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc010e060 con 0x7f9dc0102c30 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc7af9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc010e360 con 0x7f9dc01056b0 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc6096640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.336+0000 7f9dc6096640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57274/0 (socket says 172.21.15.29:57274) 2024-09-21T02:04:00.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 -- 172.21.15.29:0/2880897660 learned_addr learned my addr 172.21.15.29:0/2880897660 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc5895640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 0x7f9dc01020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc5094640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 0x7f9dc019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 -- 172.21.15.29:0/2880897660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 msgr2=0x7f9dc019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 0x7f9dc019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 -- 172.21.15.29:0/2880897660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 msgr2=0x7f9dc01020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 0x7f9dc01020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 -- 172.21.15.29:0/2880897660 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc01a18d0 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc6096640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097b0 tx=0x7f9db8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004240 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc01a1b60 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db802f380 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db802fc20 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.337+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a2100 con 0x7f9dc01082d0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.338+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00ff180 con 0x7f9dc01082d0 2024-09-21T02:04:00.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.340+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9db80043e0 con 0x7f9dc01082d0 2024-09-21T02:04:00.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.341+0000 7f9db2ffd640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 0x7f9dac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.341+0000 7f9dc5895640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 0x7f9dac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.341+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f9db8108be0 con 0x7f9dc01082d0 2024-09-21T02:04:00.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.341+0000 7f9dc5895640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 0x7f9dac0b8a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00abe0 tx=0x7f9dbc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.344+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9db80cf420 con 0x7f9dc01082d0 2024-09-21T02:04:00.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.534+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9dc002e7d0 con 0x7f9dc01082d0 2024-09-21T02:04:00.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.534+0000 7f9db2ffd640 1 -- 172.21.15.29:0/2880897660 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) ==== 100+0+11 (secure 0 0 0) 0x7f9db80d3800 con 0x7f9dc01082d0 2024-09-21T02:04:00.536 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-09-21T02:04:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.536+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 msgr2=0x7f9dac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.536+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 0x7f9dac0b8a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00abe0 tx=0x7f9dbc00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.536+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 msgr2=0x7f9dc01a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.536+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097b0 tx=0x7f9db8003c20 comp rx=0 tx=0).stop 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 shutdown_connections 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f9dac0b6510 0x7f9dac0b8a20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dc01082d0 0x7f9dc01a1160 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dc01056b0 0x7f9dc019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 --2- 172.21.15.29:0/2880897660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc0102c30 0x7f9dc01020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 >> 172.21.15.29:0/2880897660 conn(0x7f9dc00fd660 msgr2=0x7f9dc0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 shutdown_connections 2024-09-21T02:04:00.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.537+0000 7f9dc7af9640 1 -- 172.21.15.29:0/2880897660 wait complete. 2024-09-21T02:04:00.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T02:04:00.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 -- 172.21.15.29:0/1618073084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43101025a0 msgr2=0x7f4310102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 --2- 172.21.15.29:0/1618073084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43101025a0 0x7f4310102a10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4300009900 tx=0x7f430002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 -- 172.21.15.29:0/1618073084 shutdown_connections 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 --2- 172.21.15.29:0/1618073084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4310103fc0 0x7f431010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 --2- 172.21.15.29:0/1618073084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 0x7f4310103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 --2- 172.21.15.29:0/1618073084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43101025a0 0x7f4310102a10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 -- 172.21.15.29:0/1618073084 >> 172.21.15.29:0/1618073084 conn(0x7f43100fdee0 msgr2=0x7f4310100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 -- 172.21.15.29:0/1618073084 shutdown_connections 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.661+0000 7f4316139640 1 -- 172.21.15.29:0/1618073084 wait complete. 2024-09-21T02:04:00.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 Processor -- start 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 -- start start 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 0x7f4310199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 0x7f431019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e1f0 con 0x7f4310103fc0 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.662+0000 7f4316139640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e070 con 0x7f43101025a0 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f4316139640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e370 con 0x7f4310103620 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57294/0 (socket says 172.21.15.29:57294) 2024-09-21T02:04:00.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 -- 172.21.15.29:0/1778405000 learned_addr learned my addr 172.21.15.29:0/1778405000 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430f7fe640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 0x7f4310199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430effd640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 0x7f431019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 -- 172.21.15.29:0/1778405000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 msgr2=0x7f431019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 0x7f431019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 -- 172.21.15.29:0/1778405000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 msgr2=0x7f4310199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 0x7f4310199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 -- 172.21.15.29:0/1778405000 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431019b8d0 con 0x7f4310103fc0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430f7fe640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 0x7f4310199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430ffff640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f430800ccb0 tx=0x7f430800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308004b90 con 0x7f4310103fc0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431019bbc0 con 0x7f4310103fc0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4308004d30 con 0x7f4310103fc0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308005320 con 0x7f4310103fc0 2024-09-21T02:04:00.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.663+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a2340 con 0x7f4310103fc0 2024-09-21T02:04:00.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.665+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43101a2510 con 0x7f4310103fc0 2024-09-21T02:04:00.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.667+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f430800f450 con 0x7f4310103fc0 2024-09-21T02:04:00.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.667+0000 7f430cff9640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 0x7f42f80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.667+0000 7f430f7fe640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 0x7f42f80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.667+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f43080e77c0 con 0x7f4310103fc0 2024-09-21T02:04:00.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.668+0000 7f430f7fe640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 0x7f42f80b8a20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4300031440 tx=0x7f4300005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.670+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4308014070 con 0x7f4310103fc0 2024-09-21T02:04:00.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.910+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f43101a26e0 con 0x7f4310103fc0 2024-09-21T02:04:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.918+0000 7f430cff9640 1 -- 172.21.15.29:0/1778405000 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+833519 (secure 0 0 0) 0x7f43080ae060 con 0x7f4310103fc0 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 18, 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "active_gid": 14152, 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "active_name": "x", 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "active_addrs": { 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.930 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 796732990 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 796732990 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "active_addr": "172.21.15.29:6801/796732990", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "active_change": "2024-09-21T01:38:31.990721+0000", 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T02:04:00.931 INFO:teuthology.orchestra.run.smithi029.stdout: "available": true, 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "standbys": [ 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 24109, 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "z", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_features": 4540719139924082687, 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "available_modules": [ 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "alerts", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.932 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.933 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.934 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:00.936 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.937 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_destination": { 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_destination", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.938 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_from_name": { 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_from_name", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "Ceph", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email From: name", 2024-09-21T02:04:00.939 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_host": { 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_host", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP server", 2024-09-21T02:04:00.940 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_password": { 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_password", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Password to authenticate with", 2024-09-21T02:04:00.941 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_port": { 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_port", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "465", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP port", 2024-09-21T02:04:00.942 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_sender": { 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_sender", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.943 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_ssl": { 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_ssl", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.944 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_user": { 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_user", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.945 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "User to authenticate as", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.946 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "balancer", 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:00.947 INFO:teuthology.orchestra.run.smithi029.stdout: "active": { 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "active", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.948 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_time": { 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_time", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0000", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_weekday": { 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_weekday", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_max_iterations": { 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "25", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "250", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_metrics": { 2024-09-21T02:04:00.951 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_metrics", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_step": { 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_step", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:00.952 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.5", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0.001", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "0.999", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "end_time": { 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_time", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.953 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2359", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "end_weekday": { 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_weekday", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:00.954 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.955 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.956 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:00.957 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.959 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "min_score": { 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_score", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "upmap", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-compat", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "none", 2024-09-21T02:04:00.961 INFO:teuthology.orchestra.run.smithi029.stdout: "read", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap-read" 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Balancer mode", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_ids": { 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_ids", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.963 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:00.964 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_deviation": { 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_deviation", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:00.965 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_optimizations": { 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.966 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm", 2024-09-21T02:04:00.967 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_down_multiplier": { 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_down_multiplier", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_refresh_rate": { 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_refresh_rate", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "20", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.969 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_starting_port": { 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_starting_port", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4721", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_ptrace": { 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_ptrace", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:04:00.971 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_interval": { 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_interval", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.972 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.7", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.973 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_log_destination": { 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm_log_destination", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "file", 2024-09-21T02:04:00.974 INFO:teuthology.orchestra.run.smithi029.stdout: "file,syslog", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "syslog" 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "cgroups_split": { 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cgroups_split", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.975 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "config_checks_enabled": { 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_checks_enabled", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.976 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "config_dashboard": { 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_dashboard", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.977 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_alertmanager": { 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_alertmanager", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:00.978 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_base": { 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_base", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:04:00.979 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_elasticsearch": { 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.980 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "elasticsearch container image", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_grafana": { 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_grafana", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.981 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_haproxy": { 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_haproxy", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "HAproxy container image", 2024-09-21T02:04:00.982 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_agent": { 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:04:00.983 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_collector": { 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_query": { 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger query container image", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.985 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_keepalived": { 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_keepalived", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Keepalived container image", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.986 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_loki": { 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_loki", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Loki container image", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.987 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nginx": { 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nginx", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nginx container image", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_node_exporter": { 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_node_exporter", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nvmeof": { 2024-09-21T02:04:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nvmeof", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nvme-of container image", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:04:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_prometheus": { 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_prometheus", 2024-09-21T02:04:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_promtail": { 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_promtail", 2024-09-21T02:04:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Promtail container image", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba": { 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba_metrics": { 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_snmp_gateway": { 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:04:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "container_init": { 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_init", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "daemon_cache_timeout": { 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "900", 2024-09-21T02:04:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "default_registry": { 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_registry", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "device_cache_timeout": { 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_cache_timeout", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1800", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "device_enhanced_scan": { 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_enhanced_scan", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "facts_cache_timeout": { 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "facts_cache_timeout", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "grafana_dashboards_path": { 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "host_check_interval": { 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "host_check_interval", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "hw_monitoring": { 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hw_monitoring", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "inventory_list_all": { 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "inventory_list_all", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:04:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "log_refresh_metadata": { 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_refresh_metadata", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:04:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:04:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "*", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "max_count_per_host": { 2024-09-21T02:04:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_count_per_host", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd_draining_count": { 2024-09-21T02:04:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_osd_draining_count", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "migration_current": { 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "migration_current", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "internal - do not modify", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "root", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm-package", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "root" 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "oob_default_addr": { 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "oob_default_addr", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "169.254.1.1", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "prometheus_alerts_path": { 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_insecure": { 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_insecure", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_password": { 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_password", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_url": { 2024-09-21T02:04:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_url", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_username": { 2024-09-21T02:04:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_username", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "secure_monitoring_stack": { 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:04:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "service_discovery_port": { 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "service_discovery_port", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8765", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_config_file": { 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_config_file", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_interval": { 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "use_agent": { 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_agent", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "use_repo_digest": { 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_repo_digest", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.031 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_failed_host_check": { 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_daemons": { 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_hosts": { 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crash", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "retain_interval": { 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retain_interval", 2024-09-21T02:04:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "31536000", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_recent_interval": { 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_recent_interval", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1209600", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dashboard", 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:04:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:04:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:04:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:04:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_URL": { 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "[]", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "{}", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:04:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "45", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:04:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:04:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "cross_origin_url": { 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cross_origin_url", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "crt_file": { 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crt_file", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "debug": { 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "debug", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "jwt_token_ttl": { 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "jwt_token_ttl", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "28800", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.097 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.098 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "motd": { 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "motd", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.099 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "The message of the day", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.100 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.101 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8080", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.102 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.103 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl_server_port": { 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl_server_port", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8443", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "sso_oauth2": { 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sso_oauth2", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "redirect", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect" 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "url_prefix": { 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url_prefix", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_monitoring": { 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_monitoring", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "monitor device health metrics", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "mark_out_threshold": { 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mark_out_threshold", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2419200", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": { 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_name", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "device_health_metrics", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "retention_period": { 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retention_period", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.118 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15552000", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_frequency": { 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_frequency", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "self_heal": { 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "self_heal", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.120 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.121 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.122 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_threshold": { 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_threshold", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7257600", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.123 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "diskprediction_local", 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.124 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.125 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.126 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.127 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "predict_interval": { 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predict_interval", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.129 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "predictor_model": { 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predictor_model", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "prophetstor", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "influx", 2024-09-21T02:04:01.133 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": false, 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "batch_size": { 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "batch_size", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5000", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.134 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "database": { 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "database", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "ceph", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.135 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": { 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hostname", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "30", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "5", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.137 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.138 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.139 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.140 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.141 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "password": { 2024-09-21T02:04:01.142 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "password", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "port": { 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "port", 2024-09-21T02:04:01.143 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8086", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server port", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.144 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.145 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "false", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.146 INFO:teuthology.orchestra.run.smithi029.stdout: "threads": { 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threads", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "32", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "username": { 2024-09-21T02:04:01.147 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "username", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "verify_ssl": { 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "verify_ssl", 2024-09-21T02:04:01.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "true", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.149 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "insights", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.150 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.152 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "iostat", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.156 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.158 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.160 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "k8sevents", 2024-09-21T02:04:01.162 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_event_retention_days": { 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:04:01.163 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "config_check_secs": { 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_check_secs", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "10", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:04:01.164 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.165 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.167 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.169 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "localpool", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "failure_domain": { 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "failure_domain", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "host", 2024-09-21T02:04:01.171 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.172 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.175 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.176 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": { 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_size", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.177 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "num_rep": { 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "num_rep", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.178 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": { 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_num", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "128", 2024-09-21T02:04:01.179 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "prefix": { 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prefix", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.180 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.181 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "subtree": { 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subtree", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "rack", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.182 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mds_autoscaler", 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.183 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.184 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.185 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.186 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.188 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirroring", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.190 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.191 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.192 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.194 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.195 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "nfs", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.196 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.197 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.198 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.199 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.200 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.201 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:01.202 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "fail_fs": { 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fail_fs", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.203 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.204 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.205 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.206 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator": { 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:01.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "rook", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "test_orchestrator" 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Orchestrator backend", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.209 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_perf_query", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.211 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.213 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.216 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_support", 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.217 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.222 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_autoscaler", 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.225 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.226 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.228 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "threshold": { 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threshold", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.230 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1.0", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "scaling threshold", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "progress", 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.231 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_pg_recovery_event": { 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.233 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.234 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.236 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.237 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "max_completed_events": { 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_completed_events", 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "50", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.239 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.240 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "cache": { 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cache", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.241 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "exclude_perf_counters": { 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "exclude_perf_counters", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.242 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.243 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.244 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.245 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools": { 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools", 2024-09-21T02:04:01.247 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.248 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "300", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_interval": { 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_interval", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15.0", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:01.250 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "9283", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.251 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.252 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "stale_cache_strategy": { 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stale_cache_strategy", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "log", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.253 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "default", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "default", 2024-09-21T02:04:01.254 INFO:teuthology.orchestra.run.smithi029.stdout: "error" 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:01.255 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.258 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.259 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.260 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_snap_create": { 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.261 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.262 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.263 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "trash_purge_schedule": { 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "trash_purge_schedule", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.264 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "restful", 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_auth": { 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_auth", 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.266 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.267 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.270 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.271 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "max_requests": { 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_requests", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.272 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.273 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.274 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.275 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rgw", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.276 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.277 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.278 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.279 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.280 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.281 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.282 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rook", 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.283 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.284 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.285 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.286 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.288 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "storage_class": { 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "storage_class", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "local", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.289 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "selftest", 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.291 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.292 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.293 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "roption1": { 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption1", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.295 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "roption2": { 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption2", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.296 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "xyz", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption1": { 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption1", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.297 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption2": { 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption2", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.298 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption3": { 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption3", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.299 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption4": { 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption4", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.300 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption5": { 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption5", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.301 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption6": { 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption6", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.302 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption7": { 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption7", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "42", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.304 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "testkey": { 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testkey", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.305 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "testlkey": { 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testlkey", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.306 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "testnewline": { 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testnewline", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.307 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smb", 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "internal_store_backend": { 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "internal_store_backend", 2024-09-21T02:04:01.308 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.310 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.311 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.312 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.313 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.314 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "update_orchestration": { 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "update_orchestration", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.315 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snap_schedule", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_m_granularity": { 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_m_granularity", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.316 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "dump_on_update": { 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dump_on_update", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.320 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.321 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.323 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stats", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.324 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.325 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.326 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.327 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.329 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "status", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.331 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.332 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.333 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.334 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telegraf", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "address": { 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "address", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.337 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.338 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.339 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.340 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.341 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.342 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.343 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.344 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telemetry", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_basic": { 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_basic", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_crash": { 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_crash", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_device": { 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_device", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_ident": { 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_ident", 2024-09-21T02:04:01.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_perf": { 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_perf", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.348 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "contact": { 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "contact", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "description": { 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "description", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.350 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "device_url": { 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_url", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.351 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.353 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "24", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "8", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "last_opt_revision": { 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "last_opt_revision", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.354 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard": { 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.355 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard_description": { 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard_description", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.356 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.357 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.358 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.359 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.360 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.361 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "organization": { 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "organization", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.362 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "proxy": { 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "proxy", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.363 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.364 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "url": { 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "test_orchestrator", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.367 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.369 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.370 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.371 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.372 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.373 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.374 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.375 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.376 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_clones": { 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_clones", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.377 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "periodic_async_work": { 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "periodic_async_work", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.378 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Periodically check for async work", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_delay": { 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.379 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.380 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.381 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix", 2024-09-21T02:04:01.382 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "discovery_interval": { 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "discovery_interval", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "100", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.383 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "identifier": { 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "identifier", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.384 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.385 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.386 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.387 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.389 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.390 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.391 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_host": { 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_host", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.392 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_port": { 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_port", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10051", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.393 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_sender": { 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_sender", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.394 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 24125, 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "y", 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_features": 4540719139924082687, 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "available_modules": [ 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "alerts", 2024-09-21T02:04:01.395 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.396 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.397 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.398 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.399 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_destination": { 2024-09-21T02:04:01.401 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_destination", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_from_name": { 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_from_name", 2024-09-21T02:04:01.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "Ceph", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email From: name", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_host": { 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_host", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.403 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP server", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_password": { 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_password", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.404 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Password to authenticate with", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_port": { 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_port", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "465", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP port", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_sender": { 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_sender", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.406 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_ssl": { 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_ssl", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:04:01.407 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_user": { 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_user", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "User to authenticate as", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.408 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "balancer", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "active": { 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "active", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.410 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_time": { 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_time", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0000", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.411 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_weekday": { 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_weekday", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.412 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_max_iterations": { 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "25", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "250", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:04:01.413 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_metrics": { 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_metrics", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:04:01.414 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_step": { 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_step", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.5", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0.001", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "0.999", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.415 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "end_time": { 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_time", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2359", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.416 INFO:teuthology.orchestra.run.smithi029.stdout: "end_weekday": { 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_weekday", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.417 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.418 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.419 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.420 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.421 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "min_score": { 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_score", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.422 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "upmap", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.423 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-compat", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "none", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "read", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap-read" 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Balancer mode", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_ids": { 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_ids", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.424 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.425 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.426 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_deviation": { 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_deviation", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:04:01.427 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_optimizations": { 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_down_multiplier": { 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_down_multiplier", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.429 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_refresh_rate": { 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_refresh_rate", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "20", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.430 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_starting_port": { 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_starting_port", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4721", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.431 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_ptrace": { 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_ptrace", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.432 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_interval": { 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_interval", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.7", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.434 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_log_destination": { 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm_log_destination", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "file", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "file,syslog", 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "syslog" 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.435 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "cgroups_split": { 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cgroups_split", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:04:01.436 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "config_checks_enabled": { 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_checks_enabled", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.437 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "config_dashboard": { 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_dashboard", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.438 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_alertmanager": { 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_alertmanager", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_base": { 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_base", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_elasticsearch": { 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.440 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "elasticsearch container image", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_grafana": { 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_grafana", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:04:01.441 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_haproxy": { 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_haproxy", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.442 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "HAproxy container image", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_agent": { 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.443 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_collector": { 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.444 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_query": { 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger query container image", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_keepalived": { 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_keepalived", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Keepalived container image", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_loki": { 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_loki", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Loki container image", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.447 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nginx": { 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nginx", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nginx container image", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_node_exporter": { 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_node_exporter", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.449 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nvmeof": { 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nvmeof", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nvme-of container image", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:04:01.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_prometheus": { 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_prometheus", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_promtail": { 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_promtail", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Promtail container image", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba": { 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.453 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba_metrics": { 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:04:01.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_snmp_gateway": { 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:04:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "container_init": { 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_init", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.456 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "daemon_cache_timeout": { 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.457 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "900", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "default_registry": { 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_registry", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io", 2024-09-21T02:04:01.459 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "device_cache_timeout": { 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_cache_timeout", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1800", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.460 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "device_enhanced_scan": { 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_enhanced_scan", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.461 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "facts_cache_timeout": { 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "facts_cache_timeout", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.462 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "grafana_dashboards_path": { 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:04:01.463 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "host_check_interval": { 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "host_check_interval", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "hw_monitoring": { 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hw_monitoring", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.465 INFO:teuthology.orchestra.run.smithi029.stdout: "inventory_list_all": { 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "inventory_list_all", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.466 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.467 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "log_refresh_metadata": { 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_refresh_metadata", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.468 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:04:01.469 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.470 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.471 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.473 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "*", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "max_count_per_host": { 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_count_per_host", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.474 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd_draining_count": { 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_osd_draining_count", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.475 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "migration_current": { 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "migration_current", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.476 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "internal - do not modify", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "root", 2024-09-21T02:04:01.477 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm-package", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "root" 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "oob_default_addr": { 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "oob_default_addr", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.478 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "169.254.1.1", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "prometheus_alerts_path": { 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_insecure": { 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_insecure", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.480 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_password": { 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_password", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_url": { 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_url", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.482 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_username": { 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_username", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "secure_monitoring_stack": { 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.484 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "service_discovery_port": { 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "service_discovery_port", 2024-09-21T02:04:01.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8765", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_config_file": { 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_config_file", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.487 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_interval": { 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:01.489 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "use_agent": { 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_agent", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.490 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "use_repo_digest": { 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_repo_digest", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.491 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_failed_host_check": { 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_daemons": { 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.493 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_hosts": { 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crash", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.497 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.499 INFO:teuthology.orchestra.run.smithi029.stdout: "retain_interval": { 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retain_interval", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "31536000", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.500 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.501 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_recent_interval": { 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_recent_interval", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1209600", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dashboard", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.503 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.504 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.505 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.506 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.507 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.509 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.510 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.511 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.513 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.514 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.516 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.517 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.518 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_URL": { 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.519 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.520 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.522 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.524 INFO:teuthology.orchestra.run.smithi029.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "[]", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:04:01.525 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "{}", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.526 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.527 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.528 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.530 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.531 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.532 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.533 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.534 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.535 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.536 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.537 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.538 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "45", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:04:01.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:04:01.541 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:04:01.542 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:04:01.543 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.544 INFO:teuthology.orchestra.run.smithi029.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.545 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "cross_origin_url": { 2024-09-21T02:04:01.548 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cross_origin_url", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "crt_file": { 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crt_file", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.549 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "debug": { 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "debug", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.550 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "jwt_token_ttl": { 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "jwt_token_ttl", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.551 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "28800", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.552 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.553 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.556 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.557 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "motd": { 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "motd", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.558 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "The message of the day", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.559 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.560 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8080", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.561 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:01.562 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl_server_port": { 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl_server_port", 2024-09-21T02:04:01.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8443", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.564 INFO:teuthology.orchestra.run.smithi029.stdout: "sso_oauth2": { 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sso_oauth2", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:01.565 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "redirect", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect" 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "url_prefix": { 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url_prefix", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_monitoring": { 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_monitoring", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "monitor device health metrics", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.569 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.570 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.571 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.572 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.573 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "mark_out_threshold": { 2024-09-21T02:04:01.574 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mark_out_threshold", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2419200", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": { 2024-09-21T02:04:01.575 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_name", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "device_health_metrics", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "retention_period": { 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retention_period", 2024-09-21T02:04:01.576 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15552000", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_frequency": { 2024-09-21T02:04:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_frequency", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "self_heal": { 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "self_heal", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.578 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.579 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_threshold": { 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_threshold", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.581 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7257600", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "diskprediction_local", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.583 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.584 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.585 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.586 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "predict_interval": { 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predict_interval", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:01.587 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "predictor_model": { 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predictor_model", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "prophetstor", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.588 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.589 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.590 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "influx", 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": false, 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "batch_size": { 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "batch_size", 2024-09-21T02:04:01.591 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5000", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "database": { 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "database", 2024-09-21T02:04:01.592 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "ceph", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": { 2024-09-21T02:04:01.593 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hostname", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "30", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "5", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.595 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.596 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.597 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.598 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.599 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.600 INFO:teuthology.orchestra.run.smithi029.stdout: "password": { 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "password", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "port": { 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "port", 2024-09-21T02:04:01.601 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8086", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server port", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.602 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.603 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "false", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "threads": { 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threads", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "32", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "username": { 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "username", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.605 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "verify_ssl": { 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "verify_ssl", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.606 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "true", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "insights", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.608 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.609 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.610 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.611 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.612 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "iostat", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.613 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.614 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.616 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.617 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.618 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "k8sevents", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_event_retention_days": { 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "config_check_secs": { 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_check_secs", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.620 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "10", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.621 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.622 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.624 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.625 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.626 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "localpool", 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "failure_domain": { 2024-09-21T02:04:01.627 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "failure_domain", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "host", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.628 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.629 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.630 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.631 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": { 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_size", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.633 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "num_rep": { 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "num_rep", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.634 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": { 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_num", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.635 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "128", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "prefix": { 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prefix", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.636 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.637 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "subtree": { 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subtree", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "rack", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.638 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mds_autoscaler", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.639 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.640 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.642 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.643 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.644 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirroring", 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.645 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.646 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.647 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.648 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.649 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.650 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "nfs", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.653 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.654 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.655 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.656 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "fail_fs": { 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fail_fs", 2024-09-21T02:04:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.658 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.659 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.660 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.661 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.662 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator": { 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.663 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "rook", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "test_orchestrator" 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Orchestrator backend", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.664 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_perf_query", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.665 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.666 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.667 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.668 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.669 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.670 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_support", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.672 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.673 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.674 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.675 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.676 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.677 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_autoscaler", 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.678 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.679 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.680 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.681 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.682 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.683 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.684 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "threshold": { 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threshold", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:01.685 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1.0", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "scaling threshold", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "progress", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.686 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_pg_recovery_event": { 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.687 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.688 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.689 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.690 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.691 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.692 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "max_completed_events": { 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_completed_events", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "50", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.693 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.694 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.695 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.696 INFO:teuthology.orchestra.run.smithi029.stdout: "cache": { 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cache", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.697 INFO:teuthology.orchestra.run.smithi029.stdout: "exclude_perf_counters": { 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "exclude_perf_counters", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.698 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.699 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.700 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.701 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.702 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools": { 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.703 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "300", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_interval": { 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_interval", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.705 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15.0", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.706 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.707 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "9283", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.708 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "stale_cache_strategy": { 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stale_cache_strategy", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "log", 2024-09-21T02:04:01.709 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.710 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "default", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "default", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "error" 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.712 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.713 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.715 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.716 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.717 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_snap_create": { 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.718 INFO:teuthology.orchestra.run.smithi029.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.719 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "trash_purge_schedule": { 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "trash_purge_schedule", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.720 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "restful", 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.721 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_auth": { 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_auth", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.722 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.724 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.725 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.727 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "max_requests": { 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_requests", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.728 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.730 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.731 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rgw", 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.732 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.733 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.734 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.735 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.736 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.737 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.738 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rook", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.739 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.740 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.741 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.742 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.743 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.744 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "storage_class": { 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "storage_class", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "local", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "selftest", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.747 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.748 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.750 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.751 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "roption1": { 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption1", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.752 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "roption2": { 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption2", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "xyz", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.753 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption1": { 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption1", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.754 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption2": { 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption2", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.755 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption3": { 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption3", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.756 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption4": { 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption4", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.757 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption5": { 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption5", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption6": { 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption6", 2024-09-21T02:04:01.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption7": { 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption7", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.759 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "42", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.760 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "testkey": { 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testkey", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.761 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "testlkey": { 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testlkey", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.762 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "testnewline": { 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testnewline", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.763 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smb", 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "internal_store_backend": { 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "internal_store_backend", 2024-09-21T02:04:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.765 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.766 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.767 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.768 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.769 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.770 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "update_orchestration": { 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "update_orchestration", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.771 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snap_schedule", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.772 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_m_granularity": { 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_m_granularity", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "dump_on_update": { 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dump_on_update", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.776 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.777 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.778 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.779 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.780 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stats", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.781 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.782 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.783 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.784 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.785 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "status", 2024-09-21T02:04:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.787 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.788 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.789 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.790 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telegraf", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.792 INFO:teuthology.orchestra.run.smithi029.stdout: "address": { 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "address", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.793 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.794 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.796 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.797 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.798 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.799 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telemetry", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.800 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_basic": { 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_basic", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_crash": { 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_crash", 2024-09-21T02:04:01.801 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_device": { 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_device", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.802 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_ident": { 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_ident", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.803 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_perf": { 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_perf", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.804 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "contact": { 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "contact", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.805 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "description": { 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "description", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.806 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "device_url": { 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_url", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.807 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "24", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "8", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "last_opt_revision": { 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "last_opt_revision", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.810 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard": { 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard_description": { 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard_description", 2024-09-21T02:04:01.811 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.812 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.813 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.814 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.815 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.816 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "organization": { 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "organization", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.817 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "proxy": { 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "proxy", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.818 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.819 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "url": { 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.820 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "test_orchestrator", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.821 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.822 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.823 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.824 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.825 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.826 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.827 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.828 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.829 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.830 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_clones": { 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_clones", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.832 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "periodic_async_work": { 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "periodic_async_work", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.833 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Periodically check for async work", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_delay": { 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.834 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.835 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.836 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.837 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "discovery_interval": { 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "discovery_interval", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "100", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.838 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "identifier": { 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "identifier", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.839 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.840 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.841 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.844 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.845 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.846 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_host": { 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_host", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.847 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_port": { 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_port", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10051", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.848 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_sender": { 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_sender", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.849 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "modules": [ 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "dashboard", 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "iostat", 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "nfs", 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "restful" 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: "available_modules": [ 2024-09-21T02:04:01.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "alerts", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.851 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.852 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.853 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.854 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.855 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.856 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_destination": { 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_destination", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.857 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_from_name": { 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_from_name", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "Ceph", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email From: name", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_host": { 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_host", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.859 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP server", 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.861 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_password": { 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_password", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Password to authenticate with", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.862 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_port": { 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_port", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "465", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP port", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_sender": { 2024-09-21T02:04:01.863 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_sender", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_ssl": { 2024-09-21T02:04:01.864 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_ssl", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_user": { 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_user", 2024-09-21T02:04:01.865 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "User to authenticate as", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.866 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "balancer", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "active": { 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "active", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_time": { 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_time", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0000", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.869 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_weekday": { 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_weekday", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.870 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_max_iterations": { 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "25", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "250", 2024-09-21T02:04:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_metrics": { 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_metrics", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.872 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_step": { 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_step", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.5", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0.001", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "0.999", 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.873 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "end_time": { 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_time", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2359", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.874 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "end_weekday": { 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_weekday", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:04:01.875 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.876 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.877 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.878 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.880 INFO:teuthology.orchestra.run.smithi029.stdout: "min_score": { 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_score", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.881 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "upmap", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-compat", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "none", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "read", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap", 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap-read" 2024-09-21T02:04:01.882 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Balancer mode", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_ids": { 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_ids", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.883 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.884 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.885 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_deviation": { 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_deviation", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.886 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_optimizations": { 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.887 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm", 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_down_multiplier": { 2024-09-21T02:04:01.888 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_down_multiplier", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_refresh_rate": { 2024-09-21T02:04:01.889 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_refresh_rate", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "20", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_starting_port": { 2024-09-21T02:04:01.890 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_starting_port", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4721", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_ptrace": { 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_ptrace", 2024-09-21T02:04:01.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_interval": { 2024-09-21T02:04:01.892 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_interval", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:04:01.893 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.7", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_log_destination": { 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm_log_destination", 2024-09-21T02:04:01.894 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "file", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "file,syslog", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "syslog" 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.895 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "cgroups_split": { 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cgroups_split", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "config_checks_enabled": { 2024-09-21T02:04:01.896 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_checks_enabled", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "config_dashboard": { 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_dashboard", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.897 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_alertmanager": { 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_alertmanager", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.898 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_base": { 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_base", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.899 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_elasticsearch": { 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:04:01.900 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "elasticsearch container image", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_grafana": { 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_grafana", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.901 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_haproxy": { 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_haproxy", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.902 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "HAproxy container image", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_agent": { 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.903 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_collector": { 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.904 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_query": { 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:04:01.905 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger query container image", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_keepalived": { 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_keepalived", 2024-09-21T02:04:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Keepalived container image", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_loki": { 2024-09-21T02:04:01.907 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_loki", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Loki container image", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nginx": { 2024-09-21T02:04:01.908 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nginx", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nginx container image", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.909 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_node_exporter": { 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_node_exporter", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nvmeof": { 2024-09-21T02:04:01.910 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nvmeof", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nvme-of container image", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:04:01.911 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_prometheus": { 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_prometheus", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_promtail": { 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_promtail", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Promtail container image", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba": { 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.915 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba_metrics": { 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.916 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_snmp_gateway": { 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.917 INFO:teuthology.orchestra.run.smithi029.stdout: "container_init": { 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_init", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "daemon_cache_timeout": { 2024-09-21T02:04:01.918 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:04:01.919 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "900", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "default_registry": { 2024-09-21T02:04:01.920 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_registry", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "device_cache_timeout": { 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_cache_timeout", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1800", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "device_enhanced_scan": { 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_enhanced_scan", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.922 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "facts_cache_timeout": { 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "facts_cache_timeout", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.923 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "grafana_dashboards_path": { 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.924 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "host_check_interval": { 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "host_check_interval", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.925 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "hw_monitoring": { 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hw_monitoring", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.926 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "inventory_list_all": { 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "inventory_list_all", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.927 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.928 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "log_refresh_metadata": { 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_refresh_metadata", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.929 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.930 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.931 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "*", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.935 INFO:teuthology.orchestra.run.smithi029.stdout: "max_count_per_host": { 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_count_per_host", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd_draining_count": { 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_osd_draining_count", 2024-09-21T02:04:01.936 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "migration_current": { 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "migration_current", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "internal - do not modify", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.938 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "root", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm-package", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "root" 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "oob_default_addr": { 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "oob_default_addr", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.939 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "169.254.1.1", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "prometheus_alerts_path": { 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.940 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_insecure": { 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_insecure", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.941 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_password": { 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_password", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.942 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_url": { 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_url", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.943 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_username": { 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_username", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.944 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "secure_monitoring_stack": { 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "service_discovery_port": { 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "service_discovery_port", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8765", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.946 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.947 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_config_file": { 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_config_file", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.948 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.949 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_keepalive_interval": { 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:01.950 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "use_agent": { 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_agent", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.951 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "use_repo_digest": { 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_repo_digest", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.952 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_failed_host_check": { 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.953 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_daemons": { 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.954 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_hosts": { 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.955 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crash", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.956 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.957 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.958 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "retain_interval": { 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retain_interval", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "31536000", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.962 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.963 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_recent_interval": { 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_recent_interval", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1209600", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.964 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dashboard", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.965 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.966 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.967 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.968 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.969 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.970 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.971 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.972 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.973 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.974 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.975 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.976 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.979 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_URL": { 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.981 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:04:01.982 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:04:01.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:04:01.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:04:01.985 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:04:01.986 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "[]", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:04:01.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "{}", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.988 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.990 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.992 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.993 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.994 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.995 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.996 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.997 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi029.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "45", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.002 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.004 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.005 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.007 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.008 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "cross_origin_url": { 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cross_origin_url", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "crt_file": { 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crt_file", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.010 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "debug": { 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "debug", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.011 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "jwt_token_ttl": { 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "jwt_token_ttl", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "28800", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.012 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.014 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.015 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.016 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.017 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "motd": { 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "motd", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.018 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "The message of the day", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.019 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.020 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8080", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.021 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:02.022 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl_server_port": { 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl_server_port", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.023 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8443", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "sso_oauth2": { 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sso_oauth2", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "redirect", 2024-09-21T02:04:02.025 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect" 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "url_prefix": { 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url_prefix", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.027 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.028 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_monitoring": { 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_monitoring", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "monitor device health metrics", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.029 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.030 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.033 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.034 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mark_out_threshold": { 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mark_out_threshold", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2419200", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:04:02.035 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": { 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_name", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "device_health_metrics", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:04:02.036 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "retention_period": { 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retention_period", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15552000", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:04:02.037 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_frequency": { 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_frequency", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:04:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "self_heal": { 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "self_heal", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:04:02.039 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_threshold": { 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_threshold", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7257600", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.042 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "diskprediction_local", 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.043 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.044 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.045 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.046 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.047 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "predict_interval": { 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predict_interval", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-09-21T02:04:02.048 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "predictor_model": { 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predictor_model", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "prophetstor", 2024-09-21T02:04:02.049 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-09-21T02:04:02.050 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.051 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "influx", 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": false, 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "batch_size": { 2024-09-21T02:04:02.052 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "batch_size", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5000", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "database": { 2024-09-21T02:04:02.053 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "database", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "ceph", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": { 2024-09-21T02:04:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hostname", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:02.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "30", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "5", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.057 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.059 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.060 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "password": { 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "password", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.061 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "port": { 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "port", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8086", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.062 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server port", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.063 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.064 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "false", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "threads": { 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threads", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.065 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "32", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "username": { 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "username", 2024-09-21T02:04:02.066 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.067 INFO:teuthology.orchestra.run.smithi029.stdout: "verify_ssl": { 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "verify_ssl", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "true", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "insights", 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.069 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.071 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.072 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.074 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "iostat", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.076 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.077 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.078 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.079 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.080 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.081 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "k8sevents", 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.082 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_event_retention_days": { 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "config_check_secs": { 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_check_secs", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "10", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.087 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.088 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "localpool", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "failure_domain": { 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "failure_domain", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "host", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:04:02.091 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.092 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.093 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.094 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.095 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": { 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_size", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.096 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "num_rep": { 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "num_rep", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.097 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": { 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_num", 2024-09-21T02:04:02.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "128", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "prefix": { 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prefix", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.100 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "subtree": { 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subtree", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "rack", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.101 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mds_autoscaler", 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.102 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.103 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.104 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.105 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.106 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.107 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirroring", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.108 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.109 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.111 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.113 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "nfs", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.115 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.116 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.117 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.118 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.119 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.120 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "fail_fs": { 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fail_fs", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.122 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.125 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.126 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator": { 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-09-21T02:04:02.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rook", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "test_orchestrator" 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Orchestrator backend", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.128 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_perf_query", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.129 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.130 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.131 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.132 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.133 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.134 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_support", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.136 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.137 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.139 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.140 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_autoscaler", 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.142 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.143 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.144 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.146 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.148 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "threshold": { 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threshold", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1.0", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.149 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "scaling threshold", 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "progress", 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_pg_recovery_event": { 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:04:02.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.152 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.154 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.155 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.156 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "max_completed_events": { 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_completed_events", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "50", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.157 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.158 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.159 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "cache": { 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cache", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.160 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "exclude_perf_counters": { 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "exclude_perf_counters", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.161 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.162 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.163 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.165 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.166 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools": { 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.167 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "300", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.168 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_interval": { 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_interval", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15.0", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.169 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:04:02.170 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "9283", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.172 INFO:teuthology.orchestra.run.smithi029.stdout: "stale_cache_strategy": { 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stale_cache_strategy", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "log", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.173 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "default", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "default", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "error" 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.176 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.179 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.180 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_snap_create": { 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.181 INFO:teuthology.orchestra.run.smithi029.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.182 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "trash_purge_schedule": { 2024-09-21T02:04:02.183 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "trash_purge_schedule", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.184 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "restful", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_auth": { 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_auth", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.186 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.187 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.188 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.189 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "max_requests": { 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_requests", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-09-21T02:04:02.191 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.192 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.194 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rgw", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.196 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.199 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.200 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:04:02.201 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rook", 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.203 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.204 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.205 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.206 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.207 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.208 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "storage_class": { 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "storage_class", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "local", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "selftest", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.212 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "roption1": { 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption1", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "roption2": { 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption2", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "xyz", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption1": { 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption1", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption2": { 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption2", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption3": { 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption3", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.220 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption4": { 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption4", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.221 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption5": { 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption5", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.222 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption6": { 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption6", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.223 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption7": { 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption7", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-09-21T02:04:02.224 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "42", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.225 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "testkey": { 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testkey", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.226 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "testlkey": { 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testlkey", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.227 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "testnewline": { 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testnewline", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.228 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smb", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.229 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "internal_store_backend": { 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "internal_store_backend", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.231 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.232 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.233 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.236 INFO:teuthology.orchestra.run.smithi029.stdout: "update_orchestration": { 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "update_orchestration", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snap_schedule", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_m_granularity": { 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_m_granularity", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.238 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "dump_on_update": { 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dump_on_update", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:04:02.239 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.240 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.241 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.242 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.243 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.244 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stats", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.246 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.248 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.249 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.251 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "status", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.253 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.254 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.256 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.257 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.258 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telegraf", 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "address": { 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "address", 2024-09-21T02:04:02.259 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:02.260 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.261 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.262 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.263 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.264 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.265 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.266 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.267 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telemetry", 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_basic": { 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_basic", 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.268 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_crash": { 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_crash", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.269 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_device": { 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_device", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_ident": { 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_ident", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.271 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_perf": { 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_perf", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.272 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "contact": { 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "contact", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.273 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "description": { 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "description", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.274 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "device_url": { 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_url", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.275 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.276 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.277 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "24", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "8", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "last_opt_revision": { 2024-09-21T02:04:02.278 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "last_opt_revision", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard": { 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard_description": { 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard_description", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.282 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.283 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.284 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.285 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.286 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "organization": { 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "organization", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "proxy": { 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "proxy", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.288 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "url": { 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.289 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "test_orchestrator", 2024-09-21T02:04:02.290 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.291 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.292 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.293 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.296 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.297 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.298 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.299 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.300 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_clones": { 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_clones", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.301 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "periodic_async_work": { 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "periodic_async_work", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.302 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Periodically check for async work", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_delay": { 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.304 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.305 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix", 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "discovery_interval": { 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "discovery_interval", 2024-09-21T02:04:02.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "100", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "identifier": { 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "identifier", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.307 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-09-21T02:04:02.308 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.309 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.310 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.311 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.312 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.313 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "sqlite3_killpoint": { 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "dev", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.314 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_host": { 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_host", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_port": { 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_port", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10051", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_sender": { 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_sender", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "services": { 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "dashboard": "https://172.21.15.29:8443/" 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "always_on_modules": { 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "octopus": [ 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-09-21T02:04:02.318 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "pacific": [ 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-09-21T02:04:02.319 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "quincy": [ 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-09-21T02:04:02.320 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "reef": [ 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "squid": [ 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-09-21T02:04:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "last_failure_osd_epoch": 3, 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "active_clients": [ 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-09-21T02:04:02.322 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:0", 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1262382996 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "libcephsqlite", 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:0", 2024-09-21T02:04:02.323 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 516499598 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:0", 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1821711658 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.324 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:0", 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3738586778 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-21T02:04:02.325 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:04:02.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 msgr2=0x7f42f80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 0x7f42f80b8a20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4300031440 tx=0x7f4300005d50 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 msgr2=0x7f431019b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f430800ccb0 tx=0x7f430800a620 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 shutdown_connections 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f42f80b6510 0x7f42f80b8a20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4310103fc0 0x7f431019b140 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4310103620 0x7f431019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 --2- 172.21.15.29:0/1778405000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43101025a0 0x7f4310199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 >> 172.21.15.29:0/1778405000 conn(0x7f43100fdee0 msgr2=0x7f431006b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 shutdown_connections 2024-09-21T02:04:02.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:00.924+0000 7f4316139640 1 -- 172.21.15.29:0/1778405000 wait complete. 2024-09-21T02:04:02.374 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T02:04:02.374 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T02:04:02.374 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2451914505' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.375 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2880897660' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.375 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:00 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1778405000' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:01 smithi029 ceph-mon[31228]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2451914505' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2880897660' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:00 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1778405000' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:01 smithi066 ceph-mon[35001]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-21T02:04:02.376 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2451914505' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.377 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2880897660' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:02.377 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:00 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1778405000' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:04:02.377 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:01 smithi190 ceph-mon[34368]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-21T02:04:02.646 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 2310, 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 2312, 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 2678070, 2024-09-21T02:04:03.192 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 734581, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 6, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.301631052, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.112083240, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.073470396, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.097614259, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 2310, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.165392055, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000071598 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:03.193 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 934, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 1153, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 230244, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 864828, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 7, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 4, 2024-09-21T02:04:03.194 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.135369792, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.063760525, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.022548442, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.037792860, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1151, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.078616712, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000068302 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.195 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 568384 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 330, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 300, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 738092, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 180910, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:03.196 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.050290268, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.024572890, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.031981416, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000038265, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 300, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.032200834, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000107336 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.197 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 737824, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 180592 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 5427, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.189715342, 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000034957 2024-09-21T02:04:03.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 0, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 0, 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2024-09-21T02:04:03.199 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 0, 2024-09-21T02:04:03.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 94, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 20, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 74, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 66, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 6, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 0, 2024-09-21T02:04:03.201 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:03.202 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:04:03.203 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 0 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 0, 2024-09-21T02:04:03.204 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2024-09-21T02:04:03.205 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_write_failed": 0 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "evlrg": 0, 2024-09-21T02:04:03.206 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 1140, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 3, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segmjr": 2, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 4194304, 2024-09-21T02:04:03.207 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 6192779, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 6191641, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 1140 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 52, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 642, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 590, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 32, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 57, 2024-09-21T02:04:03.208 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 25, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 94, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 681, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 587, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 0, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 0, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 120972, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 321794048 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 0, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:03.209 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 0, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 0, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 0, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.210 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.211 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2024-09-21T02:04:03.212 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.214 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.215 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.216 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.217 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.219 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2024-09-21T02:04:03.220 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.222 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 1, 2024-09-21T02:04:03.223 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 2073, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:03.224 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:03.225 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:03.226 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 191758, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 95, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:03.227 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 154, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 228858, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 1682, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2024-09-21T02:04:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 3243, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 21, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 3, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 3243, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3243, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 11.628823775, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003585822 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3243, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 3244 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.229 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 3243, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 3243, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 1979, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 1262, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2024-09-21T02:04:03.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 1, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:03.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2024-09-21T02:04:03.232 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 76, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 2, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 5, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 7, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 2, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 2, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 0 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 3570, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 3305482, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 3570, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3570, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 3305482, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:03.236 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 3243, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 1998480, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3243, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 1998480, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 3243, 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 3243, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3243, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 3243, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x55711b0bd380": { 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "trackedop": { 2024-09-21T02:04:03.241 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 0 2024-09-21T02:04:03.242 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:03.242 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:04:03.976 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T02:04:03.976 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T02:04:03.976 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-21T02:04:03.976 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-21T02:04:03.976 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T02:04:03.976 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T02:04:04.094 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:03 smithi029 ceph-mon[31228]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-09-21T02:04:04.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:03 smithi190 ceph-mon[34368]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-09-21T02:04:04.235 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:04.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:03 smithi066 ceph-mon[35001]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 0, 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 2, 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 0, 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 0, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 0, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.001294496, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.000252675, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 2, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 226, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 320, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 200608, 2024-09-21T02:04:04.776 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.022621809, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.018057477, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.002001606, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 224, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.013973426, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000062381 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 200608 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 303, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 298, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 554432, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 182768, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 2, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 1, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.055991488, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.026007585, 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.034068831, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 298, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.020668601, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000069357 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 554432, 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 182768 2024-09-21T02:04:04.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:04.780 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:04.781 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:04.782 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 148721, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 30, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 968, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 2, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 301, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 509197, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 301, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 301, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 509197, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:04.785 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:04:05.618 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T02:04:05.618 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T02:04:05.618 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T02:04:05.618 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T02:04:05.618 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T02:04:05.618 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T02:04:05.877 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:06.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:05 smithi190 ceph-mon[34368]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T02:04:06.237 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:05 smithi029 ceph-mon[31228]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T02:04:06.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:05 smithi066 ceph-mon[35001]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1054, 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 1056, 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 354089, 2024-09-21T02:04:06.366 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 335173, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 7, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.157665974, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.070948734, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.032952097, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.041566901, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1054, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.095972066, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000091055 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:06.367 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 2485, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 2455, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 2226758, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 866901, 2024-09-21T02:04:06.368 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.293678249, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.106506502, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.085977296, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.094024847, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 2455, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.148258148, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000060390 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 738368, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 181872 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 5, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 226, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 1668, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 571365, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 7, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 4, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.021746442, 2024-09-21T02:04:06.370 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.016804551, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.002661791, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000063123, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 224, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.013684875, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000061093 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:06.371 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 570400 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 5367, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.188971242, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000035209 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-09-21T02:04:06.372 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 0, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 0, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.373 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 340, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 82, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 258, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 223, 2024-09-21T02:04:06.374 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 12, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2024-09-21T02:04:06.375 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 0, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 0 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2024-09-21T02:04:06.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_started": 0, 2024-09-21T02:04:06.379 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "uninline_write_failed": 0 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evlrg": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 647, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2024-09-21T02:04:06.380 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 2, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "segmjr": 2, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 4194304, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 5135898, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 5135324, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 647 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2024-09-21T02:04:06.381 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 339, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 340, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 1, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 20, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 20, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 0, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 340, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 340, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 0, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 0, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 0, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 119816, 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 318586880 2024-09-21T02:04:06.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.383 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.384 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.385 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.386 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2024-09-21T02:04:06.387 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.389 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:06.390 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.391 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.393 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 2, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 1, 2024-09-21T02:04:06.394 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 2075, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:06.395 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:06.396 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:06.397 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 210494, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 387, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 154, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:06.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 1041044, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 8491, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 3211, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 21, 2024-09-21T02:04:06.399 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 3, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 3211, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3211, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 12.338813793, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003842670 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3211, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 3212 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 3211, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 3211, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 0, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2024-09-21T02:04:06.400 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 2005, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 1204, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:06.401 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 1, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2024-09-21T02:04:06.402 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 76, 2024-09-21T02:04:06.403 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 2, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 6, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 8, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 2, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 2, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 0 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2024-09-21T02:04:06.404 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:06.405 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 3540, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 2244187, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 3540, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3540, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 2244187, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.406 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 3211, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 941599, 2024-09-21T02:04:06.407 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3211, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 941599, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:06.408 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 3211, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 3211, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 3211, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 3211, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:06.409 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x564c91bb1380": { 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:06.411 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout: "trackedop": { 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 0 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:06.412 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:04:07.102 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T02:04:07.102 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T02:04:07.102 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T02:04:07.102 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-21T02:04:07.102 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T02:04:07.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T02:04:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:07.914 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 305, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 300, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 554976, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 184048, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 0, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.056360114, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.025696669, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.034458214, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:07.915 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 300, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.021130236, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000070434 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 554976, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 184048 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 2, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 227, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 320, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 200784, 2024-09-21T02:04:07.916 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.022954897, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.018610194, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.001657765, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 225, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.018966104, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000084293 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.917 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 200784 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 0, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 2, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 0, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 0, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 2, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 1, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.000302681, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.000224061, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:07.918 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 0, 2024-09-21T02:04:07.919 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:07.920 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:07.921 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 148704, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 29, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-09-21T02:04:07.922 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 968, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 2, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:07.923 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 303, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 509457, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 303, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 303, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 509457, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-09-21T02:04:07.924 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-09-21T02:04:07.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:07.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-21T02:04:07.925 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-21T02:04:08.095 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:07 smithi029 ceph-mon[31228]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-21T02:04:08.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:07 smithi190 ceph-mon[34368]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-21T02:04:08.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:07 smithi066 ceph-mon[35001]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-21T02:04:08.650 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T02:04:08.650 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T02:04:08.650 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T02:04:08.650 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T02:04:08.650 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T02:04:08.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T02:04:08.926 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 12953, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 15928, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 4869543, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 16235544, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 15, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 1.358917645, 2024-09-21T02:04:09.456 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.674494659, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.748726260, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.153124203, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 15930, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 3.284623760, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000206191 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 18652, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.003964198, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000212 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2308416, 2024-09-21T02:04:09.457 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 4508384 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6076, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 6920, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3604039, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 6102538, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 10, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.620305400, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.301946198, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.249815565, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.077441963, 2024-09-21T02:04:09.458 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6919, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 4.137300227, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000597962 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 5799, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001419222, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000244 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 781456, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 208192 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6305, 2024-09-21T02:04:09.459 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 8859, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2534885, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 8315870, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 290, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.783778291, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.384985465, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.372764277, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.047116947, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8854, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.901110799, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000327661 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.460 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 10608, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002201097, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000207 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 488560 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.461 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4986, 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 5.913396831, 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001186000 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 173, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013464983, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000077832 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 2047, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 1851, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1851, 2024-09-21T02:04:09.463 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 4.027960144, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002176099 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 196, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 6, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 1, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 691, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 3, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 613, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 10, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 592, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 11, 2024-09-21T02:04:09.464 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 598, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 1418, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 579, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 579, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 17, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 5996, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 5224, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 187, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 15, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:09.465 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 58, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 10203, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 43, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 1377, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 24, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 924, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 3817, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 1027, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 1746173113, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 3817, 2024-09-21T02:04:09.466 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 454, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 18, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 436, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 3363, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 12, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 1458, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 1567, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 2647, 2024-09-21T02:04:09.467 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 1061, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 30, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 263 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 191, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 194, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 101, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 214, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 713, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-09-21T02:04:09.468 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 70, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 41, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 29, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:09.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 18, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 43, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 3, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:09.470 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_write_failed": 0 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 3802, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 3778, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 3778, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 24, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 2, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 74, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 73, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 73, 2024-09-21T02:04:09.471 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 1, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 1, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 1, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 14749528, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 15148516, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2125, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 51.789774116, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.024371658 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 614, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 2020, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 1406, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 57, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 188, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 131, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 613, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 2380, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 1767, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 579, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 1676, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 1097, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 138064, 2024-09-21T02:04:09.473 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 366891008 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 2254, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 67, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 2047, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 264, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 124, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1036, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.693914271, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002600303 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-09-21T02:04:09.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 70, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.093189095, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001331272 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 29, 2024-09-21T02:04:09.475 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.048350505, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001667258 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 95, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.171560113, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001805895 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.478 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.478 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 37, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.395149275, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.010679710 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 407, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.088307311, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000216971 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 57, 2024-09-21T02:04:09.481 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.113096732, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001984153 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 54, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.257310071, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004765001 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-09-21T02:04:09.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.015731649, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.007865824 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000925686, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000925686 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 9, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.024658271, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002739807 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.486 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 54, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.125767165, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002329021 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 2, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 1, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 21, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 21, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 951, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.488 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:09.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:09.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:09.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 345675, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 706, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 264, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 3, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:09.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 154, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 2118296, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 18502, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-09-21T02:04:09.493 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 8767, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 13346075, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 31, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 8767, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8767, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 172.278823567, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.019650829 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-09-21T02:04:09.494 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8767, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 12403 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 8767, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 3878, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 4889, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 620, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 1091, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 36, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 2143, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 178, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-09-21T02:04:09.495 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 3, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 572, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 3842, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 2182, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-09-21T02:04:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-09-21T02:04:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 76, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 1636, 2024-09-21T02:04:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 14, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 86 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 13, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 6, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 56, 2024-09-21T02:04:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 41, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 25330, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 8405355, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 25330, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 25330, 2024-09-21T02:04:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 8405355, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 8767, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 13350116, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 8767, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 13350116, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 8767, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 8767, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 8767, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 8767, 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:09.503 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 41, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 4141, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 41, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:09.504 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 18, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4141, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5644c26cb450": { 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 3802, 2024-09-21T02:04:09.505 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 10954212, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 3802, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 2125, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 10954212, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:09.506 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: "trackedop": { 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:09.507 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-21T02:04:10.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:09 smithi066 ceph-mon[35001]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:04:10.050 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:09 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:09 smithi190 ceph-mon[34368]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:04:10.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:09 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:09 smithi029 ceph-mon[31228]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:04:10.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:09 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:10.443 INFO:tasks.check_counter:Daemon mds.b mds.exported=43 2024-09-21T02:04:10.443 INFO:tasks.check_counter:Daemon mds.b mds.imported=24 2024-09-21T02:04:10.443 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=191 2024-09-21T02:04:10.443 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=194 2024-09-21T02:04:10.443 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T02:04:10.444 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T02:04:10.706 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6666, 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 7231, 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2795402, 2024-09-21T02:04:11.261 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 4483316, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 16, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.698207632, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.359296958, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.291812347, 2024-09-21T02:04:11.262 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.106642491, 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7226, 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.737722265, 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000378871 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:11.263 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6631, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001512046, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000228 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1261552, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 2472512 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 5313, 2024-09-21T02:04:11.264 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4339, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2719825, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 2070437, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.492065101, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.249382161, 2024-09-21T02:04:11.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.209554471, 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.044505784, 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4341, 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.472890501, 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000108935 2024-09-21T02:04:11.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 5673, 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001294577, 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000228 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:04:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 483072 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2774, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2506, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2117527, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1517166, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 46, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.277939842, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.117568816, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.116403739, 2024-09-21T02:04:11.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.041401245, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2505, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.476916470, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000190385 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1001, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000209519, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000209 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 768688, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 203568 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-09-21T02:04:11.269 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2561, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 4.587440734, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001791269 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:11.270 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 15, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000316069, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000021071 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 2705, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 2698, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2698, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 107.566000877, 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.039868792 2024-09-21T02:04:11.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 7, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 11, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 360, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 7, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 353, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 347, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 4, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 328, 2024-09-21T02:04:11.272 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 328, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 16, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 9972, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 9939, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 6, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 26, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 5664, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 10, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 328, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:11.273 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 5387, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 1028, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 1741012260, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 5387, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 5387, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:11.274 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 887, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 411, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 1058, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 111, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 6, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 11 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 99, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 398, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 211, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 345, 2024-09-21T02:04:11.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 117, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 1, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 1, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:11.276 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_write_failed": 0 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 647, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-09-21T02:04:11.277 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 647, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 2, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 5135898, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-09-21T02:04:11.278 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 492, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 8.007546226, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.016275500 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 363, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 366, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 3, 2024-09-21T02:04:11.279 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 54, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 208, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 154, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 360, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 367, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 7, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 328, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 534, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 206, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 125860, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 333332480 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-09-21T02:04:11.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 2760, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 2705, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 17, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.281 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2631, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 107.494274428, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.040856812 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 44, 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.026074535, 2024-09-21T02:04:11.283 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000592603 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.285 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-09-21T02:04:11.286 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-09-21T02:04:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 12, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.009652914, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000804409 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 10, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.008851104, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000885110 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.288 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.290 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.291 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.027147896, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.027147896 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-09-21T02:04:11.293 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 2, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 1, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 233, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 233, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 2333, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-09-21T02:04:11.294 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:11.295 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:11.296 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:11.297 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:11.298 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 294474, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 454, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:11.299 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 154, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 1291448, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 13400, 2024-09-21T02:04:11.300 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 6027, 2024-09-21T02:04:11.301 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 1011822, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 29, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 6027, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6027, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 20.099875791, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003334971 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-09-21T02:04:11.302 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6027, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 6103 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 6027, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 5388, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 639, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-09-21T02:04:11.303 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 0, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 10, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 493, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 58, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-09-21T02:04:11.304 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 13, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 5387, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 20, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:11.305 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-09-21T02:04:11.306 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-09-21T02:04:11.307 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 76, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-09-21T02:04:11.308 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 97, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 24 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-09-21T02:04:11.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 1, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 1, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 1, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 1, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:11.310 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 14749, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 6109564, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 14749, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 14749, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 6109564, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 6027, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 1011923, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 6027, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1011923, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 6027, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 6027, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 6027, 2024-09-21T02:04:11.313 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 6027, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 101, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:11.314 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 101, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5575328692b0": { 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:11.315 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 647, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 941594, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 647, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 492, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 941594, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "trackedop": { 2024-09-21T02:04:11.316 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-09-21T02:04:11.317 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:11.317 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-21T02:04:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:11 smithi066 ceph-mon[35001]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-09-21T02:04:12.049 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:11 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:11 smithi190 ceph-mon[34368]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-09-21T02:04:12.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:11 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:12.298 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T02:04:12.298 INFO:tasks.check_counter:Daemon mds.e mds.imported=10 2024-09-21T02:04:12.298 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=99 2024-09-21T02:04:12.299 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=398 2024-09-21T02:04:12.299 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T02:04:12.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T02:04:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:11 smithi029 ceph-mon[31228]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-09-21T02:04:12.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:11 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:12.559 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:04:13.112 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-21T02:04:13.112 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:13.112 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6996, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 5972, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3262942, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 5919311, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.634604348, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.316881672, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.314269475, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.047964396, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 5973, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.426976683, 2024-09-21T02:04:13.113 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000071484 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8356, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001653034, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000197 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 783024, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 204000 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1172, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1378, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 652033, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1216582, 2024-09-21T02:04:13.114 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 18, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.139146279, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.061450731, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.049135626, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.020321154, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1377, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.271864367, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000197432 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 747, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000163002, 2024-09-21T02:04:13.115 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000218 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 575632 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 5632, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 6905, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2017375, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 3935657, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 44, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.560567870, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.296527300, 2024-09-21T02:04:13.116 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.287365707, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.027653026, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6901, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.597589852, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000086594 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8577, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001532822, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000178 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1611280, 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 3133280 2024-09-21T02:04:13.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3098, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.934704077, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000947289 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.118 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000033359, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000016679 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 1924, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 1887, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1887, 2024-09-21T02:04:13.119 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 5.398595945, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002860941 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 37, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 11, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 509, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 7, 2024-09-21T02:04:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 492, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 10, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 495, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 191, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 114, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 114, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 9, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 6696, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 5589, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 37, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 26, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:13.121 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 400, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 9334, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 5, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 186, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 6, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 224, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 1650, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 1028, 2024-09-21T02:04:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 1741012260, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 1650, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 20, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 20, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 1630, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 1415, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 1373, 2024-09-21T02:04:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 2162, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 424, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 48, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 998 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 267, 2024-09-21T02:04:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 471, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 248, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 419, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 276, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 239, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 239, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-09-21T02:04:13.126 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 5, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_write_failed": 0 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-09-21T02:04:13.127 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 1988, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 1988, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 3, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 3, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-09-21T02:04:13.128 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 7699687, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1478, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 138.711621568, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.093850894 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 103, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 291, 2024-09-21T02:04:13.129 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 188, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 47, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 247, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 200, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 463, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 661, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 198, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 68, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 357, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 289, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 128248, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 332279808 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-09-21T02:04:13.130 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 3165, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 1924, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 483, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 32, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 35, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.041256240, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001178749 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 255, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.138837009, 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.008387596 2024-09-21T02:04:13.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 239, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.430908281, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001802963 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 800, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.846416892, 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001058021 2024-09-21T02:04:13.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.133 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:13.134 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.135 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 73, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.088625230, 2024-09-21T02:04:13.136 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001214044 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.014048304, 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002006900 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.137 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 239, 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1.492445016, 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006244539 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.138 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.139 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.140 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.142 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:13.143 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 239, 2024-09-21T02:04:13.146 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.346058973, 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001447945 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:13.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-09-21T02:04:13.148 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 282, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 282, 2024-09-21T02:04:13.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 936, 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:13.150 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:13.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:13.154 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:13.155 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 316940, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 237, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 154, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 856942, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 22996, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 3441, 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 3581010, 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 28, 2024-09-21T02:04:13.160 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 3441, 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3441, 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 156.138941331, 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.045376036 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-09-21T02:04:13.161 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3441, 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 3703 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 3441, 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 1650, 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 1791, 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-09-21T02:04:13.163 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-09-21T02:04:13.163 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 0, 2024-09-21T02:04:13.163 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 10, 2024-09-21T02:04:13.163 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 0, 2024-09-21T02:04:13.163 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 1478, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 61, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 6, 2024-09-21T02:04:13.164 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1650, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 20, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:13.165 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-09-21T02:04:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-09-21T02:04:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 76, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 414, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 64 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.168 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-09-21T02:04:13.169 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 13796, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 4445929, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 13796, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 13796, 2024-09-21T02:04:13.170 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4445929, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3441, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 3581010, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3441, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3581010, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:13.172 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3441, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 3441, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3441, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3441, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:13.173 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.174 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x558e18547520": { 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1988, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 3505383, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1988, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1478, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3505383, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:13.175 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout: "trackedop": { 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:13.176 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-21T02:04:13.780 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-09-21T02:04:13.780 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-09-21T02:04:13.780 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=267 2024-09-21T02:04:13.780 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=471 2024-09-21T02:04:13.780 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T02:04:13.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T02:04:14.040 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:04:14.271 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:13 smithi066 ceph-mon[35001]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T02:04:14.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:13 smithi029 ceph-mon[31228]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T02:04:14.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:13 smithi190 ceph-mon[34368]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T02:04:14.571 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-21T02:04:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1397, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1366, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1113728, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 522826, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.181225096, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.085159113, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.060326189, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.033680407, 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1365, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.088875288, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000065110 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 757904, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 186064 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:14.573 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 8, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 233, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3868, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 575882, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.017982427, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.012014075, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.004050307, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000073350, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 232, 2024-09-21T02:04:14.574 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.018456095, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000079552 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 575184 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2264, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2266, 2024-09-21T02:04:14.575 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 4079916, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 719959, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.172022919, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.062482941, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.042210984, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.058016965, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2264, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.119533197, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000052797 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:14.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:14.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 5619, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.139516723, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000024829 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-09-21T02:04:14.578 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 444, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 4, 2024-09-21T02:04:14.579 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 440, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 437, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 3, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-09-21T02:04:14.580 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-09-21T02:04:14.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:14.583 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-09-21T02:04:14.584 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_write_failed": 0 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 1988, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-09-21T02:04:14.585 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 3, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 7699687, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 7691988, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 1988 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.586 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 77, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 268, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 191, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 12, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 20, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 8, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 444, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 637, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 193, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 119876, 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 320741376 2024-09-21T02:04:14.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.589 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.590 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-09-21T02:04:14.591 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.592 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.593 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.596 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-09-21T02:04:14.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 2339, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-09-21T02:04:14.600 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:14.601 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:14.602 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 184603, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 116, 2024-09-21T02:04:14.603 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 154, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:14.604 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 512137, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 16604, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 3325, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2024-09-21T02:04:14.605 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 3325, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3325, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 14.072911272, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004232454 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3325, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 3326 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 3325, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 3325, 2024-09-21T02:04:14.606 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 1988, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1335, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-09-21T02:04:14.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:14.608 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-09-21T02:04:14.609 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 76, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 4, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 5, 2024-09-21T02:04:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 1, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:14.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 3665, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 4847387, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 3665, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3665, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4847387, 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:14.613 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:14.614 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3325, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 3505388, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3325, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3505388, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-09-21T02:04:14.615 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3325, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 3325, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3325, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3325, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.616 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:14.617 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x56196fe075f0": { 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-09-21T02:04:14.618 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "trackedop": { 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-21T02:04:14.619 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-21T02:04:15.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:14 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:15.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:14 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:15.397 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T02:04:15.397 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T02:04:15.397 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-21T02:04:15.397 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-21T02:04:15.398 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T02:04:15.398 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T02:04:15.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:14 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:15.657 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:04:16.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:16 smithi190 ceph-mon[34368]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3678, 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 4168, 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1866558, 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 3873976, 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 14, 2024-09-21T02:04:16.228 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.353869260, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.160646158, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.096823436, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.074460965, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4166, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.390407352, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000093712 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2247, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000391780, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000174 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.229 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3413, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 3592, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2583744, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1463275, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.309733740, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.135601840, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.111668993, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.059791017, 2024-09-21T02:04:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3592, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.364113710, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000101367 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1058, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000193173, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000182 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 791312, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 204208 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:16.231 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 8808, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 9285, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4173640, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 4928285, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 44, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.718785777, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.366349560, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.379364220, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.044542180, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9283, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.837098686, 2024-09-21T02:04:16.232 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000090175 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12036, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002299935, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000191 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1239840, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 2751520 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3528, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.534775941, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001285367 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 73, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002802701, 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000038393 2024-09-21T02:04:16.234 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 1548, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 1398, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1398, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 36.551471948, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.026145545 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 150, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 1, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:16.235 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 231, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 221, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 10, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 226, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 604, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 189, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 189, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 7, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 8452, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 8239, 2024-09-21T02:04:16.236 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 145, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 22, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 2, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 1766, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 2, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 587, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 13, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 795, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 6055, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 1028, 2024-09-21T02:04:16.237 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 1741012260, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 6055, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 130, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 130, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 5925, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 4, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 1448, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 905, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:16.238 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 1838, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 234, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 16, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 63 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 94, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 309, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 165, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 285, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 292, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-21T02:04:16.239 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 17, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 17, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 2, 2024-09-21T02:04:16.240 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 1155, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 1155, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-21T02:04:16.241 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 3, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 3, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 6303889, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 716, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 13.739371621, 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.019189066 2024-09-21T02:04:16.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 214, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 819, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 605, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 46, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 195, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 149, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 231, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 861, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 630, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 189, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 464, 2024-09-21T02:04:16.243 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 275, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 127604, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 335429632 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 1619, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 15, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 1548, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 95, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 33, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.031474207, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002098280 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.244 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1180, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 36.259552817, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.030728434 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.245 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 146, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.133058120, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000911356 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000115503, 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000115503 2024-09-21T02:04:16.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:16.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-21T02:04:16.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 27, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.060802618, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002251948 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.006967804, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000774200 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.010449802, 2024-09-21T02:04:16.250 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002089960 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.251 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.252 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-21T02:04:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.049051077, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003270071 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.255 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 128, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 128, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 953, 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:16.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:16.257 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:16.258 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 314989, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 319, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 616, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 7, 2024-09-21T02:04:16.259 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 154, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 789562, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 7264, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:16.260 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 7003, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 2193123, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 7003, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7003, 2024-09-21T02:04:16.261 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 33.644061819, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004804235 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7003, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 7128 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 7003, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 6066, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 937, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-09-21T02:04:16.262 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 8, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 724, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 67, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 29, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 6056, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-09-21T02:04:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-21T02:04:16.264 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 76, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-09-21T02:04:16.265 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 169, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 4, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 41 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-21T02:04:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 17, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 17, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 15895, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 7004132, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 15895, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 15895, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 7004132, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:16.268 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 7003, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2194761, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 7003, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2194761, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:16.269 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 7003, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 7003, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 7003, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 7003, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 17, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1717, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 17, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 8, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1717, 2024-09-21T02:04:16.271 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x556ef0857450": { 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1155, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 2109585, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1155, 2024-09-21T02:04:16.272 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 716, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2109585, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:16.273 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-21T02:04:16.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:15 smithi066 ceph-mon[35001]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T02:04:16.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:15 smithi029 ceph-mon[31228]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T02:04:17.034 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-09-21T02:04:17.034 INFO:tasks.check_counter:Daemon mds.c mds.imported=13 2024-09-21T02:04:17.035 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=94 2024-09-21T02:04:17.035 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=309 2024-09-21T02:04:17.035 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T02:04:17.035 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T02:04:17.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:17 smithi190 ceph-mon[34368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:17.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:17 smithi066 ceph-mon[35001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:17.331 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:04:17.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:17 smithi029 ceph-mon[31228]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1214, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1216, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4590843, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 386062, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 6, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.148526947, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.060508576, 2024-09-21T02:04:17.890 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.036907230, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.039871920, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1214, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.068670794, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000056565 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:17.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 2071, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2037, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3812294, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 735438, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.200003223, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.080680006, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.072336895, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.045117515, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2037, 2024-09-21T02:04:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.129375273, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000063512 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 766960, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 185232 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 558, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 783, 2024-09-21T02:04:17.893 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4337102, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 751193, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.068802063, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.035689923, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.013883877, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.013738254, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 781, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.033631116, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000043061 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.894 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 574384 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:17.895 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5873, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.170361778, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000029007 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-21T02:04:17.896 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 420, 2024-09-21T02:04:17.897 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 200, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 220, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 134, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 19, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:17.898 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 1027, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 1746173113, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:17.899 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:04:17.900 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:17.901 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:17.902 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 3808, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-21T02:04:17.903 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 71, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 14749528, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 15191978, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 15186092, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 3808 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.904 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 349, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 1983, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 1634, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 41, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 102, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 61, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 420, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 2074, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 1654, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 123980, 2024-09-21T02:04:17.905 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 324243456 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.906 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.907 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.909 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.910 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-21T02:04:17.910 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.910 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.911 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.912 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:17.913 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.914 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.915 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.917 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-09-21T02:04:17.918 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2393, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:17.920 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:17.921 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 202502, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 386, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-21T02:04:17.922 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 154, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 1144800, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 8754, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:17.923 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3500, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 26, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 4, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3500, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3500, 2024-09-21T02:04:17.924 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 18.740473184, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005354420 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3500, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3501 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3500, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 3500, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 2107, 2024-09-21T02:04:17.925 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1390, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:17.926 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 2, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-21T02:04:17.927 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-21T02:04:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 76, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 5, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 5, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-21T02:04:17.929 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-21T02:04:17.930 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3839, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 12374515, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3839, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3839, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 12374515, 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:17.931 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3500, 2024-09-21T02:04:17.932 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 10998467, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3500, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 10998467, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:17.933 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3500, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3500, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3500, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3500, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:17.934 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.935 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x561cf5f491e0": { 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-21T02:04:17.936 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:17.937 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-21T02:04:18.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:18 smithi190 ceph-mon[34368]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-21T02:04:18.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:18 smithi066 ceph-mon[35001]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-21T02:04:18.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:18 smithi029 ceph-mon[31228]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-21T02:04:18.785 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T02:04:18.786 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T02:04:18.786 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T02:04:18.786 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-21T02:04:18.786 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T02:04:18.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T02:04:19.032 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:04:19.538 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 4005, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 5336, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2082067, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 5648019, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 17, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.420867456, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.219360799, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.220132490, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.028352613, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5333, 2024-09-21T02:04:19.539 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.528522577, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000099104 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5456, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000930563, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000170 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1756352, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 2370592 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1082, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1305, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 597250, 2024-09-21T02:04:19.540 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1137973, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.118846381, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.058047653, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.032473249, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.020816353, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1301, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.118864385, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000091363 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 365, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000087700, 2024-09-21T02:04:19.541 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000240 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 576960 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 5158, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 3835, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2488145, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 2209515, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 42, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.364266733, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.160217814, 2024-09-21T02:04:19.542 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.221700021, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.021464535, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3838, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.205821347, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000053627 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7353, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001335767, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000181 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.543 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1463, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.554384534, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001062463 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:19.544 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 804, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.031228974, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000038842 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 1398, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 1309, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1309, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 31.666596867, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.024191441 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-21T02:04:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 89, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 177, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 2, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 153, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 22, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 175, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 273, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 152, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 152, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 10, 2024-09-21T02:04:19.546 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 3563, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 3257, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 89, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 38, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 78, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 9, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 20, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 250, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 23, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 382, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:19.547 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 1173, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 1028, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 1741012260, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 1173, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 16, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 16, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 1157, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 622, 2024-09-21T02:04:19.548 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 839, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 1088, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 264, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 143, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 924 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 146, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 327, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 177, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 285, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 149, 2024-09-21T02:04:19.549 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 230, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 230, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 20, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 1668, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 1668, 2024-09-21T02:04:19.551 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 3, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 3, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 7097023, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 535, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 53.391463014, 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.099797127 2024-09-21T02:04:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 174, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 657, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 483, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 36, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 185, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 149, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 174, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 677, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 503, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 152, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 640, 2024-09-21T02:04:19.553 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 488, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 125412, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 333336576 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 1610, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 1398, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 22, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 44, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 230, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.282795060, 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001229543 2024-09-21T02:04:19.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 487, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 20.192471611, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.041462980 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 208, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 10.684483891, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.051367711 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000169499, 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000169499 2024-09-21T02:04:19.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:19.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.558 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.559 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 152, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.065308904, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000429663 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000936857, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000936857 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 217, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.422837603, 2024-09-21T02:04:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001948560 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-21T02:04:19.562 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.564 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 13, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.017593442, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001353341 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.565 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 81, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 81, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 953, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:19.566 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:19.567 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:19.568 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 306675, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 284, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 880, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 10, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:19.569 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 154, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 624062, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 5366, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2430, 2024-09-21T02:04:19.570 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 3001440, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2430, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2430, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 81.632658612, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.033593686 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2430, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2591 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2430, 2024-09-21T02:04:19.571 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 1269, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 1161, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 96, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 632, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 134, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 242, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:19.572 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1173, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-21T02:04:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-21T02:04:19.574 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 76, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 208, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 76 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-21T02:04:19.575 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 230, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 230, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:19.576 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 10241, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 4094358, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 10241, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 10241, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4094358, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:19.577 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2430, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3024670, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2430, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3024670, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2430, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2430, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2430, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2430, 2024-09-21T02:04:19.579 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 230, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 23230, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 230, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 97, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 23230, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x5562b9c87380": { 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1668, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 2902719, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1668, 2024-09-21T02:04:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 535, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2902719, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:19.582 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-21T02:04:20.266 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:20 smithi190 ceph-mon[34368]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-21T02:04:20.267 INFO:tasks.check_counter:Daemon mds.i mds.exported=20 2024-09-21T02:04:20.268 INFO:tasks.check_counter:Daemon mds.i mds.imported=23 2024-09-21T02:04:20.268 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=146 2024-09-21T02:04:20.268 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=327 2024-09-21T02:04:20.268 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T02:04:20.268 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T02:04:20.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:20 smithi066 ceph-mon[35001]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-21T02:04:20.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:20 smithi029 ceph-mon[31228]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-21T02:04:20.514 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1248, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1250, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3227326, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 396928, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 7, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 2, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.118999371, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.046075765, 2024-09-21T02:04:21.001 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.030691923, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.032615170, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1248, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.048406476, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000038787 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:04:21.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1283, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1249, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 998994, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 486081, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.134790764, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.054075879, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.049587767, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.030482173, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1249, 2024-09-21T02:04:21.003 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.058039624, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000046468 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 768224, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 187792 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1058, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1287, 2024-09-21T02:04:21.004 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 355312, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 822462, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.143386265, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.066331082, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.029447112, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.036362156, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1285, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.099847435, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000077702 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.005 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 486336 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5475, 2024-09-21T02:04:21.006 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.155103965, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000028329 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-09-21T02:04:21.007 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-21T02:04:21.008 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 385, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 32, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 353, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 338, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 5, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:04:21.009 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-21T02:04:21.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-21T02:04:21.011 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-21T02:04:21.012 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-09-21T02:04:21.013 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 1668, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-21T02:04:21.014 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 3, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 7097023, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 7096424, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 1668 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-21T02:04:21.015 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 157, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 645, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 488, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 14, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 46, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 32, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 385, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 662, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 277, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 120108, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 318521344 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-21T02:04:21.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-21T02:04:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.018 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.020 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-21T02:04:21.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-21T02:04:21.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-21T02:04:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2399, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:04:21.028 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:04:21.029 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:04:21.030 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 195660, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 201, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7496, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 154, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:04:21.031 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 643590, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 4842, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3243, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2024-09-21T02:04:21.032 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3243, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3243, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 22.547361782, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.006952624 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3243, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3244 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3243, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 3243, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-09-21T02:04:21.033 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1993, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1248, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-21T02:04:21.034 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-21T02:04:21.035 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-21T02:04:21.036 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 76, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 5, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 6, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 1, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-21T02:04:21.037 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:21.038 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3585, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 4238597, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3585, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3585, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4238597, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.039 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3243, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2902724, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3243, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2902724, 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:21.040 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3243, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3243, 2024-09-21T02:04:21.041 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3243, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3243, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:21.042 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x5605550a52b0": { 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-21T02:04:21.043 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-21T02:04:21.044 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-21T02:04:21.045 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-21T02:04:21.828 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T02:04:21.828 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T02:04:21.829 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-21T02:04:21.829 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-21T02:04:21.829 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T02:04:21.838 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T02:04:21.838 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T02:04:21.838 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:04:21.861 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-21T02:04:21.861 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:04:21.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T02:04:22.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:22 smithi190 ceph-mon[34368]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-21T02:04:22.237 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:22.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:22 smithi066 ceph-mon[35001]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-21T02:04:22.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:22 smithi029 ceph-mon[31228]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-21T02:04:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 -- 172.21.15.29:0/2501397156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 msgr2=0x7fb6941098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 --2- 172.21.15.29:0/2501397156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb6941098c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb684009900 tx=0x7fb68402a410 comp rx=0 tx=0).stop 2024-09-21T02:04:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 -- 172.21.15.29:0/2501397156 shutdown_connections 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 --2- 172.21.15.29:0/2501397156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694109e00 0x7fb694110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 --2- 172.21.15.29:0/2501397156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb6941098c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 --2- 172.21.15.29:0/2501397156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694103bc0 0x7fb694103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 -- 172.21.15.29:0/2501397156 >> 172.21.15.29:0/2501397156 conn(0x7fb69407b910 msgr2=0x7fb69407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 -- 172.21.15.29:0/2501397156 shutdown_connections 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.772+0000 7fb69bb34640 1 -- 172.21.15.29:0/2501397156 wait complete. 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 Processor -- start 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 -- start start 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694103bc0 0x7fb694075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694109e00 0x7fb694076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb694113b10 con 0x7fb6941094b0 2024-09-21T02:04:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb694113990 con 0x7fb694103bc0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69bb34640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb694113c90 con 0x7fb694109e00 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb6990cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb6990cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42908/0 (socket says 172.21.15.29:42908) 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb6990cf640 1 -- 172.21.15.29:0/1568315532 learned_addr learned my addr 172.21.15.29:0/1568315532 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb6998d0640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694103bc0 0x7fb694075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.773+0000 7fb69a0d1640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694109e00 0x7fb694076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 -- 172.21.15.29:0/1568315532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694109e00 msgr2=0x7fb694076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694109e00 0x7fb694076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 -- 172.21.15.29:0/1568315532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694103bc0 msgr2=0x7fb694075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694103bc0 0x7fb694075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 -- 172.21.15.29:0/1568315532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6941a87e0 con 0x7fb6941094b0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb6990cf640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb684005ab0 tx=0x7fb684003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68403f070 con 0x7fb6941094b0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb684031d60 con 0x7fb6941094b0 2024-09-21T02:04:22.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684035030 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941a8a70 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.774+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6941a8f00 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.775+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694103fd0 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.777+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb684042070 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.778+0000 7fb68affd640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 0x7fb6800b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.778+0000 7fb6998d0640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 0x7fb6800b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.778+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fb6841080e0 con 0x7fb6941094b0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.778+0000 7fb6998d0640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 0x7fb6800b8aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb694077160 tx=0x7fb69000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.785+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6840d2d60 con 0x7fb6941094b0 2024-09-21T02:04:22.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.987+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb69410bbd0 con 0x7fb6941094b0 2024-09-21T02:04:22.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.988+0000 7fb68affd640 1 -- 172.21.15.29:0/1568315532 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v76) ==== 81+0+590 (secure 0 0 0) 0x7fb6840d7ca0 con 0x7fb6941094b0 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/1068014806 2024-09-22T01:38:31.990579+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/2194300330 2024-09-22T01:38:31.990579+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6801/3233923662 2024-09-22T01:38:31.990579+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/363656689 2024-09-22T01:38:04.746763+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/1115869563 2024-09-22T01:38:31.990579+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/280632558 2024-09-22T01:38:04.746763+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:0/1445197802 2024-09-22T01:38:04.746763+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6800/3233923662 2024-09-22T01:38:31.990579+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6801/2164591336 2024-09-22T01:38:04.746763+0000 2024-09-21T02:04:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6800/2164591336 2024-09-22T01:38:04.746763+0000 2024-09-21T02:04:22.990 INFO:teuthology.orchestra.run.smithi029.stderr:listed 10 entries 2024-09-21T02:04:22.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 msgr2=0x7fb6800b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 0x7fb6800b8aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb694077160 tx=0x7fb69000b390 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 msgr2=0x7fb694075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb684005ab0 tx=0x7fb684003cd0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 shutdown_connections 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fb6800b6590 0x7fb6800b8aa0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb694109e00 0x7fb694076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6941094b0 0x7fb694075c50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 --2- 172.21.15.29:0/1568315532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694103bc0 0x7fb694075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 >> 172.21.15.29:0/1568315532 conn(0x7fb69407b910 msgr2=0x7fb694105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 shutdown_connections 2024-09-21T02:04:22.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:22.991+0000 7fb69bb34640 1 -- 172.21.15.29:0/1568315532 wait complete. 2024-09-21T02:04:23.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:23 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/1568315532' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:04:23.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:23 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/1568315532' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:04:23.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:23 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/1568315532' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:04:23.733 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T02:04:23.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:23.733 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /tmp/cephfs 2024-09-21T02:04:23.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T02:04:23.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:23.980 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-21T02:04:24.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:24.043 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-09-21T02:04:24.091 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-21T02:04:24.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:24.092 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-21T02:04:24.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:24.166 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-09-21T02:04:24.229 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:24 smithi029 ceph-mon[31228]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T02:04:24.259 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T02:04:24.276 INFO:tasks.cephadm:Teardown begin 2024-09-21T02:04:24.276 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:04:24.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:24.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:24 smithi066 ceph-mon[35001]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T02:04:24.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:24 smithi190 ceph-mon[34368]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T02:04:24.881 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 -- 172.21.15.29:0/545810189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0601049c0 msgr2=0x7fe060104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 --2- 172.21.15.29:0/545810189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0601049c0 0x7fe060104dd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe050009900 tx=0x7fe05002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 -- 172.21.15.29:0/545810189 shutdown_connections 2024-09-21T02:04:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 --2- 172.21.15.29:0/545810189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe06010de20 0x7fe060110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 --2- 172.21.15.29:0/545810189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe060105310 0x7fe06010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 --2- 172.21.15.29:0/545810189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0601049c0 0x7fe060104dd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 -- 172.21.15.29:0/545810189 >> 172.21.15.29:0/545810189 conn(0x7fe060079710 msgr2=0x7fe060079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 -- 172.21.15.29:0/545810189 shutdown_connections 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.518+0000 7fe06490a640 1 -- 172.21.15.29:0/545810189 wait complete. 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 Processor -- start 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 -- start start 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0601049c0 0x7fe06019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06010de20 0x7fe06019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113aa0 con 0x7fe060105310 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113920 con 0x7fe06010de20 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe06490a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113c20 con 0x7fe0601049c0 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe05dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe05dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42920/0 (socket says 172.21.15.29:42920) 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe05dd9b640 1 -- 172.21.15.29:0/344936910 learned_addr learned my addr 172.21.15.29:0/344936910 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.519+0000 7fe05e59c640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0601049c0 0x7fe06019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05ed9d640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06010de20 0x7fe06019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 -- 172.21.15.29:0/344936910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0601049c0 msgr2=0x7fe06019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0601049c0 0x7fe06019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 -- 172.21.15.29:0/344936910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06010de20 msgr2=0x7fe06019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06010de20 0x7fe06019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 -- 172.21.15.29:0/344936910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0601a0500 con 0x7fe060105310 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe05dd9b640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400ccc0 con 0x7fe060105310 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0540108e0 con 0x7fe060105310 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054018860 con 0x7fe060105310 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0601a07f0 con 0x7fe060105310 2024-09-21T02:04:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.520+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601a6f50 con 0x7fe060105310 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.521+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe06006c280 con 0x7fe060105310 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.523+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe054010430 con 0x7fe060105310 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.523+0000 7fe04f7fe640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 0x7fe0480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.524+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fe0540e7aa0 con 0x7fe060105310 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.524+0000 7fe05e59c640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 0x7fe0480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.524+0000 7fe05e59c640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 0x7fe0480b8a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe050031440 tx=0x7fe050009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.526+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0540ae340 con 0x7fe060105310 2024-09-21T02:04:25.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.752+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe060104dd0 con 0x7fe060105310 2024-09-21T02:04:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.753+0000 7fe04f7fe640 1 -- 172.21.15.29:0/344936910 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11990 (secure 0 0 0) 0x7fe0540b2720 con 0x7fe060105310 2024-09-21T02:04:25.754 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:04:25.754 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":56,"btime":"2024-09-21T02:04:15:991515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:04:15.256401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T02:04:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 56 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 msgr2=0x7fe0480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 0x7fe0480b8a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe050031440 tx=0x7fe050009560 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 msgr2=0x7fe06019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 shutdown_connections 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe0480b6510 0x7fe0480b8a20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe06010de20 0x7fe06019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe060105310 0x7fe06019ab30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 --2- 172.21.15.29:0/344936910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0601049c0 0x7fe06019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 >> 172.21.15.29:0/344936910 conn(0x7fe060079710 msgr2=0x7fe0601079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 shutdown_connections 2024-09-21T02:04:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:25.756+0000 7fe06490a640 1 -- 172.21.15.29:0/344936910 wait complete. 2024-09-21T02:04:26.290 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:26 smithi190 ceph-mon[34368]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-21T02:04:26.290 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:26 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/344936910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:26 smithi066 ceph-mon[35001]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-21T02:04:26.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:26 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/344936910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:26.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:26 smithi029 ceph-mon[31228]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-21T02:04:26.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:26 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/344936910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:26.519 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 -- 172.21.15.29:0/2985627882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730103b60 msgr2=0x7f3730103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 --2- 172.21.15.29:0/2985627882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730103b60 0x7f3730103f70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3728009900 tx=0x7f372802a410 comp rx=0 tx=0).stop 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 -- 172.21.15.29:0/2985627882 shutdown_connections 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 --2- 172.21.15.29:0/2985627882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730109df0 0x7f3730110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 --2- 172.21.15.29:0/2985627882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 0x7f37301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 --2- 172.21.15.29:0/2985627882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730103b60 0x7f3730103f70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 -- 172.21.15.29:0/2985627882 >> 172.21.15.29:0/2985627882 conn(0x7f373007b850 msgr2=0x7f373007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 -- 172.21.15.29:0/2985627882 shutdown_connections 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 -- 172.21.15.29:0/2985627882 wait complete. 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.287+0000 7f3737da0640 1 Processor -- start 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 -- start start 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 0x7f3730075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 0x7f3730075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3730113ab0 con 0x7f3730109df0 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3730113930 con 0x7f3730103b60 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3737da0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3730113c30 con 0x7f37301094a0 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42944/0 (socket says 172.21.15.29:42944) 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3735b3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 0x7f3730075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 -- 172.21.15.29:0/953155017 learned_addr learned my addr 172.21.15.29:0/953155017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373533b640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 0x7f3730075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 -- 172.21.15.29:0/953155017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 msgr2=0x7f3730075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 0x7f3730075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 -- 172.21.15.29:0/953155017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 msgr2=0x7f3730075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 0x7f3730075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f373633d640 1 -- 172.21.15.29:0/953155017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301a8760 con 0x7f3730109df0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.288+0000 7f3735b3c640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 0x7f3730075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f373633d640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f372000eb10 tx=0x7f372000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720004b20 con 0x7f3730109df0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301a8a50 con 0x7f3730109df0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3720004cc0 con 0x7f3730109df0 2024-09-21T02:04:27.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720010670 con 0x7f3730109df0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.289+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301a8f40 con 0x7f3730109df0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.290+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3730103f70 con 0x7f3730109df0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.292+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3720005000 con 0x7f3730109df0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.292+0000 7f372effd640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 0x7f37180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.293+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f37200e7a30 con 0x7f3730109df0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.293+0000 7f3735b3c640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 0x7f37180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.293+0000 7f3735b3c640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 0x7f37180b8a20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3728000c00 tx=0x7f3728031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.295+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3720056710 con 0x7f3730109df0 2024-09-21T02:04:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.525+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3730076a10 con 0x7f3730109df0 2024-09-21T02:04:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.525+0000 7f372effd640 1 -- 172.21.15.29:0/953155017 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11990 (secure 0 0 0) 0x7f3730076a10 con 0x7f3730109df0 2024-09-21T02:04:27.526 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:04:27.526 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":56,"btime":"2024-09-21T02:04:15:991515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14712,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3427004495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3427004495},{"type":"v1","addr":"172.21.15.29:6835","nonce":3427004495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/333752891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":333752891},{"type":"v1","addr":"172.21.15.29:6837","nonce":333752891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:46:10.616066+0000","modified":"2024-09-21T02:04:15.256401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24644,"mds_2":24640,"mds_3":24625,"mds_4":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6839/3540512645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3540512645},{"type":"v1","addr":"172.21.15.29:6839","nonce":3540512645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14784":{"gid":14784,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6841/2160178023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6840","nonce":2160178023},{"type":"v1","addr":"172.21.15.29:6841","nonce":2160178023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/3991511259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3991511259},{"type":"v1","addr":"172.21.15.190:6833","nonce":3991511259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/3428222627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3428222627},{"type":"v1","addr":"172.21.15.66:6833","nonce":3428222627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/4140562793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4140562793},{"type":"v1","addr":"172.21.15.190:6835","nonce":4140562793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.66:6835/2455557806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2455557806},{"type":"v1","addr":"172.21.15.66:6835","nonce":2455557806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/2047991561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2047991561},{"type":"v1","addr":"172.21.15.190:6837","nonce":2047991561}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6839/636701059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":636701059},{"type":"v1","addr":"172.21.15.190:6839","nonce":636701059}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.66:6837/1538999358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1538999358},{"type":"v1","addr":"172.21.15.66:6837","nonce":1538999358}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6839/2350524023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2350524023},{"type":"v1","addr":"172.21.15.66:6839","nonce":2350524023}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24620,24625,24640,24644,24671]},"id":1}]} 2024-09-21T02:04:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 56 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 msgr2=0x7f37180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 0x7f37180b8a20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3728000c00 tx=0x7f3728031400 comp rx=0 tx=0).stop 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 msgr2=0x7f3730076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f372000eb10 tx=0x7f372000c2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 shutdown_connections 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f37180b6510 0x7f37180b8a20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3730109df0 0x7f3730076190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37301094a0 0x7f3730075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 --2- 172.21.15.29:0/953155017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3730103b60 0x7f3730075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 >> 172.21.15.29:0/953155017 conn(0x7f373007b850 msgr2=0x7f3730107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 shutdown_connections 2024-09-21T02:04:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:27.528+0000 7f3737da0640 1 -- 172.21.15.29:0/953155017 wait complete. 2024-09-21T02:04:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:28 smithi066 ceph-mon[35001]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T02:04:28.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:28 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/953155017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:28.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:28 smithi029 ceph-mon[31228]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T02:04:28.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:28 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/953155017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:28 smithi190 ceph-mon[34368]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T02:04:28.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:28 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/953155017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:28.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:04:28.775 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:29.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 -- 172.21.15.29:0/1563799084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a70109df0 msgr2=0x7f5a70110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 --2- 172.21.15.29:0/1563799084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a70109df0 0x7f5a70110620 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ca0 tx=0x7f5a6802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 -- 172.21.15.29:0/1563799084 shutdown_connections 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 --2- 172.21.15.29:0/1563799084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a70109df0 0x7f5a70110620 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 --2- 172.21.15.29:0/1563799084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a701094a0 0x7f5a701098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 --2- 172.21.15.29:0/1563799084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 0x7f5a70103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 -- 172.21.15.29:0/1563799084 >> 172.21.15.29:0/1563799084 conn(0x7f5a7007b850 msgr2=0x7f5a7007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 -- 172.21.15.29:0/1563799084 shutdown_connections 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.271+0000 7f5a750d1640 1 -- 172.21.15.29:0/1563799084 wait complete. 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 Processor -- start 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 -- start start 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 0x7f5a7019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a70109df0 0x7f5a7019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70113ab0 con 0x7f5a701094a0 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70113930 con 0x7f5a70109df0 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a750d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70113c30 con 0x7f5a70103b60 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a6e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.272+0000 7f5a6e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42948/0 (socket says 172.21.15.29:42948) 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 -- 172.21.15.29:0/960573447 learned_addr learned my addr 172.21.15.29:0/960573447 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6f59e640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a70109df0 0x7f5a7019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6ed9d640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 0x7f5a7019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 -- 172.21.15.29:0/960573447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 msgr2=0x7f5a7019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 0x7f5a7019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 -- 172.21.15.29:0/960573447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a70109df0 msgr2=0x7f5a7019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a70109df0 0x7f5a7019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 -- 172.21.15.29:0/960573447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a701a03f0 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a6e59c640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d950 tx=0x7f5a6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64017070 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a701a06e0 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a64009c30 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640047a0 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.273+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701a6e40 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.274+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70103f70 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.276+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5a64004020 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.277+0000 7f5a57fff640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 0x7f5a580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.277+0000 7f5a6ed9d640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 0x7f5a580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.277+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7f5a640e73b0 con 0x7f5a701094a0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.277+0000 7f5a6ed9d640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 0x7f5a580b8a20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005fd0 tx=0x7f5a60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.280+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a64020020 con 0x7f5a701094a0 2024-09-21T02:04:29.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.470+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a7019b5b0 con 0x7f5a701094a0 2024-09-21T02:04:29.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.471+0000 7f5a57fff640 1 -- 172.21.15.29:0/960573447 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22674 (secure 0 0 0) 0x7f5a7019b5b0 con 0x7f5a701094a0 2024-09-21T02:04:29.472 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-21T02:04:29.472 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"e38bc6b6-77b9-11ef-baf5-efdc52797490","created":"2024-09-21T01:37:42.108556+0000","modified":"2024-09-21T01:46:33.220142+0000","last_up_change":"2024-09-21T01:45:19.080557+0000","last_in_change":"2024-09-21T01:44:54.800797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:41:30.465145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:46:06.321761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:46:08.562000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c63b551-55db-4486-b93e-1f5773c9cdaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6803","nonce":271350255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6805","nonce":271350255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6809","nonce":271350255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":271350255},{"type":"v1","addr":"172.21.15.29:6807","nonce":271350255}]},"public_addr":"172.21.15.29:6803/271350255","cluster_addr":"172.21.15.29:6805/271350255","heartbeat_back_addr":"172.21.15.29:6809/271350255","heartbeat_front_addr":"172.21.15.29:6807/271350255","state":["exists","up"]},{"osd":1,"uuid":"9f29e6aa-4705-4810-9e3a-0118bec1dd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6801","nonce":2183970412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6803","nonce":2183970412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6807","nonce":2183970412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2183970412},{"type":"v1","addr":"172.21.15.66:6805","nonce":2183970412}]},"public_addr":"172.21.15.66:6801/2183970412","cluster_addr":"172.21.15.66:6803/2183970412","heartbeat_back_addr":"172.21.15.66:6807/2183970412","heartbeat_front_addr":"172.21.15.66:6805/2183970412","state":["exists","up"]},{"osd":2,"uuid":"ebe1d952-803c-4222-88cd-cff3887aaecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6801","nonce":2887309374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6803","nonce":2887309374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6807","nonce":2887309374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2887309374},{"type":"v1","addr":"172.21.15.190:6805","nonce":2887309374}]},"public_addr":"172.21.15.190:6801/2887309374","cluster_addr":"172.21.15.190:6803/2887309374","heartbeat_back_addr":"172.21.15.190:6807/2887309374","heartbeat_front_addr":"172.21.15.190:6805/2887309374","state":["exists","up"]},{"osd":3,"uuid":"0f101639-e43c-4377-8de2-63ffbe3ce76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6811","nonce":1440460638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6813","nonce":1440460638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6817","nonce":1440460638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1440460638},{"type":"v1","addr":"172.21.15.29:6815","nonce":1440460638}]},"public_addr":"172.21.15.29:6811/1440460638","cluster_addr":"172.21.15.29:6813/1440460638","heartbeat_back_addr":"172.21.15.29:6817/1440460638","heartbeat_front_addr":"172.21.15.29:6815/1440460638","state":["exists","up"]},{"osd":4,"uuid":"4ffbf866-c852-4333-ac24-19c595975987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6809","nonce":2683216793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6811","nonce":2683216793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6815","nonce":2683216793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2683216793},{"type":"v1","addr":"172.21.15.66:6813","nonce":2683216793}]},"public_addr":"172.21.15.66:6809/2683216793","cluster_addr":"172.21.15.66:6811/2683216793","heartbeat_back_addr":"172.21.15.66:6815/2683216793","heartbeat_front_addr":"172.21.15.66:6813/2683216793","state":["exists","up"]},{"osd":5,"uuid":"429eb433-7213-450d-a624-bb75cff27b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6809","nonce":4247249906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6811","nonce":4247249906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6815","nonce":4247249906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4247249906},{"type":"v1","addr":"172.21.15.190:6813","nonce":4247249906}]},"public_addr":"172.21.15.190:6809/4247249906","cluster_addr":"172.21.15.190:6811/4247249906","heartbeat_back_addr":"172.21.15.190:6815/4247249906","heartbeat_front_addr":"172.21.15.190:6813/4247249906","state":["exists","up"]},{"osd":6,"uuid":"b3d0fb70-1bc7-4a2d-9a80-eff59ed00770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6819","nonce":797110744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6821","nonce":797110744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6825","nonce":797110744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":797110744},{"type":"v1","addr":"172.21.15.29:6823","nonce":797110744}]},"public_addr":"172.21.15.29:6819/797110744","cluster_addr":"172.21.15.29:6821/797110744","heartbeat_back_addr":"172.21.15.29:6825/797110744","heartbeat_front_addr":"172.21.15.29:6823/797110744","state":["exists","up"]},{"osd":7,"uuid":"24bf68e6-9205-4bee-9d74-38c475daeada","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6817","nonce":3795777782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6819","nonce":3795777782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6823","nonce":3795777782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3795777782},{"type":"v1","addr":"172.21.15.66:6821","nonce":3795777782}]},"public_addr":"172.21.15.66:6817/3795777782","cluster_addr":"172.21.15.66:6819/3795777782","heartbeat_back_addr":"172.21.15.66:6823/3795777782","heartbeat_front_addr":"172.21.15.66:6821/3795777782","state":["exists","up"]},{"osd":8,"uuid":"fafc4cd9-401d-402c-8248-1a219ef74397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6817","nonce":1425209392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6819","nonce":1425209392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6823","nonce":1425209392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1425209392},{"type":"v1","addr":"172.21.15.190:6821","nonce":1425209392}]},"public_addr":"172.21.15.190:6817/1425209392","cluster_addr":"172.21.15.190:6819/1425209392","heartbeat_back_addr":"172.21.15.190:6823/1425209392","heartbeat_front_addr":"172.21.15.190:6821/1425209392","state":["exists","up"]},{"osd":9,"uuid":"d46d9b84-5641-4633-9077-c086f62e0617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6827","nonce":246841085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6829","nonce":246841085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6833","nonce":246841085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":246841085},{"type":"v1","addr":"172.21.15.29:6831","nonce":246841085}]},"public_addr":"172.21.15.29:6827/246841085","cluster_addr":"172.21.15.29:6829/246841085","heartbeat_back_addr":"172.21.15.29:6833/246841085","heartbeat_front_addr":"172.21.15.29:6831/246841085","state":["exists","up"]},{"osd":10,"uuid":"cd901192-e9bc-46ef-835f-a4dd14265604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6825","nonce":1772263510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6827","nonce":1772263510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6831","nonce":1772263510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1772263510},{"type":"v1","addr":"172.21.15.66:6829","nonce":1772263510}]},"public_addr":"172.21.15.66:6825/1772263510","cluster_addr":"172.21.15.66:6827/1772263510","heartbeat_back_addr":"172.21.15.66:6831/1772263510","heartbeat_front_addr":"172.21.15.66:6829/1772263510","state":["exists","up"]},{"osd":11,"uuid":"962f6e91-94b7-4a35-9aec-7011884d4b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6825","nonce":3508320462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6827","nonce":3508320462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6831","nonce":3508320462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3508320462},{"type":"v1","addr":"172.21.15.190:6829","nonce":3508320462}]},"public_addr":"172.21.15.190:6825/3508320462","cluster_addr":"172.21.15.190:6827/3508320462","heartbeat_back_addr":"172.21.15.190:6831/3508320462","heartbeat_front_addr":"172.21.15.190:6829/3508320462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:40:36.602501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:02.542910+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:27.369378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:41:53.844050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:19.790898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:42:44.739991+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:11.009831+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:43:36.887892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:01.733496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:26.597005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:44:51.935500+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:45:16.313180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:0/1068014806":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/2194300330":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/363656689":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1115869563":"2024-09-22T01:38:31.990579+0000","172.21.15.29:0/280632558":"2024-09-22T01:38:04.746763+0000","172.21.15.29:0/1445197802":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/3233923662":"2024-09-22T01:38:31.990579+0000","172.21.15.29:6801/2164591336":"2024-09-22T01:38:04.746763+0000","172.21.15.29:6800/2164591336":"2024-09-22T01:38:04.746763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:29.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.473+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 msgr2=0x7f5a580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.473+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 0x7f5a580b8a20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005fd0 tx=0x7f5a60005850 comp rx=0 tx=0).stop 2024-09-21T02:04:29.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.473+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 msgr2=0x7f5a7019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.473+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d950 tx=0x7f5a6400de10 comp rx=0 tx=0).stop 2024-09-21T02:04:29.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 shutdown_connections 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f5a580b6510 0x7f5a580b8a20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a70109df0 0x7f5a7019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a701094a0 0x7f5a7019aa20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 --2- 172.21.15.29:0/960573447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a70103b60 0x7f5a7019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 >> 172.21.15.29:0/960573447 conn(0x7f5a7007b850 msgr2=0x7f5a70105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 shutdown_connections 2024-09-21T02:04:29.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:29.474+0000 7f5a750d1640 1 -- 172.21.15.29:0/960573447 wait complete. 2024-09-21T02:04:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:30 smithi066 ceph-mon[35001]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-21T02:04:30.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:30 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/960573447' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:30.300 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:30 smithi029 ceph-mon[31228]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-21T02:04:30.300 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:30 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/960573447' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:30.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T02:04:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:30 smithi190 ceph-mon[34368]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-21T02:04:30.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:30 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/960573447' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:30.602 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 -- 172.21.15.29:0/3292998909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c104880 msgr2=0x7fea6c104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 --2- 172.21.15.29:0/3292998909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c104880 0x7fea6c104c90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c02a410 comp rx=0 tx=0).stop 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 -- 172.21.15.29:0/3292998909 shutdown_connections 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 --2- 172.21.15.29:0/3292998909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 0x7fea6c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 --2- 172.21.15.29:0/3292998909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c1052b0 0x7fea6c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 --2- 172.21.15.29:0/3292998909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c104880 0x7fea6c104c90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 -- 172.21.15.29:0/3292998909 >> 172.21.15.29:0/3292998909 conn(0x7fea6c079730 msgr2=0x7fea6c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 -- 172.21.15.29:0/3292998909 shutdown_connections 2024-09-21T02:04:31.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.099+0000 7fea7275a640 1 -- 172.21.15.29:0/3292998909 wait complete. 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 Processor -- start 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 -- start start 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c104880 0x7fea6c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 0x7fea6c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c113ad0 con 0x7fea6c1052b0 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c113950 con 0x7fea6c10de30 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea7275a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c113c50 con 0x7fea6c104880 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea6b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea6b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42960/0 (socket says 172.21.15.29:42960) 2024-09-21T02:04:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.100+0000 7fea6b7fe640 1 -- 172.21.15.29:0/2242753307 learned_addr learned my addr 172.21.15.29:0/2242753307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea70cf7640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 0x7fea6c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6bfff640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c104880 0x7fea6c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 -- 172.21.15.29:0/2242753307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c104880 msgr2=0x7fea6c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c104880 0x7fea6c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 -- 172.21.15.29:0/2242753307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 msgr2=0x7fea6c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 0x7fea6c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 -- 172.21.15.29:0/2242753307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c1a0470 con 0x7fea6c1052b0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea6b7fe640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fea6000b450 tx=0x7fea6000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60015020 con 0x7fea6c1052b0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea60011d40 con 0x7fea6c1052b0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c1a0760 con 0x7fea6c1052b0 2024-09-21T02:04:31.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000f340 con 0x7fea6c1052b0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.101+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c1a6ec0 con 0x7fea6c1052b0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.102+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c104c90 con 0x7fea6c1052b0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.104+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fea60018020 con 0x7fea6c1052b0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.105+0000 7fea697fa640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 0x7fea540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.105+0000 7fea6bfff640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 0x7fea540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.105+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8211+0+0 (secure 0 0 0) 0x7fea600e7180 con 0x7fea6c1052b0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.105+0000 7fea6bfff640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 0x7fea540b8a20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:31.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.108+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea6001f020 con 0x7fea6c1052b0 2024-09-21T02:04:31.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.336+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fea6c19b630 con 0x7fea6c1052b0 2024-09-21T02:04:31.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.339+0000 7fea697fa640 1 -- 172.21.15.29:0/2242753307 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7fea6000d040 con 0x7fea6c1052b0 2024-09-21T02:04:31.340 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:04:31.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.342+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 msgr2=0x7fea540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.342+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 0x7fea540b8a20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c031400 comp rx=0 tx=0).stop 2024-09-21T02:04:31.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.342+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 msgr2=0x7fea6c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.342+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fea6000b450 tx=0x7fea6000b910 comp rx=0 tx=0).stop 2024-09-21T02:04:31.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 shutdown_connections 2024-09-21T02:04:31.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fea540b6510 0x7fea540b8a20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea6c10de30 0x7fea6c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea6c1052b0 0x7fea6c19aaa0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 --2- 172.21.15.29:0/2242753307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea6c104880 0x7fea6c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.344+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 >> 172.21.15.29:0/2242753307 conn(0x7fea6c079730 msgr2=0x7fea6c1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.345+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 shutdown_connections 2024-09-21T02:04:31.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:31.345+0000 7fea7275a640 1 -- 172.21.15.29:0/2242753307 wait complete. 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a[31224]: 2024-09-21T02:04:31.337+0000 7fc00a798640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-mds[69775]: -- 172.21.15.29:0/1675177918 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638b382f860 con 0x5638b79d7180 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: starting mds.g at 2024-09-21T02:04:31.595 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a[70076]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.596 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a[70076]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.596 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-mds[70094]: -- 172.21.15.29:0/1562316270 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbd3fd1860 con 0x55cbd8179180 2024-09-21T02:04:31.596 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:04:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a[70076]: starting mds.a at 2024-09-21T02:04:31.728 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.728 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.728 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-mds[61940]: -- 172.21.15.190:0/3915662459 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55974570bc20 con 0x5597498b3600 2024-09-21T02:04:31.728 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: starting mds.f at 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-mds[62211]: -- 172.21.15.190:0/1394289501 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561575051c20 con 0x5615791f9600 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: starting mds.c at 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i[62483]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i[62483]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.729 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i[62483]: starting mds.i at 2024-09-21T02:04:31.730 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l[61664]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.730 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l[61664]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.730 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-mds[61668]: -- 172.21.15.190:0/1912586708 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56045f103860 con 0x5604632ab600 2024-09-21T02:04:31.730 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:04:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l[61664]: starting mds.l at 2024-09-21T02:04:31.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-mds[62777]: -- 172.21.15.66:0/2243489615 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea91c3d860 con 0x55ea95de5a80 2024-09-21T02:04:31.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: starting mds.k at 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-mds[63606]: -- 172.21.15.66:0/985500237 <== mon.0 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55def71d5860 con 0x55defb37da80 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: starting mds.b at 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-mds[63056]: -- 172.21.15.66:0/1613036605 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df76cc1860 con 0x55df7ae69a80 2024-09-21T02:04:31.800 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: starting mds.e at 2024-09-21T02:04:31.801 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h[63336]: ignoring --setuser ceph since I am not root 2024-09-21T02:04:31.801 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h[63336]: ignoring --setgroup ceph since I am not root 2024-09-21T02:04:31.801 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-mds[63340]: -- 172.21.15.66:0/535148613 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4caa37860 con 0x55d4cebdf180 2024-09-21T02:04:31.801 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:04:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h[63336]: starting mds.h at 2024-09-21T02:04:32.165 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:04:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T02:04:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:04:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:04:32.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:04:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:04:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:04:32.300 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:32 smithi066 ceph-mon[35001]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:04:32.344 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:04:32.345 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:32 smithi029 ceph-mon[31228]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:04:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T02:04:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:04:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:04:32.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:04:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2242753307' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:04:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:04:32.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:32 smithi190 ceph-mon[34368]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 -- 172.21.15.29:0/771541328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 msgr2=0x7f4f78103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/771541328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f78103f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f7402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 -- 172.21.15.29:0/771541328 shutdown_connections 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/771541328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f78109df0 0x7f4f78110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/771541328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f781094a0 0x7f4f781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/771541328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f78103f70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 -- 172.21.15.29:0/771541328 >> 172.21.15.29:0/771541328 conn(0x7f4f7807b850 msgr2=0x7f4f7807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 -- 172.21.15.29:0/771541328 shutdown_connections 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.922+0000 7f4f7fac4640 1 -- 172.21.15.29:0/771541328 wait complete. 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 Processor -- start 2024-09-21T02:04:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 -- start start 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f781094a0 0x7f4f7819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f78109df0 0x7f4f7819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78113ab0 con 0x7f4f78103b60 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78113930 con 0x7f4f781094a0 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7fac4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78113c30 con 0x7f4f78109df0 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7d860640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7d860640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35684/0 (socket says 172.21.15.29:35684) 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7d860640 1 -- 172.21.15.29:0/2473362790 learned_addr learned my addr 172.21.15.29:0/2473362790 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7e061640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f78109df0 0x7f4f7819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.923+0000 7f4f7d05f640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f781094a0 0x7f4f7819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 -- 172.21.15.29:0/2473362790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f78109df0 msgr2=0x7f4f7819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f78109df0 0x7f4f7819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 -- 172.21.15.29:0/2473362790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f781094a0 msgr2=0x7f4f7819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f781094a0 0x7f4f7819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 -- 172.21.15.29:0/2473362790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f781a0350 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7d860640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005940 tx=0x7f4f740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7403f030 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f781a05e0 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f74003cb0 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74035070 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.924+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f781a0b10 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.925+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f78103f70 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.927+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4f74042030 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.927+0000 7f4f6effd640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 0x7f4f640b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.927+0000 7f4f7d05f640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 0x7f4f640b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.927+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8641+0+0 (secure 0 0 0) 0x7f4f74107bb0 con 0x7f4f78103b60 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.928+0000 7f4f7d05f640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 0x7f4f640b8a50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f78106530 tx=0x7f4f6800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:32.931+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f781a0b10 con 0x7f4f78103b60 2024-09-21T02:04:33.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.156+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4f7810bbf0 con 0x7f4f78103b60 2024-09-21T02:04:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.346+0000 7f4f6effd640 1 -- 172.21.15.29:0/2473362790 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f4f740ce2a0 con 0x7f4f78103b60 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 msgr2=0x7f4f640b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 0x7f4f640b8a50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f78106530 tx=0x7f4f6800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 msgr2=0x7f4f7819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005940 tx=0x7f4f740042f0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 shutdown_connections 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7f4f640b6540 0x7f4f640b8a50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f78109df0 0x7f4f7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f781094a0 0x7f4f7819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 --2- 172.21.15.29:0/2473362790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f78103b60 0x7f4f7819a4a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 >> 172.21.15.29:0/2473362790 conn(0x7f4f7807b850 msgr2=0x7f4f78107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 shutdown_connections 2024-09-21T02:04:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:33.349+0000 7f4f7fac4640 1 -- 172.21.15.29:0/2473362790 wait complete. 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6838/3902748063,v1:172.21.15.29:6840/3902748063] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.29:6839/2164318423,v1:172.21.15.29:6841/2164318423] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6832/6495084,v1:172.21.15.190:6833/6495084] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6834/747167277,v1:172.21.15.190:6835/747167277] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6836/1969821888,v1:172.21.15.190:6837/1969821888] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.190:6838/172236525,v1:172.21.15.190:6839/172236525] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6833/2301302559,v1:172.21.15.66:6835/2301302559] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6832/642801380,v1:172.21.15.66:6834/642801380] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6836/3892162399,v1:172.21.15.66:6837/3892162399] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: mds.? [v2:172.21.15.66:6838/2335237278,v1:172.21.15.66:6839/2335237278] up:boot 2024-09-21T02:04:33.466 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:33.467 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:33 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:04:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6838/3902748063,v1:172.21.15.29:6840/3902748063] up:boot 2024-09-21T02:04:33.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.29:6839/2164318423,v1:172.21.15.29:6841/2164318423] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6832/6495084,v1:172.21.15.190:6833/6495084] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6834/747167277,v1:172.21.15.190:6835/747167277] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6836/1969821888,v1:172.21.15.190:6837/1969821888] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.190:6838/172236525,v1:172.21.15.190:6839/172236525] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6833/2301302559,v1:172.21.15.66:6835/2301302559] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6832/642801380,v1:172.21.15.66:6834/642801380] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6836/3892162399,v1:172.21.15.66:6837/3892162399] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: mds.? [v2:172.21.15.66:6838/2335237278,v1:172.21.15.66:6839/2335237278] up:boot 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:04:33.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:33.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:33 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:04:33.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6838/3902748063,v1:172.21.15.29:6840/3902748063] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.29:6839/2164318423,v1:172.21.15.29:6841/2164318423] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6832/6495084,v1:172.21.15.190:6833/6495084] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6834/747167277,v1:172.21.15.190:6835/747167277] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6836/1969821888,v1:172.21.15.190:6837/1969821888] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.190:6838/172236525,v1:172.21.15.190:6839/172236525] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6833/2301302559,v1:172.21.15.66:6835/2301302559] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6832/642801380,v1:172.21.15.66:6834/642801380] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6836/3892162399,v1:172.21.15.66:6837/3892162399] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: mds.? [v2:172.21.15.66:6838/2335237278,v1:172.21.15.66:6839/2335237278] up:boot 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:04:33.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='mgr.14152 172.21.15.29:0/1692875914' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:33.801 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:33 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:04:34.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:04:34.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:34 smithi029 ceph-mon[31228]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:04:34.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:34 smithi029 ceph-mon[31228]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:04:34.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:34 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:04:34.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:34 smithi029 ceph-mon[31228]: fsmap 12 up:standby 2024-09-21T02:04:34.697 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:34 smithi190 ceph-mon[34368]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:04:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:34 smithi190 ceph-mon[34368]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:04:34.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:34 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:04:34.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:34 smithi190 ceph-mon[34368]: fsmap 12 up:standby 2024-09-21T02:04:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:34 smithi066 ceph-mon[35001]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:04:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:34 smithi066 ceph-mon[35001]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:04:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:34 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2473362790' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:04:34.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:34 smithi066 ceph-mon[35001]: fsmap 12 up:standby 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 -- 172.21.15.29:0/3262802126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070102a60 msgr2=0x7fe070102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 --2- 172.21.15.29:0/3262802126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070102a60 0x7fe070102e70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe068009900 tx=0x7fe06802a410 comp rx=0 tx=0).stop 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 -- 172.21.15.29:0/3262802126 shutdown_connections 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 --2- 172.21.15.29:0/3262802126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe070108cf0 0x7fe07010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 --2- 172.21.15.29:0/3262802126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0701083a0 0x7fe0701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 --2- 172.21.15.29:0/3262802126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070102a60 0x7fe070102e70 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 -- 172.21.15.29:0/3262802126 >> 172.21.15.29:0/3262802126 conn(0x7fe070079730 msgr2=0x7fe070079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 -- 172.21.15.29:0/3262802126 shutdown_connections 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.178+0000 7fe075812640 1 -- 172.21.15.29:0/3262802126 wait complete. 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 Processor -- start 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 -- start start 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe070102a60 0x7fe07019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0701083a0 0x7fe07019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe070113ac0 con 0x7fe070108cf0 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe070113940 con 0x7fe0701083a0 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe075812640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe070113c40 con 0x7fe070102a60 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.179+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35706/0 (socket says 172.21.15.29:35706) 2024-09-21T02:04:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 -- 172.21.15.29:0/3404157258 learned_addr learned my addr 172.21.15.29:0/3404157258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06effd640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe070102a60 0x7fe07019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06e7fc640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0701083a0 0x7fe07019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 -- 172.21.15.29:0/3404157258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe070102a60 msgr2=0x7fe07019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe070102a60 0x7fe07019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 -- 172.21.15.29:0/3404157258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0701083a0 msgr2=0x7fe07019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0701083a0 0x7fe07019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 -- 172.21.15.29:0/3404157258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0701a0440 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe06f7fe640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe06000cad0 tx=0x7fe06000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06000ad60 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060005020 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06000f6b0 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0701a0730 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.180+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0701a6e90 con 0x7fe070108cf0 2024-09-21T02:04:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.182+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe070102e70 con 0x7fe070108cf0 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.183+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe0600051d0 con 0x7fe070108cf0 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.184+0000 7fe074810640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 0x7fe05c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.184+0000 7fe06effd640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 0x7fe05c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.184+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8641+0+0 (secure 0 0 0) 0x7fe0600e77e0 con 0x7fe070108cf0 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.185+0000 7fe06effd640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 0x7fe05c0b8a20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe068031440 tx=0x7fe068005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.187+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe060014030 con 0x7fe070108cf0 2024-09-21T02:04:35.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:35.380+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe07019b600 con 0x7fe070108cf0 2024-09-21T02:04:35.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:35 smithi190 ceph-mon[34368]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:35.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:35 smithi066 ceph-mon[35001]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:35.844 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:35 smithi029 ceph-mon[31228]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T02:04:36.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.360+0000 7fe074810640 1 -- 172.21.15.29:0/3404157258 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) ==== 181+0+0 (secure 0 0 0) 0x7fe07019b600 con 0x7fe070108cf0 2024-09-21T02:04:36.362 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:04:36.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.364+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 msgr2=0x7fe05c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:36.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.364+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 0x7fe05c0b8a20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe068031440 tx=0x7fe068005e80 comp rx=0 tx=0).stop 2024-09-21T02:04:36.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.364+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 msgr2=0x7fe07019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:36.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.364+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe06000cad0 tx=0x7fe06000ced0 comp rx=0 tx=0).stop 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 shutdown_connections 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fe05c0b6510 0x7fe05c0b8a20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe070108cf0 0x7fe07019fd10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0701083a0 0x7fe07019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 --2- 172.21.15.29:0/3404157258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe070102a60 0x7fe07019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:36.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 >> 172.21.15.29:0/3404157258 conn(0x7fe070079730 msgr2=0x7fe070101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:36.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 shutdown_connections 2024-09-21T02:04:36.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:36.365+0000 7fe075812640 1 -- 172.21.15.29:0/3404157258 wait complete. 2024-09-21T02:04:36.713 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:36 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:36.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:36 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:36.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:36 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:37.079 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:04:37.350 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:37.619 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:37 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:37.619 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:37 smithi029 ceph-mon[31228]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:04:37.620 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:37 smithi029 ceph-mon[31228]: pgmap v850: 65 pgs: 65 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:37 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:37 smithi190 ceph-mon[34368]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:04:37.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:37 smithi190 ceph-mon[34368]: pgmap v850: 65 pgs: 65 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:37 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/3404157258' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:37 smithi066 ceph-mon[35001]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:04:37.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:37 smithi066 ceph-mon[35001]: pgmap v850: 65 pgs: 65 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 -- 172.21.15.29:0/2888278287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c105270 msgr2=0x7fed2c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 --2- 172.21.15.29:0/2888278287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c105270 0x7fed2c10d8b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fed14009900 tx=0x7fed1402a410 comp rx=0 tx=0).stop 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 -- 172.21.15.29:0/2888278287 shutdown_connections 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 --2- 172.21.15.29:0/2888278287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c10ddf0 0x7fed2c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 --2- 172.21.15.29:0/2888278287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c105270 0x7fed2c10d8b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 --2- 172.21.15.29:0/2888278287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c104840 0x7fed2c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 -- 172.21.15.29:0/2888278287 >> 172.21.15.29:0/2888278287 conn(0x7fed2c079730 msgr2=0x7fed2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 -- 172.21.15.29:0/2888278287 shutdown_connections 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 -- 172.21.15.29:0/2888278287 wait complete. 2024-09-21T02:04:37.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.837+0000 7fed31583640 1 Processor -- start 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 -- start start 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c105270 0x7fed2c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c10ddf0 0x7fed2c1031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c113a70 con 0x7fed2c104840 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1138f0 con 0x7fed2c105270 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed31583640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c113bf0 con 0x7fed2c10ddf0 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed2affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:37.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed2affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35720/0 (socket says 172.21.15.29:35720) 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed2affd640 1 -- 172.21.15.29:0/2109049743 learned_addr learned my addr 172.21.15.29:0/2109049743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed2b7fe640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c10ddf0 0x7fed2c1031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.838+0000 7fed2a7fc640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c105270 0x7fed2c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 -- 172.21.15.29:0/2109049743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c10ddf0 msgr2=0x7fed2c1031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c10ddf0 0x7fed2c1031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 -- 172.21.15.29:0/2109049743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c105270 msgr2=0x7fed2c102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c105270 0x7fed2c102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 -- 172.21.15.29:0/2109049743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a8760 con 0x7fed2c104840 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed2affd640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00eaa0 tx=0x7fed1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c016070 con 0x7fed2c104840 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a8a50 con 0x7fed2c104840 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c003e20 con 0x7fed2c104840 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0116d0 con 0x7fed2c104840 2024-09-21T02:04:37.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.839+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a8f60 con 0x7fed2c104840 2024-09-21T02:04:37.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.840+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c104c50 con 0x7fed2c104840 2024-09-21T02:04:37.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.842+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fed1c019070 con 0x7fed2c104840 2024-09-21T02:04:37.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.843+0000 7fed23fff640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 0x7fed180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:37.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.843+0000 7fed2a7fc640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 0x7fed180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:37.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.843+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8087+0+0 (secure 0 0 0) 0x7fed1c0e72d0 con 0x7fed2c104840 2024-09-21T02:04:37.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.843+0000 7fed2a7fc640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 0x7fed180b8a20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed14009230 tx=0x7fed14031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:37.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:37.847+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed1c0b1f80 con 0x7fed2c104840 2024-09-21T02:04:38.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.039+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed2c1a9330 con 0x7fed2c104840 2024-09-21T02:04:38.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.374+0000 7fed23fff640 1 -- 172.21.15.29:0/2109049743 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) ==== 169+0+0 (secure 0 0 0) 0x7fed1c0b6ec0 con 0x7fed2c104840 2024-09-21T02:04:38.375 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.378+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 msgr2=0x7fed180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.378+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 0x7fed180b8a20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed14009230 tx=0x7fed14031400 comp rx=0 tx=0).stop 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.378+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 msgr2=0x7fed2c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.378+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00eaa0 tx=0x7fed1c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 shutdown_connections 2024-09-21T02:04:38.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:6800/796732990,v1:172.21.15.29:6801/796732990] conn(0x7fed180b6510 0x7fed180b8a20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed2c10ddf0 0x7fed2c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed2c105270 0x7fed2c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 --2- 172.21.15.29:0/2109049743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed2c104840 0x7fed2c102750 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 >> 172.21.15.29:0/2109049743 conn(0x7fed2c079730 msgr2=0x7fed2c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 shutdown_connections 2024-09-21T02:04:38.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-21T02:04:38.379+0000 7fed31583640 1 -- 172.21.15.29:0/2109049743 wait complete. 2024-09-21T02:04:38.724 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:38 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:38.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:38 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:38.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:38 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:04:39.364 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:04:39.399 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:04:39.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:04:39.457 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T02:04:39.458 DEBUG:teuthology.orchestra.run.smithi029:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:04:39.473 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:04:39.489 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:04:39.512 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T02:04:39.512 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T02:04:39.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a 2024-09-21T02:04:39.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 ceph-mon[31228]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:39.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 ceph-mon[31228]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:04:39.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 ceph-mon[31228]: pgmap v852: 1 pgs: 1 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:39.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 systemd[1]: Stopping Ceph mon.a for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:39 smithi190 ceph-mon[34368]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:39 smithi190 ceph-mon[34368]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:04:39.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:39 smithi190 ceph-mon[34368]: pgmap v852: 1 pgs: 1 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:39 smithi066 ceph-mon[35001]: from='client.? 172.21.15.29:0/2109049743' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:04:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:39 smithi066 ceph-mon[35001]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:04:39.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:39 smithi066 ceph-mon[35001]: pgmap v852: 1 pgs: 1 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:04:40.237 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a[31224]: 2024-09-21T02:04:39.876+0000 7fc00ffa3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:04:40.237 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a[31224]: 2024-09-21T02:04:39.876+0000 7fc00ffa3640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T02:04:40.238 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:39 smithi029 podman[110850]: 2024-09-21 02:04:39.908479105 +0000 UTC m=+0.172593945 container died 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:04:40.594 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:40 smithi029 podman[110850]: 2024-09-21 02:04:40.473095036 +0000 UTC m=+0.737209864 container cleanup 68e5ba2ba7bf82a02fbad1db6b4c865db5693480a4b844dd687b573ebe420ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T02:04:40.595 INFO:journalctl@ceph.mon.a.smithi029.stdout:Sep 21 02:04:40 smithi029 bash[110850]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-a 2024-09-21T02:04:40.924 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T02:04:40.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:40.974 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T02:04:40.974 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T02:04:40.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.b 2024-09-21T02:04:41.289 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 systemd[1]: Stopping Ceph mon.b for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:41.542 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b[34997]: 2024-09-21T02:04:41.288+0000 7faf57c40640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:04:41.543 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b[34997]: 2024-09-21T02:04:41.288+0000 7faf57c40640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T02:04:41.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 podman[67679]: 2024-09-21 02:04:41.542747171 +0000 UTC m=+0.378020052 container died 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T02:04:42.198 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 podman[67679]: 2024-09-21 02:04:41.980634746 +0000 UTC m=+0.815907629 container cleanup 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:04:42.199 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:41 smithi066 bash[67679]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b 2024-09-21T02:04:42.451 INFO:journalctl@ceph.mon.b.smithi066.stdout:Sep 21 02:04:42 smithi066 podman[67691]: 2024-09-21 02:04:42.198608749 +0000 UTC m=+0.657365118 container remove 14d13366ac4081b6d105c44ab69e7dfd8c98b5ddb05d861ad441d2d2586e6d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T02:04:42.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T02:04:42.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:42.524 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T02:04:42.525 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T02:04:42.525 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.c 2024-09-21T02:04:42.834 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:42 smithi190 systemd[1]: Stopping Ceph mon.c for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:43.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c[34364]: 2024-09-21T02:04:42.833+0000 7f9281a68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:04:43.228 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c[34364]: 2024-09-21T02:04:42.833+0000 7f9281a68640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T02:04:43.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:43 smithi190 podman[66741]: 2024-09-21 02:04:43.047562815 +0000 UTC m=+0.331993868 container died 1a326e76f88cb62738d5477853e6138c0d9da2d6057128fddbce6692c8f26ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T02:04:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:43 smithi190 podman[66741]: 2024-09-21 02:04:43.375421111 +0000 UTC m=+0.659852171 container cleanup 1a326e76f88cb62738d5477853e6138c0d9da2d6057128fddbce6692c8f26ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T02:04:43.728 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:43 smithi190 bash[66741]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c 2024-09-21T02:04:44.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 21 02:04:43 smithi190 podman[66753]: 2024-09-21 02:04:43.785491989 +0000 UTC m=+0.734304951 container remove 1a326e76f88cb62738d5477853e6138c0d9da2d6057128fddbce6692c8f26ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:04:44.059 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T02:04:44.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:44.105 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T02:04:44.105 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T02:04:44.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x 2024-09-21T02:04:44.482 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:44 smithi029 systemd[1]: Stopping Ceph mgr.x for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:44.786 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:44 smithi029 podman[110973]: 2024-09-21 02:04:44.490653794 +0000 UTC m=+0.178198785 container died c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T02:04:45.094 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:44 smithi029 podman[110973]: 2024-09-21 02:04:44.994424218 +0000 UTC m=+0.681969249 container cleanup c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:04:45.095 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:44 smithi029 bash[110973]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x 2024-09-21T02:04:45.581 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T02:04:45.603 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:45 smithi029 podman[110985]: 2024-09-21 02:04:45.350356378 +0000 UTC m=+0.859929689 container remove c936566d986512b36d49c358a58bae3e7aba88e1ba1140ca7a9883d27a2a434b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T02:04:45.604 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:45 smithi029 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T02:04:45.604 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:45 smithi029 systemd[1]: Stopped Ceph mgr.x for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T02:04:45.604 INFO:journalctl@ceph.mgr.x.smithi029.stdout:Sep 21 02:04:45 smithi029 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.x.service: Consumed 53.729s CPU time. 2024-09-21T02:04:45.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:45.619 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T02:04:45.619 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T02:04:45.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.y 2024-09-21T02:04:45.931 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 02:04:45 smithi066 systemd[1]: Stopping Ceph mgr.y for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:46.203 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 02:04:45 smithi066 podman[67803]: 2024-09-21 02:04:45.931511057 +0000 UTC m=+0.120502604 container died 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:04:46.549 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 02:04:46 smithi066 podman[67803]: 2024-09-21 02:04:46.301637245 +0000 UTC m=+0.490628793 container cleanup 214909ed47c441e0e16cc5006cd60983268a4d9b32f734006043b45f90843a0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:04:46.549 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 21 02:04:46 smithi066 bash[67803]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-y 2024-09-21T02:04:46.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T02:04:46.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:46.764 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T02:04:46.764 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T02:04:46.765 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z 2024-09-21T02:04:47.059 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:46 smithi190 systemd[1]: Stopping Ceph mgr.z for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:47.323 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 podman[66862]: 2024-09-21 02:04:47.059541972 +0000 UTC m=+0.116042510 container died 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:04:47.590 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 podman[66862]: 2024-09-21 02:04:47.404901421 +0000 UTC m=+0.461401951 container cleanup 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:04:47.590 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 bash[66862]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z 2024-09-21T02:04:47.835 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T02:04:47.856 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 podman[66875]: 2024-09-21 02:04:47.589764126 +0000 UTC m=+0.524580352 container remove 38bdce38cac824c689877303d42350e1df0be30c567063e5779406edaf408e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T02:04:47.856 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-21T02:04:47.856 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z.service: Failed with result 'exit-code'. 2024-09-21T02:04:47.856 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 systemd[1]: Stopped Ceph mgr.z for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T02:04:47.856 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 21 02:04:47 smithi190 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mgr.z.service: Consumed 7.269s CPU time. 2024-09-21T02:04:47.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:47.870 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T02:04:47.870 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T02:04:47.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.0 2024-09-21T02:04:48.179 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:47 smithi029 systemd[1]: Stopping Ceph osd.0 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:48.595 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T02:04:48.178+0000 7f28aa543640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:04:48.595 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T02:04:48.178+0000 7f28aa543640 -1 osd.0 79 *** Got signal Terminated *** 2024-09-21T02:04:48.595 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0[44616]: 2024-09-21T02:04:48.178+0000 7f28aa543640 -1 osd.0 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:04:53.942 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:53 smithi029 podman[111095]: 2024-09-21 02:04:53.603903566 +0000 UTC m=+5.532178132 container died 3f345437ca97db003eaebbf1b0f31fdd3dfd1c6a677f0275139e47be0a2d4dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:04:54.236 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111095]: 2024-09-21 02:04:54.041600877 +0000 UTC m=+5.969875451 container cleanup 3f345437ca97db003eaebbf1b0f31fdd3dfd1c6a677f0275139e47be0a2d4dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:04:54.236 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 bash[111095]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0 2024-09-21T02:04:54.595 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111111]: 2024-09-21 02:04:54.2356057 +0000 UTC m=+0.627162997 container remove 3f345437ca97db003eaebbf1b0f31fdd3dfd1c6a677f0275139e47be0a2d4dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T02:04:54.856 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111198]: 2024-09-21 02:04:54.568841276 +0000 UTC m=+0.016243418 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:04:54.856 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111198]: 2024-09-21 02:04:54.713582727 +0000 UTC m=+0.160984856 container create dc82836dddc8f8e07fcb71e9f09082a1c656df31d6860f410eac3cf7437f6c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:04:55.182 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111198]: 2024-09-21 02:04:54.986544869 +0000 UTC m=+0.433947016 container init dc82836dddc8f8e07fcb71e9f09082a1c656df31d6860f410eac3cf7437f6c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-21T02:04:55.183 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:54 smithi029 podman[111198]: 2024-09-21 02:04:54.989997301 +0000 UTC m=+0.437399433 container start dc82836dddc8f8e07fcb71e9f09082a1c656df31d6860f410eac3cf7437f6c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T02:04:55.183 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:55 smithi029 podman[111198]: 2024-09-21 02:04:55.103638336 +0000 UTC m=+0.551040475 container attach dc82836dddc8f8e07fcb71e9f09082a1c656df31d6860f410eac3cf7437f6c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:04:55.464 INFO:journalctl@ceph.osd.0.smithi029.stdout:Sep 21 02:04:55 smithi029 podman[111198]: 2024-09-21 02:04:55.182759142 +0000 UTC m=+0.630161287 container died dc82836dddc8f8e07fcb71e9f09082a1c656df31d6860f410eac3cf7437f6c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:04:55.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T02:04:55.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:04:55.875 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T02:04:55.876 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T02:04:55.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.1 2024-09-21T02:04:56.299 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:04:55 smithi066 systemd[1]: Stopping Ceph osd.1 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:04:56.799 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:04:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T02:04:56.355+0000 7f044f27a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:04:56.799 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:04:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T02:04:56.355+0000 7f044f27a640 -1 osd.1 79 *** Got signal Terminated *** 2024-09-21T02:04:56.799 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:04:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1[38945]: 2024-09-21T02:04:56.355+0000 7f044f27a640 -1 osd.1 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:02.107 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:01 smithi066 podman[67930]: 2024-09-21 02:05:01.84150362 +0000 UTC m=+5.768855990 container died 75a6f39d9974cf835e7c6cb7ec68269d74cac9634ec4f2e8e70c1cdc282f3618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T02:05:02.549 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:02 smithi066 podman[67930]: 2024-09-21 02:05:02.339786208 +0000 UTC m=+6.267138616 container cleanup 75a6f39d9974cf835e7c6cb7ec68269d74cac9634ec4f2e8e70c1cdc282f3618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:05:02.549 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:02 smithi066 bash[67930]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1 2024-09-21T02:05:02.900 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:02 smithi066 podman[67959]: 2024-09-21 02:05:02.582695472 +0000 UTC m=+0.742986952 container remove 75a6f39d9974cf835e7c6cb7ec68269d74cac9634ec4f2e8e70c1cdc282f3618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T02:05:03.194 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:02 smithi066 podman[68046]: 2024-09-21 02:05:02.803518844 +0000 UTC m=+0.016348183 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:03.194 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:02 smithi066 podman[68046]: 2024-09-21 02:05:02.908460096 +0000 UTC m=+0.121289468 container create 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:05:03.194 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:03 smithi066 podman[68046]: 2024-09-21 02:05:03.110709261 +0000 UTC m=+0.323538591 container init 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:05:03.194 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:03 smithi066 podman[68046]: 2024-09-21 02:05:03.114130578 +0000 UTC m=+0.326959911 container start 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T02:05:03.194 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:03 smithi066 podman[68046]: 2024-09-21 02:05:03.193861557 +0000 UTC m=+0.406690898 container attach 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T02:05:03.549 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:03 smithi066 podman[68046]: 2024-09-21 02:05:03.308185347 +0000 UTC m=+0.521014692 container died 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T02:05:04.000 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 21 02:05:03 smithi066 podman[68070]: 2024-09-21 02:05:03.997645588 +0000 UTC m=+0.682519488 container remove 1a1b46f6b6b7e0d1f6f044ac79f64e1d2cb6a55357b060e9fa645b65f7a1396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T02:05:04.021 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T02:05:04.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:04.067 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T02:05:04.067 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T02:05:04.067 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.2 2024-09-21T02:05:04.376 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:04 smithi190 systemd[1]: Stopping Ceph osd.2 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:04.728 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T02:05:04.375+0000 7f2bad026640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:04.729 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T02:05:04.375+0000 7f2bad026640 -1 osd.2 79 *** Got signal Terminated *** 2024-09-21T02:05:04.729 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2[38801]: 2024-09-21T02:05:04.375+0000 7f2bad026640 -1 osd.2 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:10.210 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:09 smithi190 podman[66984]: 2024-09-21 02:05:09.956815169 +0000 UTC m=+5.686660121 container died 5a8955b003790245bbe0079dbdf4f6b8d9322579c7a6c2dffe9c9bab2f6f9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T02:05:10.478 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:10 smithi190 podman[66984]: 2024-09-21 02:05:10.275114679 +0000 UTC m=+6.004959620 container cleanup 5a8955b003790245bbe0079dbdf4f6b8d9322579c7a6c2dffe9c9bab2f6f9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T02:05:10.478 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:10 smithi190 bash[66984]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2 2024-09-21T02:05:10.882 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:10 smithi190 podman[66999]: 2024-09-21 02:05:10.484703608 +0000 UTC m=+0.523707126 container remove 5a8955b003790245bbe0079dbdf4f6b8d9322579c7a6c2dffe9c9bab2f6f9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T02:05:11.196 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:10 smithi190 podman[67089]: 2024-09-21 02:05:10.784462474 +0000 UTC m=+0.016014868 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:11.196 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:10 smithi190 podman[67089]: 2024-09-21 02:05:10.893917805 +0000 UTC m=+0.125470240 container create 4994d3efcb427930cf0ff53c5ddfd72af381469eb9d68c5f2bca14819b3a9a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T02:05:11.196 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:11 smithi190 podman[67089]: 2024-09-21 02:05:11.121974085 +0000 UTC m=+0.353526483 container init 4994d3efcb427930cf0ff53c5ddfd72af381469eb9d68c5f2bca14819b3a9a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T02:05:11.197 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:11 smithi190 podman[67089]: 2024-09-21 02:05:11.125777728 +0000 UTC m=+0.357330126 container start 4994d3efcb427930cf0ff53c5ddfd72af381469eb9d68c5f2bca14819b3a9a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T02:05:11.197 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:11 smithi190 podman[67089]: 2024-09-21 02:05:11.196274818 +0000 UTC m=+0.427827220 container attach 4994d3efcb427930cf0ff53c5ddfd72af381469eb9d68c5f2bca14819b3a9a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:05:11.478 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 21 02:05:11 smithi190 podman[67089]: 2024-09-21 02:05:11.311290756 +0000 UTC m=+0.542843146 container died 4994d3efcb427930cf0ff53c5ddfd72af381469eb9d68c5f2bca14819b3a9a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T02:05:11.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T02:05:12.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:12.017 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T02:05:12.017 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T02:05:12.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.3 2024-09-21T02:05:12.345 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:12 smithi029 systemd[1]: Stopping Ceph osd.3 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:12.845 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:12 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T02:05:12.350+0000 7feb14644640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:12.845 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:12 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T02:05:12.350+0000 7feb14644640 -1 osd.3 79 *** Got signal Terminated *** 2024-09-21T02:05:12.845 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:12 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3[50723]: 2024-09-21T02:05:12.350+0000 7feb14644640 -1 osd.3 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:16.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:16 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:16.524+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:17.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:16 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:16.607+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:17.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:17 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:17.515+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:17.978 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:17.881+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:17.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:17.652+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:17.985 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:17 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:17.656+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:17.985 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:17 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:17.677+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:18.281 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:17 smithi029 podman[111319]: 2024-09-21 02:05:17.985500938 +0000 UTC m=+5.756939514 container died cef33e83099c865191a28159d7f7ee44dd5937bf237932201b40a12377fe55a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T02:05:18.594 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:18 smithi029 podman[111319]: 2024-09-21 02:05:18.472473692 +0000 UTC m=+6.243912265 container cleanup cef33e83099c865191a28159d7f7ee44dd5937bf237932201b40a12377fe55a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:05:18.595 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:18 smithi029 bash[111319]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3 2024-09-21T02:05:18.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:18 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:18.501+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:18.978 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:18.909+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:18.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:18.683+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:19.094 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:18 smithi029 podman[111336]: 2024-09-21 02:05:18.727132467 +0000 UTC m=+0.737628605 container remove cef33e83099c865191a28159d7f7ee44dd5937bf237932201b40a12377fe55a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T02:05:19.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:18 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:18.684+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:19.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:18 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:18.619+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:19.425 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:18.99922995 +0000 UTC m=+0.016055443 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:19.425 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:19.120614303 +0000 UTC m=+0.137439826 container create 3a8264f11065dc5ac27a6185f0ba80aaa7e8c362fab9df03b6950e2b4c5da0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T02:05:19.425 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:19.332700766 +0000 UTC m=+0.349526255 container init 3a8264f11065dc5ac27a6185f0ba80aaa7e8c362fab9df03b6950e2b4c5da0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:05:19.425 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:19.336228996 +0000 UTC m=+0.353054478 container start 3a8264f11065dc5ac27a6185f0ba80aaa7e8c362fab9df03b6950e2b4c5da0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T02:05:19.425 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:19.424798806 +0000 UTC m=+0.441624306 container attach 3a8264f11065dc5ac27a6185f0ba80aaa7e8c362fab9df03b6950e2b4c5da0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:05:19.687 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:19 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:19.576+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:19.687 INFO:journalctl@ceph.osd.3.smithi029.stdout:Sep 21 02:05:19 smithi029 podman[111428]: 2024-09-21 02:05:19.532000619 +0000 UTC m=+0.548826116 container died 3a8264f11065dc5ac27a6185f0ba80aaa7e8c362fab9df03b6950e2b4c5da0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T02:05:19.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:19 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:19.534+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:19.978 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:19.949+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:19.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:19.704+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:20.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:19 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:19.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:20.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T02:05:20.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:20.169 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T02:05:20.170 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T02:05:20.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.4 2024-09-21T02:05:20.493 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:20 smithi066 systemd[1]: Stopping Ceph osd.4 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:20.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:20.562+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:20.799 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T02:05:20.492+0000 7f817ec22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:20.799 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T02:05:20.492+0000 7f817ec22640 -1 osd.4 79 *** Got signal Terminated *** 2024-09-21T02:05:20.800 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:20 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T02:05:20.492+0000 7f817ec22640 -1 osd.4 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:20.945 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:20.690+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:21.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:20 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:20.708+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:21.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:20 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:20.596+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:21.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:20.944+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:21.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:21 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:21.521+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:21.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:21 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:21.662+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:21.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:21 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:21.576+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:22.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:21.953+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:22.228 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:21.733+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:22.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:22 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:22.513+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:22.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:22 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:22.693+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:22.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:22 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:22.569+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:22.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:22 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:22.716+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:22.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:22 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:22.716+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:23.478 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:22 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:22.987+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:23.478 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:22 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:22.987+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:23.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:23 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:23.532+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:23.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:23 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:23.532+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:23.799 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:23 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:23.568+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:23.844 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:23 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:23.522+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:23.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:23 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:23.669+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:24.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:23 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:23.951+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:24.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:23 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:23.951+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:24.228 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:23 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:23.755+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:24.228 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:23 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:23.755+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:24.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:24 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:24.255+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:24.799 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:24 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:24.600+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:24.799 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:24 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4[45365]: 2024-09-21T02:05:24.610+0000 7f817b20d640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.755597+0000 front 2024-09-21T02:04:59.755625+0000 (oldest deadline 2024-09-21T02:05:24.455839+0000) 2024-09-21T02:05:24.800 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:24 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:24.535+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:24.800 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:24 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:24.535+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:24.844 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:24.717+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:24.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:24.717+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:24.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:24.502+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:24.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:24.502+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:25.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:24 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:24.972+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:25.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:24 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:24.972+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:25.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:24 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:24.756+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:25.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:24 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:24.756+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:25.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:25 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:25.289+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:25.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:25 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:25.683+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:25.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:25 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:25.683+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:25.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:25 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:25.486+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:25.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:25 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:25.486+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:25.963 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:25 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:25.617+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:25.963 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:25 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:25.570+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:25.963 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:25 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:25.570+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:26.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:25 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:25.936+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:26.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:25 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:25.936+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:26.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:25 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:25.734+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:26.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:25 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:25.734+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:26.299 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:25 smithi066 podman[68149]: 2024-09-21 02:05:25.963185073 +0000 UTC m=+5.581467967 container died f2626f2c1d51f5cd6f16905d7ce57055c661b3867880815e93565a30ffa43396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T02:05:26.614 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:26 smithi066 podman[68149]: 2024-09-21 02:05:26.416462609 +0000 UTC m=+6.034745502 container cleanup f2626f2c1d51f5cd6f16905d7ce57055c661b3867880815e93565a30ffa43396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T02:05:26.614 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:26 smithi066 bash[68149]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4 2024-09-21T02:05:26.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:26.317+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:26.844 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:26.521+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:26.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:26.521+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:26.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:26.670+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:26.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:26 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:26.670+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:27.049 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:26 smithi066 podman[68168]: 2024-09-21 02:05:26.726492152 +0000 UTC m=+0.758028113 container remove f2626f2c1d51f5cd6f16905d7ce57055c661b3867880815e93565a30ffa43396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T02:05:27.050 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:26 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:26.615+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:27.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:26 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:26.613+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:27.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:26 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:26.613+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:27.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:26.955+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:27.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:26.955+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:27.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:26.763+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:27.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:26 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:26.763+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:27.413 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:27.085682132 +0000 UTC m=+0.103400288 container create 46789a2e1c599f90566163c8bc4fe07c44287686ce772430e775b733629c8896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T02:05:27.413 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:26.998684403 +0000 UTC m=+0.016402527 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:27.413 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:27.329581104 +0000 UTC m=+0.347299224 container init 46789a2e1c599f90566163c8bc4fe07c44287686ce772430e775b733629c8896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T02:05:27.413 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:27.332880862 +0000 UTC m=+0.350598979 container start 46789a2e1c599f90566163c8bc4fe07c44287686ce772430e775b733629c8896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T02:05:27.413 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:27.412891918 +0000 UTC m=+0.430610035 container attach 46789a2e1c599f90566163c8bc4fe07c44287686ce772430e775b733629c8896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T02:05:27.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:27 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:27.325+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:27.799 INFO:journalctl@ceph.osd.4.smithi066.stdout:Sep 21 02:05:27 smithi066 podman[68252]: 2024-09-21 02:05:27.528987543 +0000 UTC m=+0.546705675 container died 46789a2e1c599f90566163c8bc4fe07c44287686ce772430e775b733629c8896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T02:05:27.799 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:27 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:27.592+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:27.800 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:27 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:27.600+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:27.800 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:27 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:27.600+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:27.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:27 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:27.710+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:27.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:27 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:27.710+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:27.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:27 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:27.534+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:27.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:27 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:27.534+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:28.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:27 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:27.990+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:28.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:27 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:27.990+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:28.228 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:27 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:27.777+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:28.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:27 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:27.777+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:28.268 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T02:05:28.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:28.312 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T02:05:28.312 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T02:05:28.312 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.5 2024-09-21T02:05:28.728 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 systemd[1]: Stopping Ceph osd.5 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:28.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:28.287+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:28.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:28.514+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:28.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:28.514+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:28.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:28.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:28.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:28.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:29.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:28 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:28.594+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:29.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:28 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:28.649+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:29.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:28 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:28.649+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:29.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:29 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:29.022+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:29.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:29 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:29.022+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:29.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:28.804+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:29.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:28.804+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:29.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:28.918+0000 7f875f2d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:29.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:28.918+0000 7f875f2d0640 -1 osd.5 79 *** Got signal Terminated *** 2024-09-21T02:05:29.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:28 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:28.918+0000 7f875f2d0640 -1 osd.5 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:29.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:29 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:29.312+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:29.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:29 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:29.552+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:29.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:29 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:29.552+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:29.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:29 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:29.715+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:29.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:29 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:29.715+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:30.020 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:29 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:29.766+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:30.020 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:29 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:29.766+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:30.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:29 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:29.562+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:30.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:29 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:29.693+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:30.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:29 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:29.693+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:30.347 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:30.020+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:30.347 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:30.020+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:30.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:30.346+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:30.799 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:30 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:30.536+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:30.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:30 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:30.713+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:30.799 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:30 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:30.713+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:30.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:30 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:30.539+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:30.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:30 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:30.539+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:30.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:30 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:30.726+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:30.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:30 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:30.726+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:30.998 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:30.738+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:30.998 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:30.738+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:31.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:30.998+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:31.344 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:30 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:30.998+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:31.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:31.343+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:31.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:31.580+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:31.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:31.580+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:31.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:31.712+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:31.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:31 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:31.712+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:32.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:31.743+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:32.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:31.743+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:32.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:31.552+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:32.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:31 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:31.552+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:32.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:31.997+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:32.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:31.997+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:32.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:31.997+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:32.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:31.783+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:32.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:31 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:31.783+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:32.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:32 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:32.309+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:32.844 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:32.542+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:32.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:32.542+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:32.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:32.702+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:32.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:32.702+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:33.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:32 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:32.570+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:33.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:32 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:32.570+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:33.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:32 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:32.744+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:33.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:32 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:32.744+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:33.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:33.036+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:33.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:33.036+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:33.229 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:33.036+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:33.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:32 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:32.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:33.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:32 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:32.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:33.229 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:32 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:32.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.531775+0000 front 2024-09-21T02:05:07.531741+0000 (oldest deadline 2024-09-21T02:05:32.232039+0000) 2024-09-21T02:05:33.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:33.287+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:33.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:33.287+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:33.844 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:33 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:33.592+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:33.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:33 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:33.592+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:33.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:33 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:33.743+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:33.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:33 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:33.743+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:33.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:33 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:33.743+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:34.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:33 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:33.570+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:34.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:33 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:33.570+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:34.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:33 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:33.778+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:34.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:33 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:33.778+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:34.088 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:33.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:51.527056+0000 front 2024-09-21T02:04:51.527099+0000 (oldest deadline 2024-09-21T02:05:16.827025+0000) 2024-09-21T02:05:34.088 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:33.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.128113+0000 front 2024-09-21T02:04:59.128242+0000 (oldest deadline 2024-09-21T02:05:22.528773+0000) 2024-09-21T02:05:34.088 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:33 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5[44921]: 2024-09-21T02:05:33.798+0000 7f87578b3640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.531775+0000 front 2024-09-21T02:05:07.531741+0000 (oldest deadline 2024-09-21T02:05:32.232039+0000) 2024-09-21T02:05:34.395 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:34 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:34.310+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:34.396 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:34 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:34.310+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:34.396 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:34 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:34.086+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:34.396 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:34 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:34.086+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:34.396 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:34 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:34.086+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:34.653 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:34 smithi190 podman[67192]: 2024-09-21 02:05:34.39542014 +0000 UTC m=+5.882629485 container died 2dc86988c2979354edc7f5a0b4e9f1e6d0c39a75c278b2551cd0347d26f39ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T02:05:34.922 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:34 smithi190 podman[67192]: 2024-09-21 02:05:34.73575142 +0000 UTC m=+6.222960798 container cleanup 2dc86988c2979354edc7f5a0b4e9f1e6d0c39a75c278b2551cd0347d26f39ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T02:05:34.922 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:34 smithi190 bash[67192]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5 2024-09-21T02:05:35.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:34 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:34.553+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:35.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:34 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:34.553+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:35.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:34 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:34.791+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:35.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:34 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:34.791+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:35.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:34 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:34.791+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:34.633+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:34.633+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:34.633+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:34.783+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:34.783+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:35.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:34 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:34.783+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:35.228 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:34 smithi190 podman[67210]: 2024-09-21 02:05:34.920123572 +0000 UTC m=+0.514326188 container remove 2dc86988c2979354edc7f5a0b4e9f1e6d0c39a75c278b2551cd0347d26f39ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:05:35.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:35 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:35.094+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:35.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:35 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:35.094+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:35.228 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:35 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:35.094+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:35 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:35.331+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:35 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:35.331+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.329272536 +0000 UTC m=+0.095878471 container create 2a2a35ac94939bb67fdd7affe217391b6a94e48d02bb59914a87a18ae38fa5e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.249122169 +0000 UTC m=+0.015728104 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.548285251 +0000 UTC m=+0.314891185 container init 2a2a35ac94939bb67fdd7affe217391b6a94e48d02bb59914a87a18ae38fa5e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:05:35.623 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.551525088 +0000 UTC m=+0.318131031 container start 2a2a35ac94939bb67fdd7affe217391b6a94e48d02bb59914a87a18ae38fa5e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:05:35.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.622995055 +0000 UTC m=+0.389600992 container attach 2a2a35ac94939bb67fdd7affe217391b6a94e48d02bb59914a87a18ae38fa5e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T02:05:35.978 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 21 02:05:35 smithi190 podman[67299]: 2024-09-21 02:05:35.7414464 +0000 UTC m=+0.508052335 container died 2a2a35ac94939bb67fdd7affe217391b6a94e48d02bb59914a87a18ae38fa5e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T02:05:36.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:35.586+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:36.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:35.586+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:36.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:35.784+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:36.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:35.784+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:36.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:35 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:35.784+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:36.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:35.623+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:36.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:35.623+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:36.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:35.623+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:36.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:35.782+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:36.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:35.782+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:36.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:35 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:35.782+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:36.293 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:36 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:36.115+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:36.293 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:36 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:36.115+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:36.293 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:36 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:36.115+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:36.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T02:05:36.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:36.359 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T02:05:36.359 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T02:05:36.360 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.6 2024-09-21T02:05:36.665 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 systemd[1]: Stopping Ceph osd.6 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:36.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:36 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:36.341+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:36.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:36 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:36.341+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:36.933 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:36.753+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:36.933 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:36.753+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:36.933 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:36.753+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:36.933 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.664+0000 7f464aff8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:36.933 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.664+0000 7f464aff8640 -1 osd.6 79 *** Got signal Terminated *** 2024-09-21T02:05:36.934 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.664+0000 7f464aff8640 -1 osd.6 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:36.934 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.665+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:36.934 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.665+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:36.934 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:36.665+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:37.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:36 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:36.627+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:37.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:36 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:36.627+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:37.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:36 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:36.775+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:37.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:36 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:36.775+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:37.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:36 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:36.775+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:37.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:37 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:37.386+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:37.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:37 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:37.386+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:37.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:37 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:37.154+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:37.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:37 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:37.154+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:37.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:37 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:37.154+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:38.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:37 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:37.675+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:38.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:37 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:37.675+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:38.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:37 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:37.764+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:38.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:37 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:37.764+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:38.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:37 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:37.764+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:37.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:37.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:37.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:37.712+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:37.712+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:38.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:37 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:37.712+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:38.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:38 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:38.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:38.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:38 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:38.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:38.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:38 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:38.150+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:38.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:38 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:38.150+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:38.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:38 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:38.150+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:39.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:38.643+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:39.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:38.643+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:39.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:38.729+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:39.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:38.729+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:39.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:38 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:38.729+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:38.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:38.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:38.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:38.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:38.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:38.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:39.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:38 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:38.709+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:14.137203+0000 front 2024-09-21T02:05:14.137182+0000 (oldest deadline 2024-09-21T02:05:38.237406+0000) 2024-09-21T02:05:39.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:39 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:39.411+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:39.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:39 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:39.411+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:39.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:39 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:39.172+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:39.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:39 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:39.172+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:39.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:39 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:39.172+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:40.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:39 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:39.641+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:40.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:39 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:39.641+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:40.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:39 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:39.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:40.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:39 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:39.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:40.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:39 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:39.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:39.712+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:39.712+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:39.712+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:39.712+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:14.137203+0000 front 2024-09-21T02:05:14.137182+0000 (oldest deadline 2024-09-21T02:05:38.237406+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:39.691+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:39.691+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:39.691+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:40.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:39 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:39.691+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:40.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:40 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:40.396+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:40.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:40 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:40.396+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:40.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:40 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:40.149+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:40.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:40 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:40.149+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:40.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:40 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:40.149+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:41.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:40.623+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:41.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:40.623+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:41.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:40.723+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:41.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:40.723+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:41.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:40.723+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:41.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:40.723+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:41.094 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:40.708+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.832230+0000 front 2024-09-21T02:04:52.832546+0000 (oldest deadline 2024-09-21T02:05:15.732359+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:40.708+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:59.034398+0000 front 2024-09-21T02:04:59.034621+0000 (oldest deadline 2024-09-21T02:05:23.734535+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:40.708+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:07.736000+0000 front 2024-09-21T02:05:07.735894+0000 (oldest deadline 2024-09-21T02:05:33.635830+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6[56871]: 2024-09-21T02:05:40.708+0000 7f46435db640 -1 osd.6 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:14.137203+0000 front 2024-09-21T02:05:14.137182+0000 (oldest deadline 2024-09-21T02:05:38.237406+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:40.662+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:40.662+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:40.662+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:41.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:40 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:40.663+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:41.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:41.137+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:41.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:41.137+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:41.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:41.137+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:41.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:41.429+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:41.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:41.429+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:41.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:41 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:41.429+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:42.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:41.632+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:42.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:41.632+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:42.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:41.725+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:42.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:41.725+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:42.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:41.725+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:42.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:41 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:41.725+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:42.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:41 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:41.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:42.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:41 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:41.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:42.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:41 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:41.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:42.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:41 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:41.686+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:42.397+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:42.397+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:42.397+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:42.168+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:42.168+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:42.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:42 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:42.168+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:42.584 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:42 smithi029 podman[111532]: 2024-09-21 02:05:42.27092625 +0000 UTC m=+5.700146449 container died 3167c39b81dc6a6eb2a9cb89b09c09bcc1abcbf9dc64247867eb1af34cd46239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:42 smithi029 podman[111532]: 2024-09-21 02:05:42.783972151 +0000 UTC m=+6.213192339 container cleanup 3167c39b81dc6a6eb2a9cb89b09c09bcc1abcbf9dc64247867eb1af34cd46239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:42 smithi029 bash[111532]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:42 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:42.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:42 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:42.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:42 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:42.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:42.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:42 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:42.724+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:43.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:42.651+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:43.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:42.651+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:43.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:42.651+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:43.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:42.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:43.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:42.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:43.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:42.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:43.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:42 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:42.736+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:43.344 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111561]: 2024-09-21 02:05:43.055108843 +0000 UTC m=+0.780586019 container remove 3167c39b81dc6a6eb2a9cb89b09c09bcc1abcbf9dc64247867eb1af34cd46239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T02:05:43.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:43.356+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:43.478 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:43.356+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:43.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:43.356+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:43.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:43.210+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:43.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:43.210+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:43.479 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:43 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:43.210+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:43.727 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.368380418 +0000 UTC m=+0.016387502 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:43.728 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.490176571 +0000 UTC m=+0.138183629 container create 70f250c86b390a2584d6a6f1a3ac8c1a8804622c098149a6523ce75baecc8678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T02:05:43.728 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:43 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:43.684+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:43.728 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:43 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:43.684+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:43.728 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:43 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:43.684+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:43.728 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:43 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:43.684+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:44.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:43.680+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:44.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:43.680+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:44.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:43.680+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:44.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:43.735+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:44.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:43.735+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:44.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:43.735+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:44.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:43 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:43.735+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:44.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.727607558 +0000 UTC m=+0.375614630 container init 70f250c86b390a2584d6a6f1a3ac8c1a8804622c098149a6523ce75baecc8678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T02:05:44.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.732105616 +0000 UTC m=+0.380112675 container start 70f250c86b390a2584d6a6f1a3ac8c1a8804622c098149a6523ce75baecc8678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:05:44.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.819812815 +0000 UTC m=+0.467819887 container attach 70f250c86b390a2584d6a6f1a3ac8c1a8804622c098149a6523ce75baecc8678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T02:05:44.095 INFO:journalctl@ceph.osd.6.smithi029.stdout:Sep 21 02:05:43 smithi029 podman[111653]: 2024-09-21 02:05:43.924882162 +0000 UTC m=+0.572889231 container died 70f250c86b390a2584d6a6f1a3ac8c1a8804622c098149a6523ce75baecc8678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T02:05:44.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T02:05:44.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:44.643 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T02:05:44.643 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T02:05:44.643 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.7 2024-09-21T02:05:44.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:44.379+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:44.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:44.379+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:44.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:44.379+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:44.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:44.237+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:44.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:44.237+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:44.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:44 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:44.237+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:44.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:44 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:44.663+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:44.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:44 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:44.663+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:44.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:44 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:44.663+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:44.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:44 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:44.663+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:44.946 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:44.720+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:44.946 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:44.720+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:44.946 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:44.720+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:44.946 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:44.720+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:44.946 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.687+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.687+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.687+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 systemd[1]: Stopping Ceph osd.7 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.944+0000 7f924f8df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.944+0000 7f924f8df640 -1 osd.7 79 *** Got signal Terminated *** 2024-09-21T02:05:44.947 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:44.944+0000 7f924f8df640 -1 osd.7 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:45.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:45.353+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:45.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:45.353+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:45.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:45.353+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:45.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:45.266+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:45.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:45.266+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:45.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:45 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:45.266+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:46.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:45.725+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:46.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:45.725+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:46.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:45.725+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:46.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:45.759+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:46.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:45.759+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:46.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:45.759+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:46.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:45 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:45.759+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:46.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:45.700+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:46.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:45.700+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:46.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:45.700+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:46.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:45 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:45.700+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:46.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:46.308+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:46.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:46.308+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:46.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:46.308+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:46.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:46.303+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:46.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:46.303+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:46.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:46 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:46.303+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:47.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:46.682+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:47.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:46.682+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:46.682+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:46.728+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:46.728+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:46.728+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:46.728+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:47.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:46 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:46.728+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:47.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:46 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:46.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:47.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:46 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:46.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:47.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:46 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:46.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:47.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:46 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:46.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:47.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:47.304+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:47.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:47.304+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:47.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:47.304+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:47.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:47.281+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:47.730 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:47.281+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:47.730 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:47 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:47.281+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:48.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:47.668+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:48.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:47.668+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:48.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:47.668+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:48.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:47.668+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.595830+0000 front 2024-09-21T02:05:23.595865+0000 (oldest deadline 2024-09-21T02:05:47.096010+0000) 2024-09-21T02:05:48.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:47.777+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:48.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:47.777+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:48.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:47.777+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:48.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:47.777+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:48.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:47 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:47.777+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:48.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:47 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:47.731+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:48.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:47 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:47.731+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:48.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:47 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:47.731+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:48.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:47 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:47.731+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:48.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:48.235+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:48.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:48.235+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:48.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:48.235+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:48.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:48.348+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:48.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:48.348+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:48.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:48.348+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:48.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:48 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:48.348+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:49.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:48.684+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:49.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:48.684+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:48.684+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:48.684+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.595830+0000 front 2024-09-21T02:05:23.595865+0000 (oldest deadline 2024-09-21T02:05:47.096010+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:48.788+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:48.788+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:48.788+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:48.788+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:49.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:48.788+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:49.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:48.740+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:49.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:48.740+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:49.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:48.740+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:49.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:48 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:48.740+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:49.478 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:49.219+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:49.690 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:49.219+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:49.690 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:49.219+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:49.691 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:49.390+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:49.691 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:49.390+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:49.691 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:49.390+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:49.691 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:49 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:49.390+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:50.051 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:49.705+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.292395+0000 front 2024-09-21T02:04:57.292200+0000 (oldest deadline 2024-09-21T02:05:23.192340+0000) 2024-09-21T02:05:50.051 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:49.705+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:06.693568+0000 front 2024-09-21T02:05:06.693615+0000 (oldest deadline 2024-09-21T02:05:31.393350+0000) 2024-09-21T02:05:50.051 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:49.705+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:17.294331+0000 front 2024-09-21T02:05:17.294358+0000 (oldest deadline 2024-09-21T02:05:42.594463+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7[52012]: 2024-09-21T02:05:49.705+0000 7f9247ec2640 -1 osd.7 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.595830+0000 front 2024-09-21T02:05:23.595865+0000 (oldest deadline 2024-09-21T02:05:47.096010+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:49.825+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:49.825+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:49.825+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:49.825+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:50.052 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:49 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:49.825+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:50.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:49 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:49.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:50.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:49 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:49.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:50.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:49 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:49.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:50.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:49 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:49.748+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:50.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:50.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:50.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:50.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:50.362+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:50.260+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:50.260+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:50.260+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:50.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:50 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:50.260+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:50.736 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:50 smithi066 podman[68354]: 2024-09-21 02:05:50.475552752 +0000 UTC m=+5.634402454 container died e8acaea2482ed5f684ce1a15c77550af7814c18571d456e077936d5c6a4b2d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T02:05:51.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:50 smithi066 podman[68354]: 2024-09-21 02:05:50.835304637 +0000 UTC m=+5.994154338 container cleanup e8acaea2482ed5f684ce1a15c77550af7814c18571d456e077936d5c6a4b2d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:05:51.049 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:50 smithi066 bash[68354]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7 2024-09-21T02:05:51.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:50.803+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:51.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:50.803+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:51.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:50.803+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:51.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:50.803+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:51.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:50 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:50.803+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:51.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:50 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:50.770+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:51.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:50 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:50.770+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:51.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:50 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:50.770+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:51.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:50 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:50.770+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:51.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:50 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:50.770+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:51.457 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68368]: 2024-09-21 02:05:51.095060163 +0000 UTC m=+0.621683824 container remove e8acaea2482ed5f684ce1a15c77550af7814c18571d456e077936d5c6a4b2d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T02:05:51.721 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68458]: 2024-09-21 02:05:51.359802897 +0000 UTC m=+0.016462630 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:51.722 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68458]: 2024-09-21 02:05:51.487481476 +0000 UTC m=+0.144141202 container create 6786cef046c80ae510c0afb20cbad99c6c4bf5f356018e20f45d7b420e11d225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:51.325+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:51.325+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:51.325+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:51.325+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:51.300+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:51.300+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:51.300+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:51.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:51 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:51.300+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:52.047 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68458]: 2024-09-21 02:05:51.848465481 +0000 UTC m=+0.505125216 container init 6786cef046c80ae510c0afb20cbad99c6c4bf5f356018e20f45d7b420e11d225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:05:52.047 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68458]: 2024-09-21 02:05:51.852395015 +0000 UTC m=+0.509054747 container start 6786cef046c80ae510c0afb20cbad99c6c4bf5f356018e20f45d7b420e11d225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:51 smithi066 podman[68458]: 2024-09-21 02:05:51.923335024 +0000 UTC m=+0.579994758 container attach 6786cef046c80ae510c0afb20cbad99c6c4bf5f356018e20f45d7b420e11d225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:51.800+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:51.800+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:51.800+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:51.800+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:52.048 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:51 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:51.800+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:52.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:51 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:51.817+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:52.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:51 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:51.817+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:52.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:51 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:51.817+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:52.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:51 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:51.817+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:52.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:51 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:51.817+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:52.299 INFO:journalctl@ceph.osd.7.smithi066.stdout:Sep 21 02:05:52 smithi066 podman[68458]: 2024-09-21 02:05:52.047278288 +0000 UTC m=+0.703938029 container died 6786cef046c80ae510c0afb20cbad99c6c4bf5f356018e20f45d7b420e11d225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:05:52.599 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T02:05:52.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:05:52.640 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T02:05:52.640 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T02:05:52.640 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.8 2024-09-21T02:05:52.666 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:52.302+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:52.666 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:52.302+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:52.666 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:52.302+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:52.667 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:52.302+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:52.667 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.321+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:52.667 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.321+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:52.667 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.321+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:52.667 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.321+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:52.945 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 systemd[1]: Stopping Ceph osd.8 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:05:53.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:52.824+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:53.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:52.824+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:53.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:52.824+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:53.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:52.824+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:53.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:52.824+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:53.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:52.797+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:53.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:52.797+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:53.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:52.797+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:53.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:52.797+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:53.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:52 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:52.797+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:53.228 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.944+0000 7f58e1f1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:05:53.228 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.944+0000 7f58e1f1d640 -1 osd.8 79 *** Got signal Terminated *** 2024-09-21T02:05:53.228 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:52 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:52.944+0000 7f58e1f1d640 -1 osd.8 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:05:53.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:53.273+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:53.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:53.273+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:53.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:53.273+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:53.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:53.273+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:53.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:53.328+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:53.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:53.328+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:53.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:53.328+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:53.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:53 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:53.328+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:54.094 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:53 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:53.795+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:54.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:53 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:53.795+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:54.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:53 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:53.795+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:54.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:53 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:53.795+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:54.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:53 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:53.795+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:54.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:53 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:53.844+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:54.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:53 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:53.844+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:54.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:53 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:53.844+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:54.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:53 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:53.844+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:54.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:53 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:53.844+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:54.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:54.230+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:54.230+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:54.230+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:54.230+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:54.378+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:54.378+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:54.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:54.378+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:54.730 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:54 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:54.378+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:55.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:54 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:54.832+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:05:55.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:54 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:54.874+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:55.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:54 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:54.874+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:55.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:54 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:54.874+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:55.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:54 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:54.874+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:55.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:54 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:54.874+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:55.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:55.396+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:55.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:55.396+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:55.396+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:55.396+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:55.232+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:55.232+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:55.232+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:55.232+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:55.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:55 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:55.232+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.038738+0000 front 2024-09-21T02:05:33.038620+0000 (oldest deadline 2024-09-21T02:05:54.738680+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:56.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:55 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:55.789+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:56.201 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:55 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:55.903+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:05:56.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:56.269+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:56.728 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:56.269+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:56.269+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:56.269+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:56.269+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.038738+0000 front 2024-09-21T02:05:33.038620+0000 (oldest deadline 2024-09-21T02:05:54.738680+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:56.406+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:56.406+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:56.406+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:56.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:56.406+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:57.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:56 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:56.833+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:05:57.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:57.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:57.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:57.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:57.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:57.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:56 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:56.884+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:05:57.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:57.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:57.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:57.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:57.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:57.276+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.231679+0000 front 2024-09-21T02:05:00.231595+0000 (oldest deadline 2024-09-21T02:05:23.731588+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:57.276+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.333111+0000 front 2024-09-21T02:05:08.333175+0000 (oldest deadline 2024-09-21T02:05:32.934022+0000) 2024-09-21T02:05:57.729 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:57.276+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:15.235193+0000 front 2024-09-21T02:05:15.235318+0000 (oldest deadline 2024-09-21T02:05:40.535083+0000) 2024-09-21T02:05:57.730 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:57.276+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.336544+0000 front 2024-09-21T02:05:23.336404+0000 (oldest deadline 2024-09-21T02:05:47.936862+0000) 2024-09-21T02:05:57.730 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:57 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8[51296]: 2024-09-21T02:05:57.276+0000 7f58da500640 -1 osd.8 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.038738+0000 front 2024-09-21T02:05:33.038620+0000 (oldest deadline 2024-09-21T02:05:54.738680+0000) 2024-09-21T02:05:58.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:58.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:58.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:58.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:58.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:58.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:57 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:57.907+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:58.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:57 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:57.865+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:05:58.678 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:58 smithi190 podman[67404]: 2024-09-21 02:05:58.351737934 +0000 UTC m=+5.519064153 container died b0fb9bac760a1fb7a23908457fb00a53299b0641acac110086d654aaa34625a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:05:58.678 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:58 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:58.399+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:58.678 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:58 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:58.399+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:58.679 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:58 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:58.399+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:58.679 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:58 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:58.399+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:58.978 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:58 smithi190 podman[67404]: 2024-09-21 02:05:58.677566946 +0000 UTC m=+5.844893163 container cleanup b0fb9bac760a1fb7a23908457fb00a53299b0641acac110086d654aaa34625a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T02:05:58.978 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:58 smithi190 bash[67404]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8 2024-09-21T02:05:58.978 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:58 smithi190 podman[67419]: 2024-09-21 02:05:58.837050871 +0000 UTC m=+0.483737349 container remove b0fb9bac760a1fb7a23908457fb00a53299b0641acac110086d654aaa34625a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:05:59.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:05:59.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:05:59.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:05:59.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:05:59.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:05:59.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:58 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:58.870+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:05:59.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:05:59.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:05:59.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:05:59.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:05:59.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:05:59.346 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:58 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:58.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:59 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:59.384+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:59 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:59.384+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:59 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:59.384+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:59 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:59.384+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:05:59 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:05:59.384+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.141994106 +0000 UTC m=+0.015720563 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:05:59.557 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.254557098 +0000 UTC m=+0.128283543 container create 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:05:59.558 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.473567878 +0000 UTC m=+0.347294323 container init 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T02:05:59.558 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.476785928 +0000 UTC m=+0.350512382 container start 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T02:05:59.558 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.556829283 +0000 UTC m=+0.430555741 container attach 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T02:05:59.978 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:05:59 smithi190 podman[67510]: 2024-09-21 02:05:59.668135085 +0000 UTC m=+0.541861531 container died 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T02:06:00.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:00.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:00.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:00.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:00.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:00.096 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:05:59 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:05:59.834+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:00.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:05:59 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:05:59.863+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:00.361 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 21 02:06:00 smithi190 podman[67533]: 2024-09-21 02:06:00.34314588 +0000 UTC m=+0.669275740 container remove 4299dbe3db6a08c20555117dc2677af0ea2a52736f78b1622930b0110c24584e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T02:06:00.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T02:06:00.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:00.449 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T02:06:00.449 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T02:06:00.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.9 2024-09-21T02:06:00.621 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:00.410+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:00.622 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:00.410+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:00.622 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:00.410+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:00.622 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:00.410+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:00.622 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:00.410+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:00.768 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 systemd[1]: Stopping Ceph osd.9 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:00.768 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.766+0000 7f2dbe7c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:00.768 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.766+0000 7f2dbe7c2640 -1 osd.9 79 *** Got signal Terminated *** 2024-09-21T02:06:00.768 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.766+0000 7f2dbe7c2640 -1 osd.9 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:01.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:00 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:00.846+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:01.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:00 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:00.836+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:01.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:01 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:01.392+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:01.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:01 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:01.392+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:01.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:01 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:01.392+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:01.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:01 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:01.392+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:01.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:01 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:01.392+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:02.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:02.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:02.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:02.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:02.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:02.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:01 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:01.833+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:02.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:02.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:02.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:02.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:02.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:02.346 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:01 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:01.863+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:02.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:02 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:02.402+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:02.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:02 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:02.402+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:02.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:02 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:02.402+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:02.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:02 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:02.402+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:02.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:02 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:02.402+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:03.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:02 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:02.842+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:03.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:03.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:03.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:03.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:03.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:03.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:02 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:02.821+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:03.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:03.389+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:03.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:03.389+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:03.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:03.389+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:03.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:03.389+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:03.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:03 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:03.389+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:04.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:04.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:04.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:04.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:04.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:04.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:03 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:03.835+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:04.345 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:03 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:03.856+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:04.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:04.413+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:04.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:04.413+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:04.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:04.413+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:04.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:04.413+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:04.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:04.413+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:49.810419+0000 front 2024-09-21T02:04:49.810669+0000 (oldest deadline 2024-09-21T02:05:17.411369+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:05:00.813353+0000 front 2024-09-21T02:05:00.813691+0000 (oldest deadline 2024-09-21T02:05:23.713239+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:03.713767+0000 front 2024-09-21T02:05:03.713599+0000 (oldest deadline 2024-09-21T02:05:33.114314+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:13.114792+0000 front 2024-09-21T02:05:13.114758+0000 (oldest deadline 2024-09-21T02:05:39.014883+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:25.317269+0000 front 2024-09-21T02:05:25.317179+0000 (oldest deadline 2024-09-21T02:05:50.017017+0000) 2024-09-21T02:06:05.095 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:04 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9[63532]: 2024-09-21T02:06:04.836+0000 7f2dbadad640 -1 osd.9 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.418602+0000 front 2024-09-21T02:05:33.418803+0000 (oldest deadline 2024-09-21T02:05:54.518473+0000) 2024-09-21T02:06:05.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:05.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:05.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:05.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:05.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:05.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:05.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:04 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:04.830+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:05.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:05.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:05.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:05.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:05.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:05.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:05 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:05.374+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:06.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:06.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:06.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:06.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:06.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:06.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:06.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:05 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:05.868+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:06.578 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:06 smithi029 podman[111757]: 2024-09-21 02:06:06.307149046 +0000 UTC m=+5.654869640 container died 7401c93c67170e969bf739f1a4d31ac6f6faa81f9fbb5635ee87a5795f9de1d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:06.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:06.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:06.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:06.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:06.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:06.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:06 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:06.379+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:06.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:06 smithi029 podman[111757]: 2024-09-21 02:06:06.68545707 +0000 UTC m=+6.033177653 container cleanup 7401c93c67170e969bf739f1a4d31ac6f6faa81f9fbb5635ee87a5795f9de1d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T02:06:06.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:06 smithi029 bash[111757]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9 2024-09-21T02:06:07.273 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:06 smithi029 podman[111773]: 2024-09-21 02:06:06.956073342 +0000 UTC m=+0.642563050 container remove 7401c93c67170e969bf739f1a4d31ac6f6faa81f9fbb5635ee87a5795f9de1d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:07.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:06 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:06.876+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:07.528 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.175918928 +0000 UTC m=+0.016119253 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:06:07.528 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.299340635 +0000 UTC m=+0.139540994 container create b488636b98bed73ca2cf46f2746eb113d296376d38cd7e236d18699802b915ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T02:06:07.528 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.528415345 +0000 UTC m=+0.368615671 container init b488636b98bed73ca2cf46f2746eb113d296376d38cd7e236d18699802b915ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T02:06:07.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:07.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:07.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:07.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:07.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:07.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:07 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:07.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:07.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.53202786 +0000 UTC m=+0.372228177 container start b488636b98bed73ca2cf46f2746eb113d296376d38cd7e236d18699802b915ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T02:06:07.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.620444864 +0000 UTC m=+0.460645193 container attach b488636b98bed73ca2cf46f2746eb113d296376d38cd7e236d18699802b915ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-21T02:06:07.845 INFO:journalctl@ceph.osd.9.smithi029.stdout:Sep 21 02:06:07 smithi029 podman[111862]: 2024-09-21 02:06:07.7240357 +0000 UTC m=+0.564236024 container died b488636b98bed73ca2cf46f2746eb113d296376d38cd7e236d18699802b915ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T02:06:08.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:08.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:08.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:08.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:08.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:08.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:08.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:07 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:07.899+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:08.383 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T02:06:08.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:08.424 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T02:06:08.425 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T02:06:08.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.10 2024-09-21T02:06:08.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:08.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:08.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:08.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:08.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:08.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:08.359+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:08.738 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 systemd[1]: Stopping Ceph osd.10 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:09.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.737+0000 7f356d07c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:09.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.737+0000 7f356d07c640 -1 osd.10 79 *** Got signal Terminated *** 2024-09-21T02:06:09.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.737+0000 7f356d07c640 -1 osd.10 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:09.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:08 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:08.933+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:09.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:09.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:09.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:09.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:09.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:09.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:09 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:09.352+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:10.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:10.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:10.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:10.301 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:10.301 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:10.301 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:10.301 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:09 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:09.977+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:10.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:10.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:10.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:10.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:10.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:10.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:10 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:10.358+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:11.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:11.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:11.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:11.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:11.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:11.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:11.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:10 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:10.958+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:11.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:11.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:11.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:11.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:11.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:11.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:11.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:11 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:11.334+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:12.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:12.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:12.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:12.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:12.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:12.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:12.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:11 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:11.975+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:12.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:12.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:12 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:12.361+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:52.128119+0000 front 2024-09-21T02:04:52.128043+0000 (oldest deadline 2024-09-21T02:05:16.228172+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:57.928947+0000 front 2024-09-21T02:04:57.928966+0000 (oldest deadline 2024-09-21T02:05:23.229014+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.530146+0000 front 2024-09-21T02:05:08.530035+0000 (oldest deadline 2024-09-21T02:05:33.830126+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6814 osd.3 since back 2024-09-21T02:05:16.731197+0000 front 2024-09-21T02:05:16.731223+0000 (oldest deadline 2024-09-21T02:05:40.230998+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:21.931859+0000 front 2024-09-21T02:05:21.931885+0000 (oldest deadline 2024-09-21T02:05:46.631967+0000) 2024-09-21T02:06:13.299 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:33.534293+0000 front 2024-09-21T02:05:33.534198+0000 (oldest deadline 2024-09-21T02:05:55.134511+0000) 2024-09-21T02:06:13.300 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:12 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10[58280]: 2024-09-21T02:06:12.981+0000 7f3569667640 -1 osd.10 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:39.736208+0000 front 2024-09-21T02:05:39.736108+0000 (oldest deadline 2024-09-21T02:06:04.336687+0000) 2024-09-21T02:06:13.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:13.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:13.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:13.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:13.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:13.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:13.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:13 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:13.327+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:14.549 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:14 smithi066 podman[68573]: 2024-09-21 02:06:14.252783729 +0000 UTC m=+5.625636045 container died 67ae749cee583b87e758cb9538960e276682674d6d2e37148316ede2aea957ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:14.691 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:14 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:14.354+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:14.863 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:14 smithi066 podman[68573]: 2024-09-21 02:06:14.670335696 +0000 UTC m=+6.043188011 container cleanup 67ae749cee583b87e758cb9538960e276682674d6d2e37148316ede2aea957ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:06:14.863 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:14 smithi066 bash[68573]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10 2024-09-21T02:06:15.297 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:14 smithi066 podman[68588]: 2024-09-21 02:06:14.863373773 +0000 UTC m=+0.608117677 container remove 67ae749cee583b87e758cb9538960e276682674d6d2e37148316ede2aea957ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:06:15.549 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.200537013 +0000 UTC m=+0.016993733 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:06:15.549 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.305560713 +0000 UTC m=+0.122017415 container create fdbc7522aa474f3722c01e867e45a778ebcb90abd3bb353695c2b67a05b2cf4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T02:06:15.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:15.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:15 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:15.401+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:16.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.550185498 +0000 UTC m=+0.366642210 container init fdbc7522aa474f3722c01e867e45a778ebcb90abd3bb353695c2b67a05b2cf4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T02:06:16.049 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.553760677 +0000 UTC m=+0.370217381 container start fdbc7522aa474f3722c01e867e45a778ebcb90abd3bb353695c2b67a05b2cf4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T02:06:16.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.616620989 +0000 UTC m=+0.433077707 container attach fdbc7522aa474f3722c01e867e45a778ebcb90abd3bb353695c2b67a05b2cf4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-21T02:06:16.050 INFO:journalctl@ceph.osd.10.smithi066.stdout:Sep 21 02:06:15 smithi066 podman[68674]: 2024-09-21 02:06:15.748048682 +0000 UTC m=+0.564505386 container died fdbc7522aa474f3722c01e867e45a778ebcb90abd3bb353695c2b67a05b2cf4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:16.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T02:06:16.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:16.430 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T02:06:16.430 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T02:06:16.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.11 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:16.721 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.388+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:16.722 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 systemd[1]: Stopping Ceph osd.11 for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:16.722 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.719+0000 7f21db8ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:16.722 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.719+0000 7f21db8ec640 -1 osd.11 79 *** Got signal Terminated *** 2024-09-21T02:06:16.722 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:16 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:16.719+0000 7f21db8ec640 -1 osd.11 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:06:17.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:17.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:17 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:17.369+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:18.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:18.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:18.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:18.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:18.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:18.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:18.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:18 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:18.385+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:19.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:19.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:19.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:19.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:19.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:19.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:19.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:19 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:19.362+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:20.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:20.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:20.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:20.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:20.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:20.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:20.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:20 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:20.372+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:21.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6806 osd.0 since back 2024-09-21T02:04:53.036775+0000 front 2024-09-21T02:04:53.036650+0000 (oldest deadline 2024-09-21T02:05:17.136825+0000) 2024-09-21T02:06:21.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-09-21T02:04:58.238034+0000 front 2024-09-21T02:04:58.237990+0000 (oldest deadline 2024-09-21T02:05:22.937953+0000) 2024-09-21T02:06:21.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-21T02:05:08.838836+0000 front 2024-09-21T02:05:08.838730+0000 (oldest deadline 2024-09-21T02:05:31.639224+0000) 2024-09-21T02:06:21.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-09-21T02:05:23.841510+0000 front 2024-09-21T02:05:23.841495+0000 (oldest deadline 2024-09-21T02:05:49.741435+0000) 2024-09-21T02:06:21.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-21T02:05:31.942757+0000 front 2024-09-21T02:05:31.942785+0000 (oldest deadline 2024-09-21T02:05:58.943408+0000) 2024-09-21T02:06:21.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.29:6822 osd.6 since back 2024-09-21T02:05:38.944235+0000 front 2024-09-21T02:05:38.944084+0000 (oldest deadline 2024-09-21T02:06:05.344670+0000) 2024-09-21T02:06:21.729 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:21 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11[57525]: 2024-09-21T02:06:21.364+0000 7f21d3ecf640 -1 osd.11 79 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-09-21T02:05:45.345423+0000 front 2024-09-21T02:05:45.345317+0000 (oldest deadline 2024-09-21T02:06:11.245232+0000) 2024-09-21T02:06:22.454 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:22 smithi190 podman[67624]: 2024-09-21 02:06:22.188153722 +0000 UTC m=+5.565700220 container died 5ad1a28a723a1d0097693c54f8e8aff25d14f60c538f844cb1852fee1c085e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:06:22.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:22 smithi190 podman[67624]: 2024-09-21 02:06:22.55318931 +0000 UTC m=+5.930735849 container cleanup 5ad1a28a723a1d0097693c54f8e8aff25d14f60c538f844cb1852fee1c085e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T02:06:22.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:22 smithi190 bash[67624]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11 2024-09-21T02:06:23.138 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:22 smithi190 podman[67642]: 2024-09-21 02:06:22.812444223 +0000 UTC m=+0.615842247 container remove 5ad1a28a723a1d0097693c54f8e8aff25d14f60c538f844cb1852fee1c085e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:23.407 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.138130729 +0000 UTC m=+0.094134943 container create d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T02:06:23.407 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.060002247 +0000 UTC m=+0.016006465 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:06:23.407 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.323637066 +0000 UTC m=+0.279641279 container init d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:06:23.407 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.326742754 +0000 UTC m=+0.282746975 container start d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T02:06:23.407 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.406696072 +0000 UTC m=+0.362700300 container attach d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T02:06:23.728 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:23 smithi190 podman[67728]: 2024-09-21 02:06:23.5150414 +0000 UTC m=+0.471045614 container died d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:06:24.067 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T02:06:24.090 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:24 smithi190 podman[67751]: 2024-09-21 02:06:24.051396233 +0000 UTC m=+0.530294466 container remove d63558018f3394216a7354842083e622de8fbb53b16480d27309b5eb106c0537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T02:06:24.090 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:24 smithi190 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.11.service: Deactivated successfully. 2024-09-21T02:06:24.090 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:24 smithi190 systemd[1]: Stopped Ceph osd.11 for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T02:06:24.090 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 21 02:06:24 smithi190 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@osd.11.service: Consumed 8min 50.255s CPU time. 2024-09-21T02:06:24.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:24.105 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T02:06:24.106 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T02:06:24.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.a 2024-09-21T02:06:24.424 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:24 smithi029 systemd[1]: Stopping Ceph mds.a for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:24.844 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a[70076]: 2024-09-21T02:06:24.423+0000 7fd1ef9c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:24.845 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:24 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a[70076]: 2024-09-21T02:06:24.423+0000 7fd1ef9c6640 -1 mds.a *** got signal Terminated *** 2024-09-21T02:06:27.745 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:27 smithi029 podman[111964]: 2024-09-21 02:06:27.442584655 +0000 UTC m=+3.141170796 container died b3c3b027cc2312e36c0807902d1817bfc86f2ff57eb5dc8f1f6e2c74abfa463c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:06:28.094 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:27 smithi029 podman[111964]: 2024-09-21 02:06:27.926981167 +0000 UTC m=+3.625567312 container cleanup b3c3b027cc2312e36c0807902d1817bfc86f2ff57eb5dc8f1f6e2c74abfa463c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:28.095 INFO:journalctl@ceph.mds.a.smithi029.stdout:Sep 21 02:06:27 smithi029 bash[111964]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-a 2024-09-21T02:06:28.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T02:06:28.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:28.397 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T02:06:28.398 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T02:06:28.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.d 2024-09-21T02:06:28.735 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:28 smithi029 systemd[1]: Stopping Ceph mds.d for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:29.094 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d[69040]: 2024-09-21T02:06:28.734+0000 7f0d17167640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:29.095 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:28 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d[69040]: 2024-09-21T02:06:28.734+0000 7f0d17167640 -1 mds.d *** got signal Terminated *** 2024-09-21T02:06:31.337 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:31 smithi029 podman[112085]: 2024-09-21 02:06:31.033990746 +0000 UTC m=+2.413655453 container died b60110428a8d51bde122e75bb1e62aff5f4f5a6c39fd7a04a8435c173b304165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T02:06:31.595 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:31 smithi029 podman[112085]: 2024-09-21 02:06:31.537294647 +0000 UTC m=+2.916959353 container cleanup b60110428a8d51bde122e75bb1e62aff5f4f5a6c39fd7a04a8435c173b304165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T02:06:31.595 INFO:journalctl@ceph.mds.d.smithi029.stdout:Sep 21 02:06:31 smithi029 bash[112085]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-d 2024-09-21T02:06:31.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T02:06:31.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:31.940 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T02:06:31.940 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T02:06:31.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.g 2024-09-21T02:06:32.270 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:32 smithi029 systemd[1]: Stopping Ceph mds.g for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:32.594 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: 2024-09-21T02:06:32.269+0000 7f90f6667640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:32.595 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:32 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g[69771]: 2024-09-21T02:06:32.269+0000 7f90f6667640 -1 mds.g *** got signal Terminated *** 2024-09-21T02:06:35.756 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:35 smithi029 podman[112206]: 2024-09-21 02:06:35.450897775 +0000 UTC m=+3.292612899 container died e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T02:06:36.041 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:35 smithi029 podman[112206]: 2024-09-21 02:06:35.855091724 +0000 UTC m=+3.696806847 container cleanup e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:06:36.041 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:35 smithi029 bash[112206]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g 2024-09-21T02:06:36.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T02:06:36.305 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:36 smithi029 podman[112219]: 2024-09-21 02:06:36.040928172 +0000 UTC m=+0.579909680 container remove e76e03888d3c1d30b3d92de13f36195ae7a71b8578a1a094d869d857aaa957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T02:06:36.305 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:36 smithi029 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.g.service: Deactivated successfully. 2024-09-21T02:06:36.305 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:36 smithi029 systemd[1]: Stopped Ceph mds.g for e38bc6b6-77b9-11ef-baf5-efdc52797490. 2024-09-21T02:06:36.305 INFO:journalctl@ceph.mds.g.smithi029.stdout:Sep 21 02:06:36 smithi029 systemd[1]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.g.service: Consumed 2.815s CPU time. 2024-09-21T02:06:36.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:36.316 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T02:06:36.316 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T02:06:36.316 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.j 2024-09-21T02:06:36.594 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:36 smithi029 systemd[1]: Stopping Ceph mds.j for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:37.095 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j[68654]: 2024-09-21T02:06:36.645+0000 7f84eb582640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:37.095 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:36 smithi029 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j[68654]: 2024-09-21T02:06:36.645+0000 7f84eb582640 -1 mds.j *** got signal Terminated *** 2024-09-21T02:06:39.914 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:39 smithi029 podman[112325]: 2024-09-21 02:06:39.649839531 +0000 UTC m=+3.106823209 container died 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T02:06:40.275 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:40 smithi029 podman[112325]: 2024-09-21 02:06:40.014231251 +0000 UTC m=+3.471214929 container cleanup 5a3bebcbc17c92ba08b155faccfdd8ae99e6792f3e3a3a4618c443c263ccb133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T02:06:40.275 INFO:journalctl@ceph.mds.j.smithi029.stdout:Sep 21 02:06:40 smithi029 bash[112325]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-j 2024-09-21T02:06:40.442 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T02:06:40.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:40.478 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T02:06:40.478 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T02:06:40.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.b 2024-09-21T02:06:40.796 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:40 smithi066 systemd[1]: Stopping Ceph mds.b for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:41.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T02:06:40.795+0000 7fd702e56640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:41.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:40 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b[63602]: 2024-09-21T02:06:40.795+0000 7fd702e56640 -1 mds.b *** got signal Terminated *** 2024-09-21T02:06:43.738 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:43 smithi066 podman[68775]: 2024-09-21 02:06:43.460537026 +0000 UTC m=+2.767649045 container died d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:06:44.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:43 smithi066 podman[68775]: 2024-09-21 02:06:43.837030397 +0000 UTC m=+3.144142183 container cleanup d706560110ff756773e433dfc1725a0ac49a28c49cc3768b3a5b19ed1368abd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:06:44.049 INFO:journalctl@ceph.mds.b.smithi066.stdout:Sep 21 02:06:43 smithi066 bash[68775]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-b 2024-09-21T02:06:44.278 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T02:06:44.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:44.315 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T02:06:44.315 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T02:06:44.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.e 2024-09-21T02:06:44.642 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:44 smithi066 systemd[1]: Stopping Ceph mds.e for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:45.049 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: 2024-09-21T02:06:44.640+0000 7f4d226b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:45.049 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:44 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e[63052]: 2024-09-21T02:06:44.640+0000 7f4d226b6640 -1 mds.e *** got signal Terminated *** 2024-09-21T02:06:47.716 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:47 smithi066 podman[68907]: 2024-09-21 02:06:47.462456191 +0000 UTC m=+2.913473313 container died 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T02:06:48.049 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:47 smithi066 podman[68907]: 2024-09-21 02:06:47.823946216 +0000 UTC m=+3.274963337 container cleanup 202cd76b07a0a9145aa31ac142bc9267fd6ed145bc1c8de54d6433cc020bd9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T02:06:48.049 INFO:journalctl@ceph.mds.e.smithi066.stdout:Sep 21 02:06:47 smithi066 bash[68907]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-e 2024-09-21T02:06:48.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T02:06:48.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:48.248 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T02:06:48.248 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T02:06:48.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.h 2024-09-21T02:06:48.799 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:48 smithi066 systemd[1]: Stopping Ceph mds.h for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:48.799 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h[63336]: 2024-09-21T02:06:48.568+0000 7f0706f5a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:48.799 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:48 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h[63336]: 2024-09-21T02:06:48.568+0000 7f0706f5a640 -1 mds.h *** got signal Terminated *** 2024-09-21T02:06:51.753 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:51 smithi066 podman[69029]: 2024-09-21 02:06:51.462090061 +0000 UTC m=+2.995792033 container died 7d8ba37782dd6e2d2d837083c457cc71ea0ba285e5ef2cc2c2f01ab9c9014d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T02:06:52.049 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:51 smithi066 podman[69029]: 2024-09-21 02:06:51.844327801 +0000 UTC m=+3.378029775 container cleanup 7d8ba37782dd6e2d2d837083c457cc71ea0ba285e5ef2cc2c2f01ab9c9014d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T02:06:52.049 INFO:journalctl@ceph.mds.h.smithi066.stdout:Sep 21 02:06:51 smithi066 bash[69029]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-h 2024-09-21T02:06:52.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T02:06:52.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:52.316 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T02:06:52.316 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T02:06:52.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.k 2024-09-21T02:06:52.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:52 smithi066 systemd[1]: Stopping Ceph mds.k for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:52.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: 2024-09-21T02:06:52.622+0000 7f045f7a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:52.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:52 smithi066 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k[62773]: 2024-09-21T02:06:52.622+0000 7f045f7a3640 -1 mds.k *** got signal Terminated *** 2024-09-21T02:06:55.799 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:55 smithi066 podman[69148]: 2024-09-21 02:06:55.453556128 +0000 UTC m=+2.913228027 container died e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T02:06:56.082 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:55 smithi066 podman[69148]: 2024-09-21 02:06:55.855936744 +0000 UTC m=+3.315608639 container cleanup e2af42667462f59462355dcdc3269b24fdcacf731ec0bfd7ec4bce01010edc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:56.082 INFO:journalctl@ceph.mds.k.smithi066.stdout:Sep 21 02:06:55 smithi066 bash[69148]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-k 2024-09-21T02:06:56.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T02:06:56.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:06:56.247 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T02:06:56.248 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T02:06:56.248 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.c 2024-09-21T02:06:56.552 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:56 smithi190 systemd[1]: Stopping Ceph mds.c for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:06:56.803 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: 2024-09-21T02:06:56.550+0000 7f4afdfaf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:06:56.803 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:56 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c[62207]: 2024-09-21T02:06:56.550+0000 7f4afdfaf640 -1 mds.c *** got signal Terminated *** 2024-09-21T02:06:59.727 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:59 smithi190 podman[67833]: 2024-09-21 02:06:59.440808511 +0000 UTC m=+3.003161653 container died b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:06:59.978 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:59 smithi190 podman[67833]: 2024-09-21 02:06:59.825261039 +0000 UTC m=+3.387614172 container cleanup b258dc1a7157234c2a4f4250857990ffe0d394b98385ef4478e84be691a64333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:06:59.978 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 21 02:06:59 smithi190 bash[67833]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-c 2024-09-21T02:07:00.249 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T02:07:00.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:07:00.284 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T02:07:00.284 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T02:07:00.285 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.f 2024-09-21T02:07:00.605 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:00 smithi190 systemd[1]: Stopping Ceph mds.f for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:07:00.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: 2024-09-21T02:07:00.603+0000 7fc960286640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:07:00.978 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:00 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f[61936]: 2024-09-21T02:07:00.603+0000 7fc960286640 -1 mds.f *** got signal Terminated *** 2024-09-21T02:07:03.697 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:03 smithi190 podman[67965]: 2024-09-21 02:07:03.444099528 +0000 UTC m=+2.942765114 container died 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T02:07:03.963 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:03 smithi190 podman[67965]: 2024-09-21 02:07:03.762161278 +0000 UTC m=+3.260826856 container cleanup 80ca97b51ab78866fe4cd2324abd5da140d42845ec11d9ce70f4e7695c228ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T02:07:03.963 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 21 02:07:03 smithi190 bash[67965]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-f 2024-09-21T02:07:04.112 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T02:07:04.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:07:04.145 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T02:07:04.145 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T02:07:04.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.i 2024-09-21T02:07:04.728 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:04 smithi190 systemd[1]: Stopping Ceph mds.i for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:07:04.728 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i[62483]: 2024-09-21T02:07:04.465+0000 7fb63e036640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:07:04.728 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:04 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i[62483]: 2024-09-21T02:07:04.465+0000 7fb63e036640 -1 mds.i *** got signal Terminated *** 2024-09-21T02:07:07.691 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:07 smithi190 podman[68084]: 2024-09-21 02:07:07.434037343 +0000 UTC m=+3.066349329 container died 2c1753eadbb50da3aa154400f33e1a4d8a18a7403290b09014e6a5e04ba6b522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T02:07:07.978 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:07 smithi190 podman[68084]: 2024-09-21 02:07:07.800659722 +0000 UTC m=+3.432971707 container cleanup 2c1753eadbb50da3aa154400f33e1a4d8a18a7403290b09014e6a5e04ba6b522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T02:07:07.978 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 21 02:07:07 smithi190 bash[68084]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-i 2024-09-21T02:07:08.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T02:07:08.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:07:08.252 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T02:07:08.253 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T02:07:08.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.l 2024-09-21T02:07:08.728 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:08 smithi190 systemd[1]: Stopping Ceph mds.l for e38bc6b6-77b9-11ef-baf5-efdc52797490... 2024-09-21T02:07:08.728 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l[61664]: 2024-09-21T02:07:08.560+0000 7f4160267640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:07:08.728 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:08 smithi190 ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l[61664]: 2024-09-21T02:07:08.560+0000 7f4160267640 -1 mds.l *** got signal Terminated *** 2024-09-21T02:07:11.694 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:11 smithi190 podman[68205]: 2024-09-21 02:07:11.438824135 +0000 UTC m=+2.970322194 container died d95b20e31e9db39b2fa60a3f83ab2e13ef015ec950aa5b0ba7413430452fbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:07:11.978 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:11 smithi190 podman[68205]: 2024-09-21 02:07:11.777135064 +0000 UTC m=+3.308633120 container cleanup d95b20e31e9db39b2fa60a3f83ab2e13ef015ec950aa5b0ba7413430452fbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:07:11.978 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:11 smithi190 bash[68205]: ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l 2024-09-21T02:07:11.978 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 21 02:07:11 smithi190 podman[68217]: 2024-09-21 02:07:11.93666429 +0000 UTC m=+0.496465148 container remove d95b20e31e9db39b2fa60a3f83ab2e13ef015ec950aa5b0ba7413430452fbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:07:12.143 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e38bc6b6-77b9-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T02:07:12.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:07:12.174 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T02:07:12.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:07:12.388 INFO:teuthology.orchestra.run.smithi029.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:07:15.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:07:15.283 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:07:17.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:07:18.014 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:07:20.382 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:07:20.416 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:07:20.451 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:07:20.477 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T02:07:20.479 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi029/crash 2024-09-21T02:07:20.480 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:07:20.525 INFO:teuthology.orchestra.run.smithi029.stderr:tar: /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:07:20.525 INFO:teuthology.orchestra.run.smithi029.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:07:20.527 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi066/crash 2024-09-21T02:07:20.528 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:07:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:07:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:07:20.574 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi190/crash 2024-09-21T02:07:20.575 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:07:20.607 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:07:20.607 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:07:20.608 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T02:07:20.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T02:07:20.651 INFO:teuthology.orchestra.run.smithi029.stdout:2024-09-21T01:53:38.935761+0000 mon.a (mon.0) 1203 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T02:07:20.652 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-21T02:07:20.652 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[SEC\]' /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T02:07:20.733 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]' /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T02:07:20.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[WRN\]' /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T02:07:20.889 INFO:teuthology.orchestra.run.smithi029.stdout:2024-09-21T01:53:38.935761+0000 mon.a (mon.0) 1203 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T02:07:20.890 INFO:tasks.cephadm:Compressing logs... 2024-09-21T02:07:20.890 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:07:20.932 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:07:20.935 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:07:20.956 INFO:teuthology.orchestra.run.smithi029.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T02:07:20.956 INFO:teuthology.orchestra.run.smithi029.stderr:: No such file or directory 2024-09-21T02:07:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T02:07:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:07:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:07:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T02:07:20.959 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:07:20.959 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T02:07:20.960 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T02:07:20.960 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log: /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 82.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:07:20.960 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T02:07:20.960 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.g.log: -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:07:20.962 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:07:20.962 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T02:07:20.965 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr: 82.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr: 90.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:07:20.966 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T02:07:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log: 87.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:07:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.10.log: No such file or directory 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.b.log: /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:07:20.970 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:07:20.972 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.l.log: 87.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:07:20.972 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T02:07:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:07:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T02:07:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:07:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.x.log: 95.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T02:07:20.976 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T02:07:20.980 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:07:20.981 INFO:teuthology.orchestra.run.smithi066.stderr: 92.8% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T02:07:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T02:07:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T02:07:20.984 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.z.log: 92.9% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T02:07:20.984 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T02:07:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:07:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T02:07:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:07:20.988 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T02:07:20.993 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:07:20.996 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:07:21.001 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:07:21.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:07:21.022 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:07:21.119 INFO:teuthology.orchestra.run.smithi029.stderr: 93.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T02:07:21.180 INFO:teuthology.orchestra.run.smithi029.stderr: 93.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T02:07:21.226 INFO:teuthology.orchestra.run.smithi066.stderr: 94.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T02:07:21.252 INFO:teuthology.orchestra.run.smithi029.stderr: 90.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T02:07:21.281 INFO:teuthology.orchestra.run.smithi190.stderr: 93.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T02:07:21.519 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T02:07:21.548 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T02:07:21.562 INFO:teuthology.orchestra.run.smithi066.stderr: 91.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T02:07:22.132 INFO:teuthology.orchestra.run.smithi029.stderr: 90.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T02:07:23.506 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T02:07:24.265 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T02:07:25.113 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T02:07:25.113 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T02:07:27.062 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T02:07:35.276 INFO:teuthology.orchestra.run.smithi190.stderr: 94.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T02:07:36.175 INFO:teuthology.orchestra.run.smithi066.stderr: 94.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T02:07:38.035 INFO:teuthology.orchestra.run.smithi029.stderr: 94.1% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T02:07:38.565 INFO:teuthology.orchestra.run.smithi190.stderr: 94.3% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T02:07:40.478 INFO:teuthology.orchestra.run.smithi029.stderr: 94.1% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T02:07:41.019 INFO:teuthology.orchestra.run.smithi066.stderr: 93.8% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T02:07:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 94.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T02:07:42.318 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T02:07:42.384 INFO:teuthology.orchestra.run.smithi190.stderr: 94.4% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T02:07:42.386 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-09-21T02:07:42.386 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m21.436s 2024-09-21T02:07:42.386 INFO:teuthology.orchestra.run.smithi190.stderr:user 1m18.638s 2024-09-21T02:07:42.386 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m2.920s 2024-09-21T02:07:44.982 INFO:teuthology.orchestra.run.smithi029.stderr: 94.1% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T02:07:48.936 INFO:teuthology.orchestra.run.smithi029.stderr: 94.2% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T02:07:48.938 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-21T02:07:48.938 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m27.992s 2024-09-21T02:07:48.938 INFO:teuthology.orchestra.run.smithi029.stderr:user 1m27.217s 2024-09-21T02:07:48.938 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m3.053s 2024-09-21T02:07:51.391 INFO:teuthology.orchestra.run.smithi066.stderr: 94.0% -- replaced with /var/log/ceph/e38bc6b6-77b9-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T02:07:51.393 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-21T02:07:51.393 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m30.442s 2024-09-21T02:07:51.393 INFO:teuthology.orchestra.run.smithi066.stderr:user 1m38.527s 2024-09-21T02:07:51.393 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m3.129s 2024-09-21T02:07:51.393 INFO:tasks.cephadm:Archiving logs... 2024-09-21T02:07:51.394 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi029/log 2024-09-21T02:07:51.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:08:03.082 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi066/log 2024-09-21T02:08:03.083 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:08:15.070 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi190/log 2024-09-21T02:08:15.071 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:08:25.367 INFO:tasks.cephadm:Removing cluster... 2024-09-21T02:08:25.368 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force 2024-09-21T02:08:25.581 INFO:teuthology.orchestra.run.smithi029.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:08:26.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force 2024-09-21T02:08:26.240 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:08:26.634 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e38bc6b6-77b9-11ef-baf5-efdc52797490 --force 2024-09-21T02:08:26.835 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: e38bc6b6-77b9-11ef-baf5-efdc52797490 2024-09-21T02:08:27.189 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T02:08:27.189 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:08:27.209 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:08:27.226 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:08:27.242 INFO:tasks.cephadm:Teardown complete 2024-09-21T02:08:27.242 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T02:08:27.254 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T02:08:27.254 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:08:27.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:08:27.268 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:08:27.369 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T02:08:27.379 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T02:08:27.379 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:08:27.382 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:08:27.384 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:08:27.396 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:08:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:08:27.398 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:08:27.400 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:08:27.400 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-21T02:08:27.401 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +77us[ +95us] +/- 65ms 2024-09-21T02:08:27.401 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -701us[ -701us] +/- 59ms 2024-09-21T02:08:27.401 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +836us[ +836us] +/- 70ms 2024-09-21T02:08:27.401 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:08:27.402 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:08:27.403 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-09-21T02:08:27.403 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 +81us[ +104us] +/- 65ms 2024-09-21T02:08:27.403 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -692us[ -692us] +/- 59ms 2024-09-21T02:08:27.403 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +851us[ +851us] +/- 70ms 2024-09-21T02:08:27.403 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +94us[ +103us] +/- 65ms 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -684us[ -684us] +/- 59ms 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +857us[ +857us] +/- 70ms 2024-09-21T02:08:27.404 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:08:27.405 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T02:08:27.415 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T02:08:27.415 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T02:08:27.426 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:08:27.530 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:08:27.597 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:08:27.663 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:08:27.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:27.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:08:27.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:27.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:08:27.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:27.751 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T02:08:27.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T02:08:27.796 INFO:teuthology.task.internal:Duration was 2305.693627 seconds 2024-09-21T02:08:27.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T02:08:27.806 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T02:08:27.807 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:08:27.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:08:27.811 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:08:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:08:27.850 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:08:27.850 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:08:28.348 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T02:08:28.348 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-09-21T02:08:28.349 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:08:28.373 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-09-21T02:08:28.374 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:08:28.416 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-09-21T02:08:28.417 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:08:28.447 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T02:08:28.447 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:08:28.450 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:08:28.459 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:08:28.515 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T02:08:28.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:08:28.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:08:28.526 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:08:29.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T02:08:30.062 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T02:08:30.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:08:30.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:08:30.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:08:30.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T02:08:30.279 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.290 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.310 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-09-21T02:08:30.317 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-09-21T02:08:30.318 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-09-21T02:08:30.347 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:30.385 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:30.407 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:08:30.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:30.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T02:08:30.450 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T02:08:30.457 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi029 2024-09-21T02:08:30.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:08:30.626 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi066 2024-09-21T02:08:30.626 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:08:31.102 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913386/remote/smithi190 2024-09-21T02:08:31.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:08:31.521 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T02:08:31.522 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:08:31.526 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:08:31.532 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:08:31.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T02:08:31.665 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T02:08:31.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T02:08:31.737 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T02:08:31.738 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:08:31.743 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:08:31.748 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:08:31.760 INFO:teuthology.orchestra.run.smithi029.stdout: 265805 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:08 /home/ubuntu/cephtest 2024-09-21T02:08:31.765 INFO:teuthology.orchestra.run.smithi066.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:08 /home/ubuntu/cephtest 2024-09-21T02:08:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:08 /home/ubuntu/cephtest 2024-09-21T02:08:31.772 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T02:08:31.786 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T02:08:31.834 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 2305.6936271190643 failure_reason: '"2024-09-21T01:53:38.935761+0000 mon.a (mon.0) 1203 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-09-21T02:08:31.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:08:31.933 INFO:teuthology.run:FAIL